[cisco-voip] Differences between DRS and CDR SFTP activities

Thu Mar 14 11:58:48 EDT 2013

The system has been configured to disallow SSH access, but still allow SFTP. I believe this was done by removing the default shell so it hangs at login. Pretty sure this isn't the problem since it works when the system is mounted with the other permissions. 

That being said, I tested what you said using WinSCP with the custom option setting the 0666 as the permissions. It failed on the one mount where the userID is not the owner, but it succeeded on the mount where the userID is the owner. 

I will bring this back to my sys admin team to see if there is something he can do to alleviate this. 

--- 
Lelio Fulgenzi, B.A. 
Senior Analyst (CCS) * University of Guelph * Guelph, Ontario N1G 2W1 
(519) 824-4120 x56354 (519) 767-1060 FAX (ANNU) 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
Cooking with unix is easy. You just sed it and forget it. 
- LFJ (with apologies to Mr. Popeil) 

----- Original Message -----
From: "Wes Sisk" <wsisk at cisco.com> 
To: "Lelio Fulgenzi" <lelio at uoguelph.ca> 
Cc: "Ryan Ratliff" <rratliff at cisco.com>, "voip puck" <cisco-voip at puck.nether.net> 
Sent: Thursday, March 14, 2013 11:46:10 AM 
Subject: Re: [cisco-voip] Differences between DRS and CDR SFTP activities 

Interesting, the code says to chmod ONLY IF the cdrrep is attempting the transfer. 

If you ssh to the destination server and then attempt to chmod 666 then what happens? 

Another interesting possibility is CSCuc22414. I wonder if a hung thread could be holding open a file with the same name such that attempting to transfer a file with the same name gives an error. This is a bit of a stretch, the first issue of chmod seems the more likely problem. Can you check? 

-wes 

On Mar 13, 2013, at 4:42 PM, Lelio Fulgenzi < lelio at uoguelph.ca > wrote: 

So I had a few moments to kick up the traces to debug and also setup a mounted directory with the same user/group permissions from a non-NetApp device. I'm getting the same errors, nothing much more to add unfortunately. I honestly think this is a bug. The owner/group of the target directory should make no difference as long as the userID being used has write access. 

In the meantime, we've mounted the same file system twice, once using the owner/group I want to use in the long run, and once matching it with the userID configured in the SFTP operation. 

===== 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:147) - ftpClient constructor 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:149) - host = <snip> 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:150) - user = <snip> 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:151) - passwd = <snip> 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:152) - localDir = null 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:153) - remoteDir = null 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:154) - secureFTP = true 
2013-03-13 16:19:32,605 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:155) - permKey = true 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:156) - Component Name : cdrrep 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:178) - connect()--> 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:181) - connect(): connect for SFTP 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:89) - Inside the SFTPConnection Constructor 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:233) - checkConnection--> 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:237) - checkConnection(): username : <snip> 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:238) - checkConnection(): password : <snip> 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:239) - checkConnection(): hostname : <snip> 
2013-03-13 16:19:32,606 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:152) - getSshClient--> 
2013-03-13 16:19:32,607 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:209) - getSshConnector--> 
2013-03-13 16:19:32,608 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:164) - SSH Connection object obtained 
2013-03-13 16:19:32,678 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:246) - The SshClient is Initialized 
2013-03-13 16:19:32,690 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:264) - The User is Authenticated. Need to get the SftpClient 
2013-03-13 16:19:32,700 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:268) - SftpClient is obtained Successfully 
2013-03-13 16:19:32,700 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:288) - <--checkConnection 
2013-03-13 16:19:32,700 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:199) - The status of checkConnection : true 
2013-03-13 16:19:32,700 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:216) - connect():Successfully connect to <snip> 
2013-03-13 16:19:32,700 INFO [Thread-11] cdrrep.FtpManager (FtpManager.java:78) - FtpManager constructor 2: Establish server connection to host [<snip>] successfully! 
2013-03-13 16:19:32,701 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:198) - 32 folders found under /var/log/active/cm/cdr_repository/destination3 
2013-03-13 16:19:32,703 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:217) - 15 files found under /var/log/active/cm/cdr_repository/destination3/20130313 
2013-03-13 16:19:32,703 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:272) - checking folder /var/log/active/cm/cdr_repository/destination3/20130313 
2013-03-13 16:19:32,704 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:301) - Processing link: /var/log/active/cm/cdr_repository/destination3/20130313/cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:32,704 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:340) - cdr_iptccm105-Cluster_07_201303132013_63263 size is 5993 
2013-03-13 16:19:32,704 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:342) - cdr file to be sent is /var/log/active/cm/cdr_repository/preserve/20130313/cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:32,704 DEBUG [Thread-11] cdrrep.Util (Util.java:102) - checkJVMFreeMemory(): --> 
2013-03-13 16:19:32,704 DEBUG [Thread-11] cdrrep.Util (Util.java:158) - checkJVMFreeMemory(): <-- 
2013-03-13 16:19:32,704 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:345) - CDRREP before sending the file: JVM status: [Max = 227.56MB, Total = 46.62MB, Free = 36.28MB (77.81%), Used = 10.35MB (22.19%)] 
2013-03-13 16:19:33,706 DEBUG [Thread-11] cdrrep.FtpManager (FtpManager.java:178) - sendFile(): --> 
2013-03-13 16:19:33,706 DEBUG [Thread-11] cdrrep.FtpManager (FtpManager.java:184) - FtpManager 2 got file: cdr_iptccm105-Cluster_07_201303132013_63263 to send 
2013-03-13 16:19:33,707 DEBUG [Thread-11] cdrrep.FtpManager (FtpManager.java:197) - sendFile(): Try bldSendFileCmds 
2013-03-13 16:19:33,707 DEBUG [Thread-11] cdrrep.FtpManager (FtpManager.java:198) - FtpManager - sendFile() : The remote directory to send the files : /data/ipticnc/incoming/misc/cdrtmp/ 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:783) - bldSendFileCmds(): --> 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:796) - bldSendFileCmds(): - no need to build command file 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:797) - local directory: /var/log/active/cm/cdr_repository/preserve/20130313/ 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:798) - remote directory: /data/ipticnc/incoming/misc/cdrtmp/ 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:799) - final remote directory: null 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:801) - file: cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:33,707 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:803) - bldSendFileCmds(): return <-- 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:433) - sendFiles(): --> 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:444) - sendFiles(): file to send = /var/log/active/cm/cdr_repository/preserve/20130313/cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:445) - sendFiles(): remoteDirectory = /data/ipticnc/incoming/misc/cdrtmp/ 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:455) - sendFiles(): localFilePath = /var/log/active/cm/cdr_repository/preserve/20130313/cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:456) - sendFiles(): remoteFilePath = /data/ipticnc/incoming/misc/cdrtmp/cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:460) - sendFiles(): uploadFile 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:100) - sendFileThroughSFTP --> 
2013-03-13 16:19:33,708 DEBUG [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:111) - Running put command to copy the file 
2013-03-13 16:19:33,728 ERROR [Thread-11] sftpapi.SFTPConnection (SFTPConnection.java:131) - Sftp Status Exception : com.maverick.sftp.SftpStatusException: Permission denied 
2013-03-13 16:19:33,728 DEBUG [Thread-11] sftpapi.ftpClient (ftpClient.java:472) - Sftp Status Exception 
2013-03-13 16:19:33,728 ERROR [Thread-11] sftpapi.ftpClient (ftpClient.java:473) - SFTP error in transferring file to another machine. 
2013-03-13 16:19:33,728 ERROR [Thread-11] sftpapi.ftpClient (ftpClient.java:474) - com.maverick.sftp.SftpStatusException: Permission denied 
2013-03-13 16:19:33,728 ERROR [Thread-11] sftpapi.ftpClient (ftpClient.java:493) - Exception while sending files: com.maverick.sftp.SftpStatusException: Permission denied 
2013-03-13 16:19:33,728 ERROR [Thread-11] cdrrep.FtpManager (FtpManager.java:223) - SFTP/FTP failed: cdr_iptccm105-Cluster_07_201303132013_63263 to <snip> 
2013-03-13 16:19:33,729 ERROR [Thread-11] cdrrep.LogConfig (LogConfig.java:157) - Java Exception happened in my CDRSender, with Exception: 
java.lang.Exception: SFTP/FTP failed: cdr_iptccm105-Cluster_07_201303132013_63263 to <snip> 
at com.cisco.ccm.cdrdlv.cdrrep.FtpManager.sendFile(FtpManager.java:225) 
at com.cisco.ccm.cdrdlv.cdrrep.CDRSender.run(CDRSender.java:349) 

2013-03-13 16:19:33,729 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:364) - sendSuccess=false, numOfFailure=13 
2013-03-13 16:19:33,729 ERROR [Thread-11] cdrrep.CDRSender (CDRSender.java:365) - java.lang.Exception: SFTP/FTP failed: cdr_iptccm105-Cluster_07_201303132013_63263 to <snip> 
2013-03-13 16:19:33,729 ERROR [Thread-11] cdrrep.CDRSender (CDRSender.java:366) - Unable to successfully send file to outside billing server <snip> 
2013-03-13 16:19:33,729 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:389) - More than two times of failure, raise alarm 
2013-03-13 16:19:33,729 INFO [Thread-11] alarmutil.AlarmSender (AlarmSender.java:101) - alarmName: CDRFileDeliveryFailureContinues 
2013-03-13 16:19:33,729 DEBUG [Thread-11] alarmutil.AlarmSender (AlarmSender.java:107) - paramName[0] = BillingServerAddress -> paramValue[0] = <snip> 
2013-03-13 16:19:33,729 DEBUG [Thread-11] alarmutil.AlarmSender (AlarmSender.java:113) - sendAlarm ... 
2013-03-13 16:19:33,729 DEBUG [Thread-11] cdrrep.CDRSender (CDRSender.java:402) - failed sending /var/log/active/cm/cdr_repository/preserve/20130313/cdr_iptccm105-Cluster_07_201303132013_63263 
2013-03-13 16:19:33,730 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:403) - resendonfailure flag valuetrue 
2013-03-13 16:19:33,730 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:406) - resendonfailure flag valuetrue 
2013-03-13 16:19:33,730 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:407) - host value <snip> 

===== 

--- 
Lelio Fulgenzi, B.A. 
Senior Analyst (CCS) * University of Guelph * Guelph, Ontario N1G 2W1 
(519) 824-4120 x56354 (519) 767-1060 FAX (ANNU) 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
Cooking with unix is easy. You just sed it and forget it. 
- LFJ (with apologies to Mr. Popeil) 

----- Original Message -----
From: "Ryan Ratliff" < rratliff at cisco.com > 
To: "Lelio Fulgenzi" < lelio at uoguelph.ca > 
Cc: "Erick B." < erickbee at gmail.com >, "voip puck" < cisco-voip at puck.nether.net > 
Sent: Friday, February 22, 2013 4:46:26 PM 
Subject: Re: [cisco-voip] Differences between DRS and CDR SFTP activities 

What I would hope to see is the exact sftp operation that was being done, but only in hopes that it would give a clue as to what permissions issue we are getting hit by. 

-Ryan 

On Feb 22, 2013, at 4:28 PM, Lelio Fulgenzi < lelio at uoguelph.ca > wrote: 

not sure what, if any, logs we're going to get from the NetApp. I checked the configuration of the traces and out of Fatal, Error, Warn, Info, Debug; I'm on Info. 

I'm sure Debug will give us more data, but will it be useful? 

--- 
Lelio Fulgenzi, B.A. 
Senior Analyst (CCS) * University of Guelph * Guelph, Ontario N1G 2W1 
(519) 824-4120 x56354 (519) 767-1060 FAX (ANNU) 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
Cooking with unix is easy. You just sed it and forget it. 
- LFJ (with apologies to Mr. Popeil) 

----- Original Message -----
From: "Ryan Ratliff" < rratliff at cisco.com > 
To: "Lelio Fulgenzi" < lelio at uoguelph.ca > 
Cc: "Erick B." < erickbee at gmail.com >, "voip puck" < cisco-voip at puck.nether.net > 
Sent: Friday, February 22, 2013 3:14:04 PM 
Subject: Re: [cisco-voip] Differences between DRS and CDR SFTP activities 

2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.SFTPConnection (SFTPConnection.java:131) - Sftp Status Exception : com.maverick.sftp.SftpStatusException: Permission denied 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.ftpClient (ftpClient.java:473) - SFTP error in transferring file to another machine. 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.ftpClient (ftpClient.java:474) - com.maverick.sftp.SftpStatusException: Permission denied 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.ftpClient (ftpClient.java:493) - Exception while sending files: com.maverick.sftp.SftpStatusException: Permission denied 
2013-02-21 14:02:39,903 ERROR [Thread-10] cdrrep.FtpManager (FtpManager.java:223) - SFTP/FTP failed: cmr_iptccm105-Cluster_04_201302211900_88292 to blackjack.cs.uoguelph.ca 

Hard to see exactly what it's doing since I guess these aren't on detailed level. Any logs from the NetApp to see what it thinks is going on? 

-Ryan 

On Feb 22, 2013, at 2:21 PM, Lelio Fulgenzi < lelio at uoguelph.ca > wrote: 

Well, coincidentally enough, we're using a NetApp. :( 

The log entries from the CDR Repository Manager follow. The CDR Agent just has wakeup statements. 

We currently have two of billing servers configured, but during troubleshooting, we had to delete/recreate, and I think during this time, it was still trying to clean up the deleted one which is why you see the three destinations. 

sending to hiroller succeeds and to blackjack it fails. hiroller is a linux box as well, but it is CIFS mounting a windows server filesystem. 

===== 
file:cdrrepmgr00094.log 

2013-02-21 14:02:29,578 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 3 
2013-02-21 14:02:30,271 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 1 
2013-02-21 14:02:33,194 INFO [Thread-8] cdrrep.FileMonitor (FileMonitor.java:132) - 2 file(s) found under /var/log/active/cm/cdr_repository/tmp 
2013-02-21 14:02:33,196 INFO [Thread-8] cdrrep.FileMonitor (FileMonitor.java:274) - Processed file: /var/log/active/cm/cdr_repository/tmp/cmr_iptccm105-Cluster_07_201302211902_60910 
2013-02-21 14:02:33,197 INFO [Thread-8] cdrrep.FileMonitor (FileMonitor.java:274) - Processed file: /var/log/active/cm/cdr_repository/tmp/cdr_iptccm105-Cluster_07_201302211902_60910 
2013-02-21 14:02:35,582 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 3 
2013-02-21 14:02:36,274 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 1 
2013-02-21 14:02:36,275 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:345) - CDRREP before sending the file: JVM status: [Max = 227.56MB, Total = 46.44MB, Free = 13.68MB (29.46%), Used = 32.76MB (70.54%)] 
2013-02-21 14:02:36,705 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 2 
2013-02-21 14:02:36,705 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:166) - There was file delivery failure for destination 2 in last round, re-connect server now! 
2013-02-21 14:02:37,302 INFO [Thread-9] cdrrep.FtpManager (FtpManager.java:227) - FTP success: cmr_iptccm105-Cluster_07_201302211902_60910 to hiroller.cs.uoguelph.ca 
2013-02-21 14:02:37,302 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:352) - CDRREP after sending the file: JVM status: [Max = 227.56MB, Total = 46.44MB, Free = 13.47MB (29.01%), Used = 32.97MB (70.99%)] 
2013-02-21 14:02:38,788 INFO [Thread-10] cdrrep.FtpManager (FtpManager.java:78) - FtpManager constructor 1: Establish server connection to host [ blackjack.cs.uoguelph.ca ] successfully! 
2013-02-21 14:02:38,789 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:345) - CDRREP before sending the file: JVM status: [Max = 227.56MB, Total = 46.00MB, Free = 15.44MB (33.56%), Used = 30.56MB (66.44%)] 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.SFTPConnection (SFTPConnection.java:131) - Sftp Status Exception : com.maverick.sftp.SftpStatusException: Permission denied 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.ftpClient (ftpClient.java:473) - SFTP error in transferring file to another machine. 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.ftpClient (ftpClient.java:474) - com.maverick.sftp.SftpStatusException: Permission denied 
2013-02-21 14:02:39,903 ERROR [Thread-10] sftpapi.ftpClient (ftpClient.java:493) - Exception while sending files: com.maverick.sftp.SftpStatusException: Permission denied 
2013-02-21 14:02:39,903 ERROR [Thread-10] cdrrep.FtpManager (FtpManager.java:223) - SFTP/FTP failed: cmr_iptccm105-Cluster_04_201302211900_88292 to blackjack.cs.uoguelph.ca 
2013-02-21 14:02:39,904 ERROR [Thread-10] cdrrep.LogConfig (LogConfig.java:157) - Java Exception happened in my CDRSender, with Exception: 
java.lang.Exception: SFTP/FTP failed: cmr_iptccm105-Cluster_04_201302211900_88292 to blackjack.cs.uoguelph.ca 
at com.cisco.ccm.cdrdlv.cdrrep.FtpManager.sendFile(FtpManager.java:225) 
at com.cisco.ccm.cdrdlv.cdrrep.CDRSender.run(CDRSender.java:349) 

2013-02-21 14:02:39,904 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:364) - sendSuccess=false, numOfFailure=5 
2013-02-21 14:02:39,904 ERROR [Thread-10] cdrrep.CDRSender (CDRSender.java:365) - java.lang.Exception: SFTP/FTP failed: cmr_iptccm105-Cluster_04_201302211900_88292 to blackjack.cs.uoguelph.ca 
2013-02-21 14:02:39,904 ERROR [Thread-10] cdrrep.CDRSender (CDRSender.java:366) - Unable to successfully send file to outside billing server blackjack.cs.uoguelph.ca 
2013-02-21 14:02:39,904 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:389) - More than two times of failure, raise alarm 
2013-02-21 14:02:39,904 INFO [Thread-10] alarmutil.AlarmSender (AlarmSender.java:101) - alarmName: CDRFileDeliveryFailureContinues 
2013-02-21 14:02:39,904 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:403) - resendonfailure flag valuetrue 
2013-02-21 14:02:39,904 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:406) - resendonfailure flag valuetrue 
2013-02-21 14:02:39,905 INFO [Thread-10] cdrrep.CDRSender (CDRSender.java:407) - host value blackjack.cs.uoguelph.ca 
2013-02-21 14:02:41,304 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:345) - CDRREP before sending the file: JVM status: [Max = 227.56MB, Total = 46.00MB, Free = 14.84MB (32.27%), Used = 31.16MB (67.73%)] 
2013-02-21 14:02:41,585 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 3 
2013-02-21 14:02:42,367 INFO [Thread-9] cdrrep.FtpManager (FtpManager.java:227) - FTP success: cdr_iptccm105-Cluster_07_201302211902_60910 to hiroller.cs.uoguelph.ca 
2013-02-21 14:02:42,367 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:352) - CDRREP after sending the file: JVM status: [Max = 227.56MB, Total = 46.00MB, Free = 14.61MB (31.77%), Used = 31.39MB (68.23%)] 
2013-02-21 14:02:47,588 INFO [Thread-11] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 3 
2013-02-21 14:02:48,370 INFO [Thread-9] cdrrep.CDRSender (CDRSender.java:105) - CDRSender wakes up for destination 1 

===== 

--- 
Lelio Fulgenzi, B.A. 
Senior Analyst (CCS) * University of Guelph * Guelph, Ontario N1G 2W1 
(519) 824-4120 x56354 (519) 767-1060 FAX (ANNU) 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
Cooking with unix is easy. You just sed it and forget it. 
- LFJ (with apologies to Mr. Popeil) 

----- Original Message -----
From: "Erick B." < erickbee at gmail.com > 
To: "Lelio Fulgenzi" < lelio at uoguelph.ca > 
Cc: "Ryan Ratliff" < rratliff at cisco.com >, "voip puck" < cisco-voip at puck.nether.net > 
Sent: Friday, February 22, 2013 2:14:56 PM 
Subject: Re: [cisco-voip] Differences between DRS and CDR SFTP activities 

I have noticed a difference to with the DRS and CDR SFTP jobs, going to a NetApp. DRS worked fine but CDR billing job had issues with NetApp and moved CDR to use Windows server with different SFTP server. Something on my to-do list of things to figure out one of these days... 

On Fri, Feb 22, 2013 at 1:06 PM, Lelio Fulgenzi < lelio at uoguelph.ca > wrote: 

hmmm, never thought about collecting those. i'll go look right now. but here's what I see from the syslog host we're sending events to: 

Feb 21 13:24:09 iptccm105.cs.uoguelph.ca 2272: : : 697: Feb 21 18:24:09.952 UTC : %CCM_CDR_REP-CDRREP-3-CDRFileDeliveryFailureContinues: (s)FTP delivery of CDR files failed on retries. BillingServerAddress: blackjack.cs.uoguelph.ca App ID:Cisco CDR Repository Manager Cluster ID: Node ID:iptccm105 

--- 
Lelio Fulgenzi, B.A. 
Senior Analyst (CCS) * University of Guelph * Guelph, Ontario N1G 2W1 
(519) 824-4120 x56354 (519) 767-1060 FAX (ANNU) 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
Cooking with unix is easy. You just sed it and forget it. 
- LFJ (with apologies to Mr. Popeil) 

From: "Ryan Ratliff" < rratliff at cisco.com > 
To: "Lelio Fulgenzi" < lelio at uoguelph.ca > 
Cc: cisco-voip at puck.nether.net 
Sent: Friday, February 22, 2013 2:01:13 PM 
Subject: Re: [cisco-voip] Differences between DRS and CDR SFTP activities 

What do the logs for CDR Repository Manager (or CDR Agent, I don't recall offhand which one does the sftp) say about why it's failing? The only immediate difference I can think of is the nature of the files being written though that doesn't really explain why filesystem ownership has an impact. 

-Ryan 

On Feb 22, 2013, at 1:42 PM, Lelio Fulgenzi < lelio at uoguelph.ca > wrote: 

So, we've set up an SFTP server on a linux box and setup a (mounted) filesystem where a user is not the owner, but part of the group that has rw access. WinSCP works fine, user is able to write files. DRS is fine, backups work fine. CDR billing server does not work, transfers fail with partially written files and then others don't go through at all. 

The only way we got CDR transfers to work, was when use used a local filesystem where the user was the owner used a user owned the mounted filesystem it worked. Again, DRS works fine without any of these issues. 

Can anyone speak to the nuances between the DRS SFTP process and the CDR SFTP process and why one would work and one would not? 

--- 
Lelio Fulgenzi, B.A. 
Senior Analyst (CCS) * University of Guelph * Guelph, Ontario N1G 2W1 
(519) 824-4120 x56354 (519) 767-1060 FAX (ANNU) 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
Cooking with unix is easy. You just sed it and forget it. 
- LFJ (with apologies to Mr. Popeil) 

_______________________________________________ 
cisco-voip mailing list 
cisco-voip at puck.nether.net 
https://puck.nether.net/mailman/listinfo/cisco-voip 

_______________________________________________ 
cisco-voip mailing list 
cisco-voip at puck.nether.net 
https://puck.nether.net/mailman/listinfo/cisco-voip 
_______________________________________________ 
cisco-voip mailing list 
cisco-voip at puck.nether.net 
https://puck.nether.net/mailman/listinfo/cisco-voip 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20130314/05356ed5/attachment.html>