[j-nsp] out-lynxed

[Carl Hayter]

On Tue, Sep 14, 2004 at 01:11:51PM -1000, Randy Bush wrote:
> often, i operate remotely, and hence can not do a lot of tricks
> using datacenter, i.e. well-connected, hosts.
> 
>   o some router i want to upgrade is in a remote pop
> 
>   o i am personally on a dsl line (read "slow") a few thousand
>     kilometers away on hawaii
> 
>   o so i don't want to use galeon/mozilla to download the image
>     to my desk/laptop to then push it back over the same dsl
>     line to the router
> 
>   o there are perfectly nice headless freebsd boxes right next
>     to the router, with gige to multiple stm-Xs to the net.
> 
>   o so i run lynx on one of those boxes to fetch the software
>     from your site
> 
> except i can no longer do this.  and, in case 2004-0912-0025,
> jtac tells me that "IT says we do not support Lynx. All our
> servers use standard coded browsers and that is all they will
> support now. Sorry for the inconvenience."
> 
> how do others work in this kind of situation?  any trix?

WWW::Mechanize...

this *used* to work, haven't tried lately and don't have WWW::Mechanize
on this machine to test.  downloads the code and documentation.

#!/usr/bin/perl
use strict;
use warnings;
$|++;

my $user = '*username-here*';
my $pass = '*password-here*';

my $ver = '6.0';       # lazy version matching
my $ext = 'tgz';       # pdf|tgz documentation

use WWW::Mechanize;
use WWW::Mechanize::FormFiller;
use URI::URL;

my $agent = WWW::Mechanize->new();
my $formfiller = WWW::Mechanize::FormFiller->new();

$agent->env_proxy();

$agent->agent('Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)');

$agent->get('https://www.juniper.net/support/');

$agent->form(2);
$formfiller->add_filler( 'USER' => Fixed => $user );
$formfiller->add_filler( 'PASSWORD' => Fixed => $pass );
$formfiller->fill_form($agent->current_form);
$agent->submit();

$agent->follow(qr(Download software));
$agent->follow(qr(Encrypted));

my @links;
my @all_links = $agent->links();

# code link has the version number and //download.juniper.net in it

my @targets = grep {
    m!\Q$ver! and m!\Q//download.juniper.net/!
    } map {$_->[0]} @all_links;

$agent->follow(qr(Software Documentation: Release \Q$ver\E));
@all_links = $agent->links();

# get the index, the tgz or pdf files and the release notes

push @targets, grep {
    m!\Q-comprehensive-index.pdf! or
    ( m!/download/! and m!$ext$! ) or
    m!/download/rn-sw-\d+\.pdf!
    } map {$_->[0]} @all_links;

# uncomment line below for testing
#  print join $/, @targets, ''; exit;

my $base = $agent->uri;
for my $target (@targets) {
	my $url = URI::URL->new($target,$base);
	$target = $url->path;
	$target =~ s!^(.*/)?([^/]+)$!$2!;
	$url = $url->abs;
	print "mirroring $target$/";
	$agent->mirror($url,$target);
};

----
Carl Hayter
ISD - Data Network Operations
University of Southern California
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
Url : https://puck.nether.net/pipermail/juniper-nsp/attachments/20040914/0b3e1f02/attachment.bin