From rocky@panix.com  Sun Oct 13 11:35:52 1996
Return-Path: rocky@panix.com
Received: from panix.com (panix.com [198.7.0.2]) by mailhost.pfmc.net (8.7.6/8.6.12) with ESMTP id LAA19899 for <domain-faq@pfmc.net>; Sun, 13 Oct 1996 11:35:51 -0400 (EDT)
Received: (from rocky@localhost) by panix.com (8.7.6/8.7/PanixU1.3) id LAA09072; Sun, 13 Oct 1996 11:35:51 -0400 (EDT)
Date: Sun, 13 Oct 1996 11:35:51 -0400 (EDT)
From: "R. Bernstein" <rocky@panix.com>
Message-Id: <199610131535.LAA09072@panix.com>
To: domain-faq@pfmc.net
Subject: revising 1.17 (Obtaining the latest cache file)
Return-Receipt-to: rocky@panix.com
Reply-to: rocky@panix.com
X-Status: 
Status: OR

Some small caveat about this 1.17, "Obtaining the latest cache file".
The anonymous FTP to update this and the dig work if in fact you are
on the Internet and your root servers are in fact the "real" ones.

Many companies run internal root servers so that the Internet root
servers are not directly accessible except via a firewall. In this case,
there are generally fake root servers and thus FTPing Internet
root server information is probably not going to be of much use.

The second method suggested
      dig @a.root-servers.net. . ns

also might not be of use either unless the internal root servers are also
called a.root-servers.net. Of course substituting with the right name
for a.root-servers.net would work. 

So might  "dnsquery -tns ." but this might not return authoritative
answers. 

If the task of periodically obtaining the latest cache file is to be
automated periodically (and you want to do a very thorough job), you
might have to deal with the fact that a.root-servers.net (or some
other fake root server) might not be available. For example a
transient network outage. So one might want to loop over a list of
root servers.

One good guess of what these would be can be in the existing root
cache. Or for Internet root servers:
a.root-servers.net b.root-servers.net, etc.

I had posted to comp.protocols.tcp-ip.domains a Perl program that attempts
to deal with both of these problems.

From rocky@panix.com Tue Nov  5 22:28:42 EST 1996
Article: 10161 of comp.protocols.tcp-ip.domains
Path: news.pfmc.net!news.sprintlink.net!news-pen-4.sprintlink.net!snunews.snu.ac.kr!news.kaist.ac.kr!news.kreonet.re.kr!newsfeed.dacom.co.kr!arclight.uoregon.edu!nntp.primenet.com!feed1.news.erols.com!panix!news.panix.com!news.panix.com!rocky
From: rocky@panix.com (R. Bernstein)
Newsgroups: comp.protocols.tcp-ip.domains
Subject: Re: Keeping db.cache current (program included)
Date: 10 Oct 1996 19:12:52 GMT
Organization: PANIX Public Access Unix, NYC
Lines: 320
Distribution: inet
Message-ID: <ROCKY.96Oct10151252@panix.panix.com>
References: <g4k9t5vbjv.fsf@buster.peccary.sty> <53bl0q$lqn@news.next.com>
	<ROCKY.96Oct8010711@panix.panix.com> <53dppk$n7r@somnet.sandia.gov>
NNTP-Posting-Host: panix.com
In-reply-to: wllarso@sandia.gov's message of 8 Oct 1996 08:51:31 -0600
Status: OR

In article <53dppk$n7r@somnet.sandia.gov> wllarso@sandia.gov (William L. Larson) writes:

   You could also use "dig @ROOT . ns > /tmp/db.cache" where "ROOT"
   would be replaced with "A.ROOT-SERVERS.NET", "B.ROOT-SERVERS.NET",
   "C.ROOT-SERVERS.NET", etc.  Hopefully the "*.ROOT-SERVERS.NET" will
   be authoritative for the root zone. :)


Not necessarily. One may have internal root servers. We do. They are
not called A.ROOT-SERVERS.NET, etc. Perhaps they should be? But then
perhaps that would be confusing too.

Yes, I am being pedantic about all this. And it looks like I am in the
minority on this: trying to do have a general, robust and thorough
job; one that doesn't depend on existing knowledge about how the root
servers are *currently* set up, and gives correct information in the
face of non-authoritative nameservers that may be screwed up. And
although there seems to be no disagreement on this, the solution
should be based on information obtained from DNS databases such as
queried from DNS resolver code.

Recall that TERP.UMD.EDU or something like that was once in there or at
least it's in the O'Reilly DNS & BIND book which is in sad need of
revision. If a really good program were given there, it *wouldn't*
need revision, at least not for this part of it.

Here's the program I'm currently using. I am *almost* happy with it.
Note the care that is used in testing the integrety of the cache
file---it is not just overwritten with a redirect from the output of
a command that one hopes will work.

-------
#!/usr/local/bin/perl
# Time-stamp: <96/10/06 08:35:09 rocky>
# Todo: better option processing; option for initial candidate root servers.
# option for 999999 number? 
# comments in db.cache? 
sub usage {
    print "$prog [-h] [-i old_db.cache] [-o new_db.cache] [-n] [-s] [-q]

Create a new root cache (db.cache or db.root) from the DNS information.

If the -h flag is given or there are any invalid switches, this help is
printed. 

If the -i flag is given, the root servers listed in this db.cache are 
consulted first. Otherwise the default nameservers that are in effect
for the host that runs the program are consulted.

If the -o flag is given, this is where the output will be placed. 

If the -s flag is given, the output is sorted in string order.

If the -n flag is given, the output from dnsquery or nsquery is not removed.

For now, nslookup is used to get information. To try dnsquery instead, 
use the -q flag.
";
}

# Algorithm:
#   Use dnsquery or nslookup to look for root nameservers.
#   Parse the output to see if the answer is authoritative. 
#   If it is authoritative, then we have output that we can parse for 
#   IN or A RRs. If it isn't, there may be an indication of where we *can*
#   find authoritative RRs in the output. 
# 
#   A work list @TRY_ROOT is maintained for candidate nameservers that might be 
#   authoritative. Good initial choices would be the current root cache file 
#   (db.cache, db.root), or we can use the default nameserver and follow
#   where it says is authoritative. So that we can ensure no looping,
#   we should keep track of nameservers that we've already tried.

&initialize();
require "getopts.pl";
&process_options();
&get_TRY_ROOT_servers() if $DB_CACHE_OLD;
#
# Try a server until we get an authoritative list.
#
while(@TRY_ROOT) {
    $root = pop(@TRY_ROOT);
    next if defined($TRIED_ROOT{$root});
    &lookup_root_servers($root);
    if (&have_root_servers($TMPFILE)) {
	&format_output(*LOOKUP_OUTPUT);
	last;
    }
}
&move_in_root_cache();

#
# Clean up.
#
$count = unlink($TMPFILE) if !$opt_n;

exit 0;

#
# Initialize various things...
#
sub initialize {
    ($prog = $0) =~ s/.*\///;
    $TMPFILE="/tmp/ns.$$";
    $CACHETMP="/tmp/db.cache.tmp";
    @TRY_ROOT = ('');
    
    $ENV{'PATH'}="/usr/bin:/usr/ucb:/usr/sbin:/bin:/usr/local/bin";

    # match things like: 
    # .       99999999 IN NS milton.peccary.sty.
    $cache_root_pat = '^\.\s+\d+\s+IN\s+NS\s+(\S+)$';
}

sub process_options {
    &usage() if !Getopts('i:o:hsnq');
    &usage if $opt_h;
    $DB_CACHE_NEW = $opt_o if $opt_o;
    $DB_CACHE_OLD = $opt_i if $opt_i;

    # Are we using dnsquery or nsquery? 
    $cmd = `which dnsquery`; chop($cmd);
    if ($opt_q && $? == 0 && -x $cmd) {
	$using_dnsquery=1;

	$root_pat = $cache_root_pat;

	# match things like: 
	# buster.peccary.sty.	99999999 IN A	9.9.9.9
        # Note: the IN is "saved" in the pattern below merely to keep the place
        # holders $3 in dnsquery and nslookup to be the same so that they can
        # share common code.
	$in_a_pat = '^(\S+)\s+\d+\s+(IN)\s+A\s+([0-9\.]+)$';
    } else {
	# using nslookup
	$using_dnsquery=0;

	# match things like: 
        # (root) nameserver = buster.peccary.sty
	$root_pat = '^\(root\)\s+nameserver = (.*)$';

	# match things like: 
        #   buster.peccary.sty	internet address = 9.9.9.9
        #   buster.peccary.sty	inet address = 9.9.9.9
	$in_a_pat = '^(\S+)\s+(internet|inet) address =\s+([0-9\.]+)$';
    }
}

#
# Get a list of all root servers in the current (old) db.cache.
# Global @TRY_ROOT is set with these.
#
sub get_TRY_ROOT_servers {
    if (!open(OLD_CACHE, "$DB_CACHE_OLD")) {
	warn "Can't open old db cache file: ${DB_CACHE_OLD}: $!";
	warn "Will try normal using current default nameserver configuration.";
	return;
    }
    
    @TRY_ROOT=('');
    while (<OLD_CACHE>) {
	if (/$cache_root_pat/) {
	    push @TRY_ROOT, $1;
	}
    }
}

#
# Look up root server information and store it in temporary file $TMPFILE.
#
sub lookup_root_servers {
    local($ROOT) = @_;
    local($opt);
    if ($using_dnsquery) {
	$opt = "-s $ROOT" if $ROOT;
	$output = `dnsquery $opt -tns . 2>&1 >$TMPFILE`;
    } else {
	$output = `nslookup -type=ns . $ROOT 2>&1 >$TMPFILE`;
    }
    $TRIED_ROOT{$ROOT} = 1; # Note that we've been there, done that.
}

#
# See if we have authoritative root server info. 
# This parses nslookup output.
#
sub have_root_servers {
    local($query_output) = @_;
    open(INPUT, "$query_output") || 
	die "Can't read input file $query_output: $!";
    @LOOKUP_OUTPUT=<INPUT>;
    close(INPUT);
    return $using_dnsquery 
	? &parse_root_dnsquery() 
	: &parse_root_nslookup();
}

#
# Generate the the db.cache file from the nslookup output.  Store
# it in a temporary file.
# Note: dnsquery can list a address RRs more than once. So we check for
# duplicate A's (and NS's which may be overkill...)
sub format_output {
    local(*LOOKUP_OUTPUT) = @_;
    local(@NEW_OUTPUT);
    local($have_a) = 0;
    local($have_ns) = 0;
    $[ = 1;			# set array base to 1
    foreach (@LOOKUP_OUTPUT) {
	if (/$root_pat/i) {
	    $name = $1;
	    $name .= '.' if $name !~ /\.$/;
	    next if defined($NS{$name});
	    push(@NEW_OUTPUT, 
		 sprintf "%-20s 99999999 IN  NS %s\n", ".", $name);
	    $have_ns = 1;
	    $NS{$name} = 1;
	} elsif (/$in_a_pat/i) {
	    $name = $1;
	    $address = $3;
	    $name .= '.' if $name !~ /\.$/;
	    next if defined($A{$name});
	    push(@NEW_OUTPUT, 
		 sprintf "%-20s 99999999 IN  A  %s\n", "$name", $address);
	    $have_a = 1;
	    $A{$name} = 1;
	}
    }
    if ($have_a && $have_ns) {
	@NEW_OUTPUT = sort(@NEW_OUTPUT) if $opt_s;
	open(OUTPUT, ">$CACHETMP") || 
	    die "Can't write output file $TMPFILE: $!";
	print OUTPUT @NEW_OUTPUT;
	close(OUTPUT);
    } else {
	warn "Didn't get any NS RRs, so I'm not writing any output." 
	    if !$have_ns;
	warn "Didn't get any A RRs, so I'm not writing any output." 
	    if !$have_a;
    }
}

sub parse_root_nslookup {
    local($authoritative) = 1;
    local($found_in_a) = 0;
    foreach (@LOOKUP_OUTPUT) {
	if (/^Non-authoritative answer|^Authoritative answers can be found/) {
	    # Go for the gold...
	    $authoritative = 0;
	    next ;
	}
	return 0 
	    if /No address \(A\) records available for/;
	return 0 
	    if /[Cc]an't find/;
        next if /$root_pat/;
	if (/$in_a_pat$/) {
            $root = $1;
            next if defined($TRIED_ROOT{$root});
 	    push(@TRY_ROOT, $root);
            $found_in_a = 1;
        }
    }
    return $authoritative && $found_in_a;
}

sub parse_root_dnsquery {
    local($authoritative) = 1;
    local($found_in_a) = 0;
    foreach (@LOOKUP_OUTPUT) {
        #;; flags: qr aa rd ra; Ques: 1, Ans: 3, Auth: 1, Addit: 1
	if (/^;; flags:.* Auth: 0.*/) {
	    # Go for the gold...
	    $authoritative = 0;
	    next ;
	}
	return 0 
	    if /^Query failed/;
        next if /$root_pat/;
	if (/$in_a_pat$/) {
            $root = $1;
            next if defined($TRIED_ROOT{$root});
 	    push(@TRY_ROOT, $root);
            $found_in_a = 1;
        }
    }
    return $authoritative && $found_in_a;
}

#
# Move the new db.cache in place if it is not empty.
# A bit of caution is taken. We test that the file is non-null,
# The file is moved only if it is different, otherwise we "touch" the
# file so it is known that this update process ran. 
# Finally we save the old db.cache file. Just in case....
#
sub move_in_root_cache {
    if ( -f $CACHETMP && -s $CACHETMP && $DB_CACHE_NEW ) {
	if (-f $DB_CACHE_NEW) { 
	    $output = `cmp $CACHETMP $DB_CACHE_NEW`;
	    if ( ($? >> 8) != 0 ) {
		# Save the old, move in the new.
		rename $DB_CACHE_NEW, "${DB_CACHE_NEW}.old";
		$output = `mv $CACHETMP $DB_CACHE_NEW`;
	    } else {  
		# Make note we've run this. remove the temporary cache file.
		$now = time;
		utime $now, $now, $DB_CACHE_NEW;
		unlink $CACHETMP;
	    }
	} else {
	    # No pre-existing root cache file. We're free just to move in 
	    # the new file...
	    $output = `mv $CACHETMP $DB_CACHE_NEW`;
	}
    }
}





