[nflug] Unix Command / Bash Question...

[Robert Meyer]

And of course in Linux, you now have the '-u' option to sort which filters out
duplicate lines on the sort.

Cheers!

Bob

--- Robert Meyer <meyer_rm at yahoo.com> wrote:

> From the 'uniq' man page:
> Discard  all but one of successive identical lines from INPUT (or stan-
> dard input), writing to OUTPUT (or standard output).
> 
> The usual pipe process is:
> 
> someprogram | sort | uniq ... whatever else
> 
> So this is expected behaviour...
> 
> Cheers!
> 
> Bob
> 
> --- DANIEL MAGNUSZEWSKI <dmagnuszewski at mandtbank.com> wrote:
> 
> > Ahhh yes...there lies the problem!
> > 
> > Thanks.
> > 
> > >>> rao3 at buffalo.edu 01/27/06 10:35 AM >>>
> > Could it have to do with adjacency?
> > 
> > Example:
> > a
> > b
> > a
> > 
> > uniq will not see lines a and a to be the same, since the line with b 
> > 
> > is intervening.
> > 
> > But, should you grep first, then b is filtered out:
> > 
> > uniq then grep:
> > input:
> > a
> > b
> > a
> > 
> > after uniq:
> > a
> > b
> > a
> > 
> > after grep:
> > a
> > a
> > 
> > 
> > grep then uniq:
> > input:
> > a
> > b
> > a
> > 
> > after grep
> > a
> > a
> > 
> > after uniq:
> > a
> > 
> > Maybe you want to throw in a sort in the pipe?
> > 
> > 
> > On Jan 27, 2006, at 10:25 AM, DANIEL MAGNUSZEWSKI wrote:
> > 
> > > All,
> > >
> > > I wrote a Perl script to parse a file for ip addresses and
> > hostnames,
> > > and to print out in the format of:
> > >
> > > $ip,$hostname
> > >
> > > There are some duplicates in the list, so I was just using some Unix
> > > commands and pipes to rid them from the output. While doing this, I 
> > 
> > > ran
> > > into something interesting.
> > >
> > > If I do:
> > >
> > > user at server:/opt/cwscripts# ./dcraudit.pl | uniq | grep -i
> > '10.x.x.x'
> > >
> > > I get the output of:
> > >
> > > 10.x.x.x,router1
> > > 10.x.x.x,router1
> > >
> > > If I switch the 'uniq' and 'grep' and run:
> > >
> > > user at server:/opt/cwscripts# ./dcraudit.pl |  grep -i '10.x.x.x' |
> > uniq
> > >
> > > I get the output of:
> > >
> > > 10.x.x.x,router1
> > >
> > > What is the technical reason for this? I assumed that it would work
> > > either way.
> > >
> > > Thanks.
> > >
> > > -Dan
> > >
> > > _______________________________________________
> > > nflug mailing list
> > > nflug at nflug.org 
> > > http://www.nflug.org/mailman/listinfo/nflug 
> > >
> > 
> > _______________________________________________
> > nflug mailing list
> > nflug at nflug.org 
> > http://www.nflug.org/mailman/listinfo/nflug 
> > 
> > 
> > _______________________________________________
> > nflug mailing list
> > nflug at nflug.org
> > http://www.nflug.org/mailman/listinfo/nflug
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
> _______________________________________________
> nflug mailing list
> nflug at nflug.org
> http://www.nflug.org/mailman/listinfo/nflug
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
_______________________________________________
nflug mailing list
nflug at nflug.org
http://www.nflug.org/mailman/listinfo/nflug