[TAG] 2c tip: filtering in-place

[Jay R. Ashworth]

On Sun, Mar 21, 2004 at 10:14:27PM -0500, Ben Okopnik wrote:
> On Fri, Mar 19, 2004 at 01:44:22PM -0500, Jay R. Ashworth wrote:
> > On Fri, Mar 19, 2004 at 10:39:34AM -0500, Ben Okopnik wrote:
> > > On Fri, Mar 19, 2004 at 02:33:21PM +0530, Kapil Hari Paranjape wrote:
> > > > I have always thought that filtering files "in-place" was
> > > > not possible from the command line...
> > > > ...until today---one lives and learns.
> > > > 
> > > >   dd if=file bs=4k | filter | dd of=file bs=4k conv=notrunc
> > > > 
> > > > Where "file" is the file you want to filter and "filter" 
> > > > is the filtering program you want to apply.
> > > 
> > > The "buffer" program does exactly the same as the above; the process is
> > > called "reblocking".
> > > 
> > > buffer < foo | filter > foo
> > 
> > <interrupts self while raising hand to complain>
> > Oh, cause buffer reads the entire file before the '>' can stomp it?
> > Well, that's not *exactly* the same... :-)
>  
> Hell, Jay, by that standard of nitpickiness, nothing ever is. I mean,
> all those microseconds have passed, many electrons have travelled some
> distance in their orbit, we've moved along the direction indicated by
> the entropy arrow...
> 
> The meaning of "exactly" for us humans always means "close enough".

Well, in this case, my point was that I question whether it's "close
enough", particularly to end up in a tip...

> > Doesn't that still depend on order of evaluation by the shell?  Is that
> > defined?
> 
> Well, *yeah* - just about as definitively as anything in Bash is.
> Otherwise Kapil's method wouldn't work either. Neither would piping
> anything through "sort". The left side of the pipe has to terminate
> before the right side can do anything with the output; in many cases,
> there _is_ no output until just before the left side terminates.

In fact I think that's wrong: I don't think the dd method *does* depend
on order of eval; the writing copy of dd can't try to write a block
until it *has* it, so I believe that that method is *guaranteed* never
to stomp data.

> Any conversation that includes "yeah, but csh..." is, in the parlance of
> an acquaintance, "right off the bugfuck side of the spectrum."

Three sigma West of ridiculous, yes.  Did *I* mention csh?  :-)

Cheers,
-- jra
-- 
Jay R. Ashworth                                                jra at baylink.com
Member of the Technical Staff     Baylink                             RFC 2100
The Suncoast Freenet         The Things I Think
Tampa Bay, Florida        http://baylink.pitas.com             +1 727 647 1274

        "They had engineers in my day, too."  -- Perry Vance Nelson