[TAG] 2c tip: filtering in-place
Jay R. Ashworth
jra at baylink.com
Mon Mar 22 18:10:31 MSK 2004
On Sun, Mar 21, 2004 at 10:14:27PM -0500, Ben Okopnik wrote:
> On Fri, Mar 19, 2004 at 01:44:22PM -0500, Jay R. Ashworth wrote:
> > On Fri, Mar 19, 2004 at 10:39:34AM -0500, Ben Okopnik wrote:
> > > On Fri, Mar 19, 2004 at 02:33:21PM +0530, Kapil Hari Paranjape wrote:
> > > > I have always thought that filtering files "in-place" was
> > > > not possible from the command line...
> > > > ...until today---one lives and learns.
> > > >
> > > > dd if=file bs=4k | filter | dd of=file bs=4k conv=notrunc
> > > >
> > > > Where "file" is the file you want to filter and "filter"
> > > > is the filtering program you want to apply.
> > >
> > > The "buffer" program does exactly the same as the above; the process is
> > > called "reblocking".
> > >
> > > buffer < foo | filter > foo
> > <interrupts self while raising hand to complain>
> > Oh, cause buffer reads the entire file before the '>' can stomp it?
> > Well, that's not *exactly* the same... :-)
> Hell, Jay, by that standard of nitpickiness, nothing ever is. I mean,
> all those microseconds have passed, many electrons have travelled some
> distance in their orbit, we've moved along the direction indicated by
> the entropy arrow...
> The meaning of "exactly" for us humans always means "close enough".
Well, in this case, my point was that I question whether it's "close
enough", particularly to end up in a tip...
> > Doesn't that still depend on order of evaluation by the shell? Is that
> > defined?
> Well, *yeah* - just about as definitively as anything in Bash is.
> Otherwise Kapil's method wouldn't work either. Neither would piping
> anything through "sort". The left side of the pipe has to terminate
> before the right side can do anything with the output; in many cases,
> there _is_ no output until just before the left side terminates.
In fact I think that's wrong: I don't think the dd method *does* depend
on order of eval; the writing copy of dd can't try to write a block
until it *has* it, so I believe that that method is *guaranteed* never
to stomp data.
> Any conversation that includes "yeah, but csh..." is, in the parlance of
> an acquaintance, "right off the bugfuck side of the spectrum."
Three sigma West of ridiculous, yes. Did *I* mention csh? :-)
Jay R. Ashworth jra at baylink.com
Member of the Technical Staff Baylink RFC 2100
The Suncoast Freenet The Things I Think
Tampa Bay, Florida http://baylink.pitas.com +1 727 647 1274
"They had engineers in my day, too." -- Perry Vance Nelson
More information about the TAG