[TAG] Reading/writing large buffers to Fibre Channel
jimregan at o2.ie
Tue Sep 28 04:49:20 MSD 2004
Jimen Ching wrote:
> On Mon, 27 Sep 2004, Jimmy O'Regan wrote:
>>Jimen Ching wrote:
>>>I have a problem of reading and writing very large (4MB) buffers to a disk
>>>via Fibre Channel. Fibre Channel works best when you send large amounts
>>>of data over the wire (light).
>>If you have enough RAM, try using a ramdisk - create a filesystem as
>>usual, but on one of the ramdisk devices - /dev/ram* or /dev/rd/*
> I'm not sure I understand the answer. Or maybe I didn't explain my
> question clearly.
No, I the misunderstanding was on my part. I was simply answering this:
"I'm looking for ways to optimize the reading and writing of
this 4MB data buffer."
> I want to write raw data to a disk via Fibre Channel. Each block of raw
> data is 4MB large. I need to write 95MB/s of data for about half an hour
> or so. 95MB/s, at 60 sec per minute, and 30 minutes equals 171Gig.
> I guess I could put one second worth of raw data into ramdisk, and do a
> copy to the Fibre Channel SCSI RAID. Then write the next second of raw
> data to another ramdisk and switch back-and-forth. But I'm not sure if a
> cp is any faster than a fwrite. Is this what you're suggesting?
No, I was placing more importance on the step where you add a file
header to the data in the buffer.
Going by this: http://linuxgazette.net/102/piszcz.html you'd be much
better off accessing the disk as ext2 instead of ext3 - the journal is
probably what's slowing you down.
More information about the TAG