stringstream performance problem (Dinkumware)

stringstream performance problem (Dinkumware)

Post by Remo Eiche » Mon, 15 Sep 2003 02:32:40


(first sorry for my bad english)

i have a big performance problem with following (the file is 354KB):

my environment:
vc6 sp5
2.4 ghz P4
512 MB RAM

code:

fstream f("file",ios_base::in | ios_base::out | ios_base::binary);
stringstream ss;
ss << f.rdbuf();

this code use 6 seconds to finish !! this is very very slow.

i tryed many different methods:

1)

fstream f("file",ios_base::in | ios_base::out | ios_base::binary);
istream::pos_type size = stream_size(f);

string buffer;
buffer.reserve(size);
istream_iterator<char> f_iter(f);
copy(f_iter, istream_iterator<char>(),buffer.begin());

this use 0.057 seconds

2)

fstream f("file",ios_base::in | ios_base::out | ios_base::binary);
vector<char> buffer;
istream_iterator<char> f_iter(f);
copy(f_iter, istream_iterator<char>(),back_insert_iterator< vector<char>

this use 0.072

3) the fastest

fstream f("file",ios_base::in | ios_base::out | ios_base::binary);
istream::pos_type size = stream_size(f);
char* buffer = new char[size];
f.read(buffer,size);
delete [] buffer;

this use 0.007 seconds.

i found that the performance problem is the minimum allocation-size in the
implementation of basic_streambuf from Dinkumware. This is 32 byte !? when i
use a my patched streambuf (65535) and insert this in the stringstream then
the first example use 0.055 second !

my question:

1) what is the "correct" and "stl-like" way to solve this performance
problem without changing the interface of the sstream ? ( i don't wan't use
static array c-style)
2) is the following code correct to inject the patched streambuf ?

basic_stringbufx< char, char_traits<char>, allocator<char> > strbufx;
stringstream ss;
ss.ostream::rdbuf(&strbufx);
ss << f.rdbuf();

thanks for your input

Remo Eichenberger

btw. the vc7.1 - compiler optimize the first sample and has 0.01 second !!
:-)
 
 
 

stringstream performance problem (Dinkumware)

Post by tom_usene » Tue, 16 Sep 2003 18:05:35

On Sat, 13 Sep 2003 19:32:40 +0200, "Remo Eichenberger"



That code isn't correct for two reasons - it won't copy whitespace,
and it writes past "end". Instead, use this:

string buffer;
buffer.resize(size); //reserve doesn't change the size
istreambuf_iterator<char> begin(f), end;
copy(begin, end, buffer.begin());



In this case you want:

istreambuf_iterator<char> beg(f), end;
vector<char> buffer(beg, end); //use iter range constructor

If you know the size of the file, resizing the vector first will give
you a gain.


Yes, Dinkumware became aware of this many years ago. They patched
their library to fix it years ago, probably last millenium:

http://www.yqcomputer.com/


The STL way is probably to use vector:
istreambuf_iterator<char> beg(f), end;
vector<char> buffer(beg, end); //use iter range constructor


ss.rdbuf(&strbufx); //will work fine


That looks fine. Provide default template args and typedefs for
stringbufx, so you can do:

typedef basic_stringbufx<char> stringbufx;


Yes, that's more like it (using exponential growth for the stringbuf).

Tom