C ++：提高ifstream二进制文件的读取速度

Question

I am rewriting small program from PHP to C++. The idea is basically to read through 32Gb file on an SSD and do some simple operations on it.

I am using Visual Studio 2012 with x64 release build. PHP is 5.3 32bit.

The problem is that bare reading speed in PHP is higher, than in C++, and this really puzzles me. PHP does ~350 Mb/s and C++/ifstream code does 180 Mb/sec.

Code is really simple:

ifstream datafile("data.txt", ios::binary);

while(datafile.read((char*)buffer, data_per_chunk)) {
//  do stuff;

I've tried different buffer sizes up to 16Mb and it did little difference. I also tried to set internal buffer via datafile.rdbuf()->pubsetbuf(...) but it also didn't made a difference.

Is there any hints on how to speed ifstream up without reverting to ancient C-level interface? I would like to at least reach PHP level of performance. Maybe some fancy read-ahead / cache settings or something.

I understand that memory-mapped files could likely help, but would prefer to tweak settings of ifstream, if it's possible to keep things simple given that file is significantly larger than physical RAM and larger than 4Gb i.e. no-go for potential 32-bit builds.

Answer 1

It appeared that you can reach maximum SSD reading speed even with ifstream.

To do so, you need to set internal ifstream readbuffer to ~2Mb, which is where peak SSD read speed happening, while fitting nicely in L2 cache of CPU. Then you need to readout data in chunks smaller than internal buffer. I've got best results reading data in 8-16kB chunks, but it only about 1% faster than reading in 1Mb chunks.

Setting ifstream internal buffer:

ifstream datafile("base.txt", ios::binary);
datafile.rdbuf()->pubsetbuf(iobuf, sizeof iobuf);

With all these tweaks I've got 495 Mb/sec read speed which is close to theoretical maximum of M500 480Gb SSD. During execution CPU load was 5%, which means that it was not really limited by ifstream implementation overhead.

I found no observable speed difference between ifstream and std::basic_filebuf.