NOTE: Resolved - see answer below. Question kept for reference.
Something is causing nginx or php or phpbb (or something else) to add 1 byte to the beginning of avatar files downloaded from phpbb. This corrupts the file.
To exclude external factors like Cloudflare, I have now setup phpbb as non-https, localhost only, compression off.
The problem can be seen from the following code:
Get the file from localhost - no caching or cloudflare to interfere:
wget http://localhost/forum/download/file.php?avatar=4625_1413540046.jpg -O avatest_local.jpg
‘avatest_local.jpg’ saved [6419/6419]
Test the file
jpeginfo avatest_local.jpg
avatest_local.jpg Not a JPEG file: starts with 0x0a 0xff [ERROR]
Find the identical file (phpbb does weird things to filenames)
find /var/www/forum/images/avatars/upload -size 6419c
/var/www/forum/images/avatars/upload2a36dc33069249d6b1187fd84d7fc957_4625.jpg
Test THAT file and find it to be good
jpeginfo /var/www/forum/images/avatars/upload/2a36dc33069249d6b1187fd84d7fc957_4625.jpg
./2a36dc33069249d6b1187fd84d7fc957_4625.jpg 80 x 80 24bit JFIF N 6419
Examine the first few bytes of the original file:
xdd /var/www/forum/images/avatars/upload/2a36dc33069249d6b1187fd84d7fc957_4625.jpg
0000000: ffd8 ffe0 0010 4a46 4946 0001 0101 013a ......JFIF.....:
0000010: 013a 0000 ffed 0036 5068 6f74 6f73 686f .:.....6Photosho
0000020: 7020 332e 3000 3842 494d 0404 0000 0000 p 3.0.8BIM......
Examine the first few bytes of the downloaded file and see the extra bit:
xdd avatest_local.jpg
0000000: 0aff d8ff e000 104a 4649 4600 0101 0101 .......JFIF.....
0000010: 3a01 3a00 00ff ed00 3650 686f 746f 7368 :.:.....6Photosh
0000020: 6f70 2033 2e30 0038 4249 4d04 0400 0000 op 3.0.8BIM...
Strip the first bit back out of downloaded file to prove the point:
tail -c +2 avatest_local.jpg > avatest_fixed.jpg
jpeginfo avatest_fixed.jpg
avatest_fixed.jpg 80 x 80 24bit JFIF N 6418
May or may not be relevant, but if I use the chrome header viewer inspector, the http response headers for avatars show the following:
content-disposition inline; filename=4625.jpg
But if I use redbot, it shows:
Content-Disposition: inline; filename*=UTF-8''4625.jpg
and warns that
The Content-Disposition header doesn't have a 'filename' parameter.
However, on the official phpbb board, the content-disposition headers seem correct, although as you can see below, this doesn't seem to affect other methods of viewing the image.
Summary:
When downloading or viewing an avatar in the format
/download/file.php?avatar=4625_1413540046.jpg
a single byte is added to the file, making it invalid.When viewing an image like
/download/file.php?id=2871
the image is totally valid and displays OK.All image files currently on the server are valid.
- I have run php in "lint" mode and all files seem valid.
- I have tried running the board as "localhost" in non-https to completely remove external factors.
- I have disabled all the extensions, and even removed their files.
- This board was upgraded by deleting all core files and copying the new ones over.
- Nginx serves image and other files directly perfectly well, and other software on the same server runs fine.
Server details:
nginx 1.9.4
PHP 5.6.4-4ubuntu6.3
phpbb 3.1.6
Linux 3.19.0-30-generic #34-Ubuntu SMP Fri Oct 2 22:07:32 UTC 2015 i686 i686 i686 GNU/Linux
One entire day spent on this, and both myself and the phpbb forum are totally flummoxed. Only started happening since update to phpbb 3.1.6. Clearly it's a problem unique to me. Or maybe not?