在Apache下，单个目录中的许多文件会导致更长的加载时间吗？

Question

Even if there seem to exist a few duplicate questions, I think this one is unique. I'm not asking if there are any limits, it's only about performance drawbacks in context of Apache. Or unix file system in general.

Lets say if I request a file from an Apache server

http://example.com/media/example.jpg

does it matter how many files there are in the same directory "media"?

The reason I'm asking is that my PHP application generates images on the fly.

Once created, it places it at the same location the PHP script would trigger due to ModRewrite. If the file exists, Apache will skip the whole PHP execution and directly serve the static image instead. Some kind of gateway cache if you want to call it that way.

Apache has basically two things to do:

1. Check if the file exists
2. Serve the file or forward the request to PHP

Till now, I have about 25.000 files with about 8 GB in this single directory. I expect it to grow at least 10 times in the next years.

While I don't face any issues managing these files, I have the slight feeling that it keeps getting slower when requesting them via HTTP. So I wondered if this is really what happens or if it's just my subjective impression.

Answer 1

Most file systems based on the Berkeley FFS will degrade in performance with large numbers of files in one directory due to multiple levels of indirection.

I don't know about other file systems like HFS or NTFS, but my suspicion is that they may well suffer from the same issue.

I once had to deal with a similar issue and ended up using a map for the files.

I think it was something like md5 myfilename-00001 yielding (for example): e5948ba174d28e80886a48336dcdf4a4 which I then put into a file named e5/94/8ba174d28e80886a48336dcdf4a4. Then a map file mapped 'myfilename-00001' to 'e5/94/8ba174d28e80886a48336dcdf4a4'. This not-quite-elegant solution worked for my purposes and it only took a little bit of code.