Searching on SO and Google, yields the following solution: move the icu*.dll files to your Apache's bin directory. Only problems with this is:
- I don't use apache. I use IIS.
- I don't have icu*.dll files in my PHP directory.
And that last thing is weird, because I should have them in my PHP directory!
It seems that the MSI installers (both 5.3.26 and 5.3. 29) just fail to install these for me. The zipped PHP packages DO include them. So I used the zipped PHP package in stead of the PHP that I got from the MSI installer.
Also I made sure the PHP installation directory, where those icu*.dll were, is set in my system PATH variable.