too long

Question

I have a dom document to which I would like to append some special characters.

What I am doing is first to detect the special character in a string with a regexp. For that, I am transforming the utf-8 characters on html with

$string = "Test string 1,§,†,‡";
$string_html = htmlentities($string_html, ENT_QUOTES, "UTF-8");

This works fine and I am getting there:

"Test string 1,§,†,‡"

Now, I would like to get the special characters only. I am using for that a regexp which is returning an array with:

[0] => '&sect';
[1] => '&dagger';
[2] => '&Dagger';

Now, I would like to append those special characters to my dom document, but in unicode. I was trying:

$string_utf8 = html_entity_decode($string_html);

$dom_output->createElement( 'string', utf8_encode($string_utf8));

The results are the next:

The § character is displayed as §, and the † and ‡ are staying as that.

Any idea why?

Answer 1

The issue with § is that it is at position 0xA7 in the unicode table. However, that particular character is encoded as the two-byte sequence 0xC2A7. Your output is being rendered as ISO-8859-1 or similar encoding, resulting in this  (0xC2) appearing before your intended character (since coincidentally it has the same last byte - it needn't, I frequently see é encoded as é for example)

However the dagger and double-dagger are much further in the unicode table: 0x2020 and 0x2021 respectively. As such, there is no confusion when it comes to encoding them.

I think the problem may be caused by the PHP file itself being encoded in a single-byte character set, and it is this that is causing the issues with 0xA7. Make sure your PHP file is encoded in the correct format, otherwise it will make incorrect assumptions about your strings.