对于某些字符，PHP和Clojure（Java）代码之间的原始MD5 base64编码字符串的结果不同

Question

I have a server that does create a hash using the following code:

base64_encode(md5("some value", true))

What I have to do is to produce the same hash value in Clojure (using Java interop). What I did is to create the following Clojure functions:

(defn md5-raw [s]
  (let [algorithm (java.security.MessageDigest/getInstance "MD5")
    size (* 2 (.getDigestLength algorithm))]
    (.digest algorithm (.getBytes s))))

(defn bytes-to-base64-string [bytes]
  (String. (b64/encode bytes) "UTF-8"))

Then I use that code that way:

(bytes-to-base64-string (md5-raw "some value")

Now, everything works fine with normal strings. However, after processing multiple different examples, I found the the following character is causing issues:

’

This is the UTF-8 character #8217.

If I run the following PHP code:

base64_encode(md5("’", true))

What is returned is:

yOy9/y97p/GfapveLVQAHA==

If I run the following Clojure code:

(bytes-to-base64-string (md5-raw "’"))

I get the following value:

aF1ZConzUtEGRN2YXaKpoQ==

Why is that? I am suspecting a character encoding issue, but everything appears to be handled as UTF-8 as far as I can see.

Answer 1

Not everything can be guaranteed to be UTF-8 in your example, the following expression depends on your default charset:

(.getBytes s)

You should - well, actually this depends on your use case - use:

(.getBytes s "UTF-8")

Demonstration:

(defn md5-with-charset
  [s charset]
  (let [algorithm (java.security.MessageDigest/getInstance "MD5")]
    (.digest algorithm (.getBytes s charset))))

(b64 (md5-with-charset "’" "UTF-8"))  ;; => "yOy9/y97p/GfapveLVQAHA=="
(b64 (md5-with-charset "’" "ASCII"))  ;; => "0UV7csP7MjomcRJa7z6rXQ=="
(b64 (md5-with-charset "’" "UTF-16")) ;; => "3CLVThylT2KkrocdUpxIpg=="
(b64 (md5-with-charset "’" "UTF-32")) ;; => "iHBMMMzkWTbPU+n8GCHitQ=="

(where b64 is a base64 encoding step)

---

And I found it:

(b64 (md5-with-charset "’" "windows-1250")) ;; => "aF1ZConzUtEGRN2YXaKpoQ=="