Very rarely, I'm getting a CRC Mismatch after sending a GZIP-compressed string to a MySQL server and back. Here's how the data flows.
I have a large string in my Android app. I compress it into a byte array using
public synchronized static byte[] compress(String str) throws IOException {
ByteArrayOutputStream os = new ByteArrayOutputStream(str.length());
GZIPOutputStream gz_out = new GZIPOutputStream(os);
gz_out.write(str.getBytes());
gz_out.finish();
gz_out.flush();
gz_out.close();
os.flush();
os.close();
byte[] test = os.toByteArray();
String check = decompress(test, false); // making sure CRC Mismatch doesn't occur here.
if (check == null) {
return compress(str);
}
return test;
}
public synchronized static String decompress(byte[] compressed, boolean throwException) {
try {
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
} catch (IOException e) {
if (throwException)
throw new RuntimeException("Failed to decompress GZIP.", e);
else {
Log.e(log_tag, "Failed to decompress GZIP.", e);
return null;
}
}
}
I send it to my server like this:
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.addBinaryBody("data", data);
InputStream is = null;
try {
HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, 20000);
HttpConnectionParams.setSoTimeout(httpParams, 20000);
HttpClient httpclient = new DefaultHttpClient(httpParams);
HttpPost httppost = new HttpPost(SERVER_ADDRESS + "sendData.php");
httppost.setEntity(builder.build());
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
is = entity.getContent();
} catch (Exception e) {
Log.e("FriendlyFire", "Error in http connection", e);
return null;
}
// Then read response...
sendData.php looks like this:
//I read the bytes into a variable using
$data = $_POST['data'];
//I create a MySQL connection using
$conn = new mysqli(...);
//I prepare a statement using
$stmt = $conn->prepare("CALL database.saveData(?);");
//I attach the data using
$stmt->bind_param('s', $data);
//I execute the statement using
$stmt->execute();
//I close the statement using
$stmt->close();
//I close the connection using
$conn->close();
The MySQL procedure looks like this:
database.saveData(data BLOB)
BEGIN
INSERT INTO Table (columnName) VALUES (data);
END
To download the data to my Android app, I do this:
HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, 10000);
HttpConnectionParams.setSoTimeout(httpParams, 10000);
HttpClient httpclient = new DefaultHttpClient(httpParams);
HttpPost httppost = new HttpPost(SERVER_ADDRESS + "getData.php");
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
byte[] data = EntityUtils.toByteArray(entity);
getData.php looks like this:
$conn = mysqli_connect(...);
$db = mysqli_select_db($conn, "database");
$result = mysqli_query($conn, "CALL database.getData()");
$num_results = mysqli_num_rows($result);
if ($num_results > 0){
while ($row = mysqli_fetch_array($result, MYSQLI_ASSOC))
{
$filedata = $row['columnName'];
header("Content-length: ".strlen($filedata));
header("Content-disposition: download; filename=data.bin");
echo $filedata;
}
} else {
echo "EMPTY";
}
And the getData() procedure looks like this:
database.getData()
BEGIN
SELECT columnName FROM Table LIMIT 1;
END
Like I said, every now and then I get a CRC mismatch error after trying to decompress what I read from the server. I suspect the problem lies in my PHP code, but it could be in my Android code. I'm reasonably certain the SQL procedures are fine.
GZIP says 99.9% of problems are due to transfers being done in ASCII instead of Binary. I've done my best to keep the data in binary, but I must be missing something. Like I said, I suspect my PHP code, in particular the upload code. I've downloaded the BLOB directly from the MySQL server and it also gives the CRC mismatch error.
I've tried preparing the mysqli statement using bind_param('b', $data), but the resulting data in the SQL Table is 0 bytes and I can't find much on using the 'b' parameter type. Am I supposed to use _FILES instead of _POST? Would that even make a difference?
EDIT: I've made further changes to the Android and PHP upload code:
In Android I've changed
builder.addBinaryBody("data", data);
to
builder.addBinaryBody("data", data, Content.DEFAULT_BINARY, "data.bin");
And PHP changes:
$filename = $_FILES['data']['tmp_name'];
//I create a MySQL connection using
$conn = new mysqli(...);
//I prepare a statement using
$stmt = $conn->prepare("CALL database.saveData(?);");
//I attach the data using
$stmt->bind_param('b', $data);
$fp = fopen($filename, "r");
while ($data = fread($fp, 1024)) {
$stmt2->send_long_data(0, $data);
}
//I execute the statement using
$stmt->execute();
//I close the statement using
$stmt->close();
//I close the connection using
$conn->close();
So this successfully sends the data to the MySQL server using the 'b' parameter. The resulting data in the server is not empty. I've downloaded the data and successfully performed a gunzip on it. If I completely stop getting the CRC mismatch, I'll mark this as the correct answer.
