从Android应用程序上传大文件会导致内存错误，尽管缓冲

Question

I'm trying to upload a large video file (about 900mb) to a PHP server using POST. I'm using FileInputStream to read the file and writing it to a HTTPUrlConnection using a DataOutputStream. I eventually get out of memory despite a 1024 byte buffer, below is my code

String result = "";

String lineEnd = "
";
String twoHyphens = "--";
String boundary = "*****";
try {

    FileInputStream fileInputStream = new FileInputStream(
            new File(path));

    URL url = new URL(ServerUtils.ip + "upload.php");
    HttpURLConnection conn = (HttpURLConnection) url.openConnection();
    conn.setDoInput(true);
    conn.setDoOutput(true);
    conn.setUseCaches(false);
    conn.setRequestMethod("POST");
    conn.setRequestProperty("Connection", "Keep-Alive");
    conn.setRequestProperty("Content-Type",
            "multipart/form-data;boundary=" + boundary);

    DataOutputStream dos = new DataOutputStream(new BufferedOutputStream(conn.getOutputStream()));
    dos.writeBytes(twoHyphens + boundary + lineEnd);
    dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\""
            + path + "\"" + lineEnd);
    dos.writeBytes(lineEnd);

    int count;
    byte[]  buffer = new byte[1024];
    while((count = fileInputStream.read(buffer)) > 0){
        dos.write(buffer, 0 , count);
    }

    dos.writeBytes(lineEnd);
    dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);

    fileInputStream.close();
    dos.flush();
    dos.close();
    BufferedReader rd = new BufferedReader(new InputStreamReader(
            conn.getInputStream()));
    String line;
    while ((line = rd.readLine()) != null) {
        result += line;
    }
    rd.close();

} catch (Exception e) {
    e.printStackTrace();
}

And here is the error

Throwing OutOfMemoryError "Failed to allocate a 28 byte allocation with 0 free bytes and 3GB until OOM" (recursive case)
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art: "FinalizerDaemon" daemon prio=5 tid=5 Runnable
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:   | group="system" sCount=0 dsCount=0 obj=0x32c071c0 self=0xae085200
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:   | sysTid=4387 nice=0 cgrp=default sched=0/0 handle=0xb42ff930
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:   | state=R schedstat=( 862325936 45498700 428 ) utm=43 stm=43 core=2 HZ=100
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:   | stack=0xb41fd000-0xb41ff000 stackSize=1038KB
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:   | held mutexes= "mutator lock"(shared held)
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:     at com.android.internal.os.BinderInternal$GcWatcher.finalize(BinderInternal.java:53)
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:     at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:202)
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:     at java.lang.Daemons$FinalizerDaemon.run(Daemons.java:185)
08-26 14:48:56.276 4379-4387/com.hamzahrmalik.groupwatch W/art:     at java.lang.Thread.run(Thread.java:818)
08-26 14:48:56.277 4379-4387/com.hamzahrmalik.groupwatch E/System: Uncaught exception thrown by finalizer
08-26 14:48:56.277 4379-4387/com.hamzahrmalik.groupwatch I/art: Waiting for a blocking GC Alloc
08-26 14:48:56.277 4379-4722/com.hamzahrmalik.groupwatch I/art: Waiting for a blocking GC Alloc
08-26 14:48:56.277 4379-4414/com.hamzahrmalik.groupwatch I/art: Waiting for a blocking GC Alloc
08-26 14:48:56.313 4379-4385/com.hamzahrmalik.groupwatch I/art: Clamp target GC heap from 208MB to 192MB
08-26 14:48:56.313 4379-4385/com.hamzahrmalik.groupwatch I/art: Alloc concurrent mark sweep GC freed 0(0B) AllocSpace objects, 0(0B) LOS objects, 0% free, 192MB/192MB, paused 1.360ms total 38.641ms
08-26 14:48:56.313 4379-4387/com.hamzahrmalik.groupwatch I/art: WaitForGcToComplete blocked for 36.526ms for cause Alloc
08-26 14:48:56.313 4379-4387/com.hamzahrmalik.groupwatch I/art: Starting a blocking GC Alloc
08-26 14:48:56.313 4379-4722/com.hamzahrmalik.groupwatch I/art: WaitForGcToComplete blocked for 108.918ms for cause Alloc
08-26 14:48:56.313 4379-4722/com.hamzahrmalik.groupwatch I/art: Starting a blocking GC Alloc
08-26 14:48:56.313 4379-4414/com.hamzahrmalik.groupwatch I/art: Waiting for a blocking GC Alloc
08-26 14:48:56.314 4379-4387/com.hamzahrmalik.groupwatch I/art: Waiting for a blocking GC Alloc
08-26 14:48:56.314 4379-4385/com.hamzahrmalik.groupwatch W/art: Throwing OutOfMemoryError "Failed to allocate a 28 byte allocation with 0 free bytes and 3GB until OOM" (recursive case)
08-26 14:48:56.338 4379-4385/com.hamzahrmalik.groupwatch W/art: "JDWP" daemon prio=5 tid=3 Runnable
08-26 14:48:56.339 4379-4385/com.hamzahrmalik.groupwatch W/art:   | group="system" sCount=0 dsCount=0 obj=0x32c07100 self=0xae083900
08-26 14:48:56.339 4379-4385/com.hamzahrmalik.groupwatch W/art:   | sysTid=4385 nice=0 cgrp=default sched=0/0 handle=0xb455a930
08-26 14:48:56.339 4379-4385/com.hamzahrmalik.groupwatch W/art:   | state=R schedstat=( 298147394 24472603 183 ) utm=25 stm=4 core=3 HZ=100
08-26 14:48:56.339 4379-4385/com.hamzahrmalik.groupwatch W/art:   | stack=0xb445e000-0xb4460000 stackSize=1014KB
08-26 14:48:56.339 4379-4385/com.hamzahrmalik.groupwatch W/art:   | held mutexes= "mutator lock"(shared held)

Answer 1

The problem is most probably caused because the system is buffering the request before sending it to the server. To prevent that (the request form being buffered), you can use:

conn.setChunkedStreamingMode(....);

Where conn is an instance of HttpURLConnection.

More details