如何在不跳过的情况下从Javascript页面获取多个文件上传到服务器？

Question

I'm working on a research experiment which uses getUserMedia, implemented in recorder.js, to record .wav files from the user's microphone and XMLHttpRequest to upload them to the server. Each file is about 3 seconds long and there are 36 files in total. The files are recorded one after another and sent to the server as soon as they are recorded.

The problem I'm experiencing is that not all of the files end up on the server. Apparently the script or the php script are unable to catch up with all the requests in a row. How can I make sure that I get all the files? These are important research data, so I need every recording.

Here's the code that sends the files to the server. The audio data is a blob:

var filename = subjectID + item__number;    
xhr.onload=function(e) {
    if(this.readyState === 4) {
    console.log("Server returned: ",e.target.responseText);
    }
};
var fd=new FormData();
fd.append("audio_data",blob, filename);
xhr.open("POST","upload_wav.php",true);
xhr.send(fd);

And this is the php file on the server side:

print_r($_FILES);     
$input = $_FILES['audio_data']['tmp_name'];
$output = "audio/".$_FILES['audio_data']['name'].".wav";
move_uploaded_file($input, $output)

This way of doing things is basically copied from this website: Using Recorder.js to capture WAV audio in HTML5 and upload it to your server or download locally

I have already tried making the XMLHttpRequest wait by using

   while (xhr.readyState != 4) 
   { 
     console.log("Waiting for server...")
   }

It just caused the page to hang.

Would it be better to use ajax than XMLHttp Request? Is there something I can do to make sure that all the files get uploaded? I'm pretty new to Javascript so code examples are appreciated.

Answer 1

I have no idea what your architecture looks like, but here is a potential solution that will work to solve your problem.

The solution uses the Web Worker API to off load the file uploading to a sub-process. This is done with the Worker Interface of that API. This approach will work because there is no contention of the single thread of the main process - web workers work in their own processes.

Using this approach, we do three basic things:

1. create a new worker passing a script to execute
2. pass messages to the worker for the worker to deal with
3. pass messages back to the main process for status updates/replies/resolved data transformation/etc.

The code is heavily commented below to help you understand what is happening and where.

This is the main JavaScript file (script.js)

// Create a sub process to handle the file uploads
///// STEP 1: create a worker and execute the worker.js file immediately
let worker = new Worker('worker.js');
// Ficticious upload count for demonstration
let uploadCount = 12;
// repeatedly build and send files every 700ms 
// This is repeated until uplaodCount == 0
let builder = setInterval(buildDetails, 700);

// Recieve message from the sub-process and pipe them to the view
///// STEP 2: listen for messages from the worker and do something with them
worker.onmessage = e => {
  let p = document.createElement('pre');
  // e.data represents the message data sent from the sub-process
  p.innerText = e.data;
  document.body.appendChild(p);
};

/**
 * Sort of a mock to build up your BLOB (fake here of-course)
 * 
 * Post the data needed for the FormData() to the worker to handle.
 */ 
function buildDetails() {
  let filename = 'subject1234';
  let blob = new Blob(['1234']);
  ///// STEP 3: Send a message to the worker with file details
  worker.postMessage({
    name: "audio_data",
    blob: blob,
    filename: filename
  });
  // Decrease the count
  uploadCount--;
  // if count is zero (== false) stop the fake process
  if (!uploadCount) clearInterval(builder);
}

This is the sub-process JavaScript file (worker.js)

// IGNORE the 'fetch_mock.js' import that is only here to avoid having to stand up a server
// FormDataPolyFill.js is needed in browsers that don't yet support FormData() in workers
importScripts('FormDataPolyFill.js', 'fetch_mock.js');
// RXJS provides a full suite of asynchronous capabilities based around Reactive Programming (nothing to do with ReactJS);
// The need for your use case is that there are guarantees that the stream of inputs will all be processed
importScripts('https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.3.3/rxjs.umd.js');

// We create a "Subject" that acts as a vessel for our files to upload
let forms = new rxjs.Subject();
// This says "every time the forms Subject is updated, run the postfile function and send the next item from the stream"
forms.subscribe(postFile);

// Listen for messages from the main process and run doIt each time a message is recieved
onmessage = doIt;

/**
 * Takes an event object containing the message
 * 
 * The message is presumably the file details
 */ 
function doIt(e) {
  var fd = new FormData();
  // e.data represents our details object with three properties
  fd.append(e.data.name, e.data.blob, e.data.filename);
  // Now, place this FormData object into our stream of them so it can be processed
  forms.next(fd);
}
// Instead of using XHR, this uses the newer fetch() API based upon Promises
// https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
function postFile(fd) {
  // Post the file to the server (This is blocked in fetch_mock.js and doesn't go anywhere)
  fetch('fake', {
      method: 'post',
      body: fd,
    })
    .then((fd) => {
      // After the XHR request is complete, 'Then' post a message back to the main thread (If there is a need);
      postMessage("sent: " + JSON.stringify(fd));
    });
}

Since this will not run in stackoverflow, I've created a plunker so that you can run this example: http://plnkr.co/edit/kFY6gcYq627PZOATXOnk

If all this seems complicated, you've presented a complicated problem to solve. :-)

Hope this helps.