AJAX解析了大量的JSON并存储在localStorage中

Question

I am building an app that should be able to work offline. I am using jQuery Mobile, PHP, MySQL and jStorage in order to easily use HTML5 localStorage

I am trying to figure out the best way to download the data into the local device localStorage and using it at later stages without slowing down or crashing the browser.

I have around 5000 records (500 Kb of data) in a MySQL table, and I need the app to download all this data so that it can be used at a second stage while offline.

download_script.php returns all the records in JSON format e.g.

{"1":{"1":{"p_bar":"10.30","v_0":"0.0312207306000000","h_vap":"311.78","p_10c":"99.99"}},"2":{"1":{"p_bar":"10.40","v_0":"0.0309405941000000","h_vap":"311.29","p_10c":"0.00"}},

I was wondering if there is any way I could optimize the following script (e.g. in order to not hang the browser, and possibly to display the percentage of data that is being downloaded)

$.ajax({
  url: "download_script.php",
  cache: false,
  success: function(big_json_dump){
  $.jStorage.set('some_key', big_json_dump);
 }
});

Can this approach be optimized e.g. using radpidjson? How can I change it so to display a live percentage of data downloaded?

Answer 1

(You had a good discussion in the comments but the question is left unanswered...)

"in order to not hang the browser" you need to split the data into smaller parts and fetch every part separately (e.g. in a loop). Otherwise, parsing a big chunk of JSON can hang the browser for a few milliseconds. Rapidjson can't help there, because Rapidjson is C++ and the browser talks JavaScript.

"to display the percentage of data" you need to inform the browser's JavaScript about the total amount of parts first. Again, nothing to do with Rapidjson.

There's also JavaScript streaming JSON parsers (like Clarinet) which, theoretically, can parse the large JSON document in chunks. This gives you more control about the parsing, at the cost of much CPU and programming complexity. You'll probably need to introduce a Web Worker or an aritifial moments of inactivity to keep the browser responsive. But if you're using a Web Worker, you can do the standard JSON.parse in there: http://igorminar.github.io/webworker-json-perf/; http://blog.softwareispoetry.com/2013/05/using-web-workers-to-jsonparse-large.html