node.js http响应数据在数据大小很大时终止

Question

I am using nodejs and sending http post request to server. On server side I am running php. My server is returning correct data around 9KB but data in nodejs client is terminated. It works fine if data is less than 6KB. Following is my code

        var reqPost = https.request(optionspost, function(res) {
        res.on('data', function(d) {
            Console.log('Cloud Resp:', d);
            var jsonObj = JSON.parse(d);
        });
    }); 

My print Console.log('Cloud Resp:', d) prints data up to 8KB only. Can someone please help me to understand if this limit is imposed by nodejs or something else and how can I increase the limit

Answer 1

I think your data is chunked during transfering PHP server ---> Node server

• I assume you are using native https module to request from Node side (Correct me if I am wrong)

So in data event you need to concat the chunk. But you parse it only in end event. If you parse in data event it will show you error for JSON.parse() because of incomplete data

Here is the sample code, it works with a 500kb data as I tested. Basically native node does not has data limitation in code level.

var http = require("https");

var options = {
  "method": "GET",
  "hostname": "c16db448-d912-4ce8-823a-db6c51e09878.mock.pstmn.io"
};

var req = http.request(options, function (res) {
  var chunks = '';

  res.on("data", function (chunk) {
    console.log(chunk.length)
    chunks += chunk;
  });

  res.on("end", function () {
    const object = JSON.parse(chunks)
    console.log(object.length)
    console.log(Buffer.byteLength(chunks, 'utf8') / 1024 + " kbytes");
  });
});