在 forEach 循环中使用 async / await

Are there any issues with using async/await in a forEach loop? I'm trying to loop through an array of files and await on the contents of each file.

import fs from 'fs-promise'

async function printFiles () {
  const files = await getFilePaths() // Assume this works fine

  files.forEach(async (file) => {
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  })
}

printFiles()

This code does work, but could something go wrong with this? I had someone tell me that you're not supposed to use async/await in a higher order function like this so I just wanted to ask if there was any issue with this.

转载于:https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop

12个回答

Sure the code does work, but I'm pretty sure it doesn't do what you expect it to do. It just fires off multiple asynchronous calls, but the printFiles function does immediately return after that.

If you want to read the files in sequence, you cannot use forEach indeed. Just use a modern for … of loop instead, in which await will work as expected:

async function printFiles () {
  const files = await getFilePaths();

  for (const file of files) {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  }
}

If you want to read the files in parallel, you cannot use forEach indeed. Each of the async callback function calls does return a promise, but you're throwing them away instead of awaiting them. Just use map instead, and you can await the array of promises that you'll get with Promise.all:

async function printFiles () {
  const files = await getFilePaths();

  await Promise.all(files.map(async (file) => {
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  }));
}
csdnceshi52
妄徒之命 join is a method on arrays of strings, it doesn't wait for promises. What you can do is var string = (await Promise.all(…)).join(…).
2 年多之前 回复
csdnceshi50
三生石@ Another "couldn't you...", couldn't one use a .join with this as well to wait for all promises to be complete? Then you can iterate through the files in the list, etc. I really like @Babak's Task and traversable list. I'm going to find an excuse to use that somewhere as soon as I can.
2 年多之前 回复
csdnceshi52
妄徒之命 I thought "and wait for it to finish" was implied. But still, if "cannot" conveys the same as "absolutely should not" I'm fine. Keep it simple for the beginners: they should never use forEach, that's all they need to know.
2 年多之前 回复
csdnceshi56
lrony* Oh, now I understand what you mean by that part. But it is really vague, because technically you can use it but that doesn't mean you should. Your answer basically conveys that it is not possible to run requests in parallel using forEach.
2 年多之前 回复
csdnceshi52
妄徒之命 You cannot use it because you cannot await the result. Of course if you used it - like the OP did - then they would run in parallel, but forEach is absolutely unfit as you said yourself.
2 年多之前 回复
csdnceshi56
lrony* However, I still don't understand how this is correct: If you want to read the files in parallel, you cannot use forEach indeed.
2 年多之前 回复
csdnceshi56
lrony* Of course there are cases where the for..of might be the only way to go (e.g if each request has a dependency on the previous request), but a beginner might always opt for the for..of solution because it is simpler.
2 年多之前 回复
csdnceshi52
妄徒之命 Yes, that's what I wrote in my comment above. Of course it doesn't make any sense and is unfit to solve the OPs problem, regardless whether with forEach or for…of. And no, I would not compare the two (for…of+await vs Promise.all+map) in terms of "performance" at all - they are just different, in many other more important regards. Choosing sequential vs parallel executing is a decision that involves many other factors, and of course parallel typically finishes faster.
2 年多之前 回复
csdnceshi56
lrony* Are you suggesting a case where you don't await whatever request you make inside of a for..of ? If so, alright but you do agree that your second example is way more performant than your first example ?
2 年多之前 回复
csdnceshi52
妄徒之命 But that's wrong: for…of is not bad for performance in comparison to forEach if used in the same unfit way without awaiting anything. And yes, "cannot be used" means "is unfit".
2 年多之前 回复
csdnceshi56
lrony* However, how is this true (from your answer): If you want to read the files in parallel, you cannot use forEach indeed. I think you can do that, same thing as your map example, except with forEach it will look ugly and unfit for an async/await.
2 年多之前 回复
csdnceshi56
lrony* I could have used forEach (would work the same as map regarding execution time). But as you can see from my previous comments, I was talking about either forEach or map versus for..of. I said: for..of is bad for performance in comparison to forEach/map.
2 年多之前 回复
csdnceshi52
妄徒之命 Yes, that's basically the two approaches from my answer. Neither of them uses forEach. Where's the problem?
2 年多之前 回复
csdnceshi56
lrony* I am talking about this: gist.github.com/doubleOrt/598659adfefdd941f2a98512f0f7f078
2 年多之前 回复
csdnceshi52
妄徒之命 No. queries.forEach(q => asyncRequest(q)); does exactly the same as for (const q of queries) asyncRequest(q);. There is no difference in performance, and both will run the requests in parallel. Of course, in neither you can wait for anything - for how to do that, see my answer.
2 年多之前 回复
csdnceshi56
lrony* for..of is bad for performance in comparison to forEach/map, because for..of will stop in each iteration, so if I make 3 requests, each request will have to wait for the preceding requests to complete. But with forEach/map, all the requests will be made in parallel.
2 年多之前 回复
csdnceshi52
妄徒之命 I don't understand. What is bad for performance in comparison to what?
2 年多之前 回复
csdnceshi56
lrony* Isn't that bad for performance ? If you need to make 3 requests, you can't make them all in parallel, so if each takes one second, your program will take 3 seconds to run. But if you use map or forEach, the requests will run in parallel.
2 年多之前 回复
csdnceshi52
妄徒之命 If you don't intend to await them, then for…of would work equally to forEach. No, I really mean that paragraph to emphasise that there is no place for .forEach in modern JS code.
2 年多之前 回复
csdnceshi56
lrony* I think the second paragraph should be "If you want to read the files in parallel, you cannot use for..of" ? Because you can read the files in parallel with the forEach, it's just that it will look very ugly because you cannot await it.
2 年多之前 回复
csdnceshi64
游.程 For those who don't know, Promise.all returns a single promise that resolves when all the promises in the array are resolved. (basically waits for all promises to conclude)
2 年多之前 回复
weixin_41568174
from.. This answer is the best one: await Promise.all(_.map(arr, async (val) => {...}); solved my issue. Of course, each async callback returns a promise that I was not awaiting on.
2 年多之前 回复
csdnceshi71
Memor.の stackoverflow.com/questions/48349699/…
2 年多之前 回复
csdnceshi68
local-host I also had this doubt, I found this link that I found useful Iteration_protocols
2 年多之前 回复
csdnceshi52
妄徒之命 It's a syntactic loop and will work with await, but you rarely will need to enumerate object properties.
大约 3 年之前 回复
csdnceshi63
elliott.david So as regards await in which camp does for( in ) reside? thumbs down with forEach or thumbs up with for of ??
大约 3 年之前 回复
csdnceshi76
斗士狗 When you come to learn about JS promises, but instead use half an hour translating latin. Hope you're proud @Bergi ;)
大约 3 年之前 回复
csdnceshi52
妄徒之命 Not really, an async function is quite different from a Promise executor callback, but yes the map callback returns a promise in both cases.
3 年多之前 回复
csdnceshi73
喵-见缝插针 So files.map(async (file) => ... is equivalent to files.map((file) => new Promise((rej, res) => { ...?
3 年多之前 回复
csdnceshi51
旧行李 Great example. Thank you. I used Array.from to search and replace a dom nodelist (HTML table rows). Map needs an array type collection. For array like objects like the Dom Nodelist you can use Array.from.
3 年多之前 回复
csdnceshi52
妄徒之命 In short, because it was designed to work :-) await suspends the current function evaluation, including all control structures. Yes, it is quite similar to generators in that regard (which is why they are used to polyfill async/await).
接近 4 年之前 回复
weixin_41568110
七度&光 ok i know why... Using Babel will transform async/await to generator function and using forEach means that each iteration has an individual generator function, which has nothing to do with the others. so they will be executed independently and has no context of next() with others. Actually, a simple for() loop also works because the iterations are also in one single generator function.
接近 4 年之前 回复
weixin_41568110
七度&光 Could you please explain why does for ... of ... work?
接近 4 年之前 回复

With ES2018, you are able to greatly simplify all of the above answers to:

async function printFiles () {
  const files = await getFilePaths()

  for await (const file of fs.readFile(file, 'utf8')) {
    console.log(contents)
  }
}

See spec: https://github.com/tc39/proposal-async-iteration


2018-09-10: This answer has been getting a lot attention recently, please see Axel Rauschmayer's blog post for further information about asynchronous iteration: http://2ality.com/2016/10/asynchronous-iteration.html

csdnceshi52
妄徒之命 Shouldn't it be contents instead of file in the iterator
接近 2 年之前 回复
csdnceshi70
笑故挽风 How's the performance of this?
接近 2 年之前 回复
csdnceshi71
Memor.の Unsure where file is defined. This is ported 1:1 from the question above.
接近 2 年之前 回复
csdnceshi54
hurriedly% Where is file defined in your code?
接近 2 年之前 回复
csdnceshi55
~Onlooker Upvoted for show to the community a very new approach
大约 2 年之前 回复
weixin_41568127
?yb? Upvoted, would be great if you could put a link to the spec in your answer for anyone who wants to know more about async iteration.
大约 2 年之前 回复

Here are some forEach async prototypes:

Array.prototype.forEachAsync = async function (fn) {
    for (let t of this) { await fn(t) }
}

Array.prototype.forEachAsyncParallel = async function (fn) {
    await Promise.all(this.map(fn));
}
csdnceshi52
妄徒之命 They should be standalone functions. We've got modules to not pollute globals with our personal things.
一年多之前 回复
weixin_41568127
?yb? As long as the name is unique in the future (like I'd use _forEachAsync) this is reasonable. I also think it's the nicest answer as it saves a lot of boilerplate code.
2 年多之前 回复
weixin_41568196
撒拉嘿哟木头 Although I'd hesitate to add things directly to the prototype, this is a nice async forEach implementation
2 年多之前 回复

In addition to @Bergi’s answer, I’d like to offer a third alternative. It's very similar to @Bergi’s 2nd example, but instead of awaiting each readFile individually, you create an array of promises, each which you await at the end.

import fs from 'fs-promise';
async function printFiles () {
  const files = await getFilePaths();

  const promises = files.map((file) => fs.readFile(file, 'utf8'))

  const contents = await Promise.all(promises)

  contents.forEach(console.log);
}

Note that the function passed to .map() does not need to be async, since fs.readFile returns a Promise object anyway. Therefore promises is an array of Promise objects, which can be sent to Promise.all().

In @Bergi’s answer, the console may log file contents out of order. For example if a really small file finishes reading before a really large file, it will be logged first, even if the small file comes after the large file in the files array. However, in my method above, you are guaranteed the console will log the files in the same order as they are read.

Instead of Promise.all in conjunction with Array.prototype.map (which does not guarantee the order in which the Promises are resolved), I use Array.prototype.reduce, starting with a resolved Promise:

async function printFiles () {
  const files = await getFilePaths();

  await files.reduce(async (promise, file) => {
    // This line will wait for the last async function to finish.
    // The first iteration uses an already resolved Promise
    // so, it will immediately continue.
    await promise;
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  }, Promise.resolve());
}
weixin_41568131
10.24 The files will be processed sequentially, one at a time.
大约 2 年之前 回复
csdnceshi72
谁还没个明天 This is very clever! Thank you!
大约 2 年之前 回复
csdnceshi59
ℙℕℤℝ This is pretty cool. Am I right in thinking the files will be read in order and not all at once?
大约 2 年之前 回复
weixin_41568110
七度&光 This works perfectly, thank you so much. Could you explain what is happening here with Promise.resolve() and await promise;?
2 年多之前 回复

To me using Promise.all() with map() is a bit difficult to understand and verbose, but if you want to do it in plain JS that's your best shot I guess.

If you don't mind adding a module, I implemented the Array iteration methods so they can be used in a very straightforward way with async/await.

An example with your case:

const { forEach } = require('p-iteration');
const fs = require('fs-promise');

async function printFiles () {
  const files = await getFilePaths();

  await forEach(files, async (file) => {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  });
}

printFiles()

p-iteration

csdnceshi74
7*4 Wow, p-iteration is so smooth. Saved my day!
2 年多之前 回复

Both the solutions above work, however, Antonio's does the job with less code, here is how it helped me resolve data from my database, from several different child refs and then pushing them all into an array and resolving it in a promise after all is done:

Promise.all(PacksList.map((pack)=>{
    return fireBaseRef.child(pack.folderPath).once('value',(snap)=>{
        snap.forEach( childSnap => {
            const file = childSnap.val()
            file.id = childSnap.key;
            allItems.push( file )
        })
    })
})).then(()=>store.dispatch( actions.allMockupItems(allItems)))

it's pretty painless to pop a couple methods in a file that will handle asynchronous data in a serialized order and give a more conventional flavour to your code. For example:

module.exports = function () {
  var self = this;

  this.each = async (items, fn) => {
    if (items && items.length) {
      await Promise.all(
        items.map(async (item) => {
          await fn(item);
        }));
    }
  };

  this.reduce = async (items, fn, initialValue) => {
    await self.each(
      items, async (item) => {
        initialValue = await fn(initialValue, item);
      });
    return initialValue;
  };
};

now, assuming that's saved at './myAsync.js' you can do something similar to the below in an adjacent file:

...
/* your server setup here */
...
var MyAsync = require('./myAsync');
var Cat = require('./models/Cat');
var Doje = require('./models/Doje');
var example = async () => {
  var myAsync = new MyAsync();
  var doje = await Doje.findOne({ name: 'Doje', noises: [] }).save();
  var cleanParams = [];

  // FOR EACH EXAMPLE
  await myAsync.each(['bork', 'concern', 'heck'], 
    async (elem) => {
      if (elem !== 'heck') {
        await doje.update({ $push: { 'noises': elem }});
      }
    });

  var cat = await Cat.findOne({ name: 'Nyan' });

  // REDUCE EXAMPLE
  var friendsOfNyanCat = await myAsync.reduce(cat.friends,
    async (catArray, friendId) => {
      var friend = await Friend.findById(friendId);
      if (friend.name !== 'Long cat') {
        catArray.push(friend.name);
      }
    }, []);
  // Assuming Long Cat was a friend of Nyan Cat...
  assert(friendsOfNyanCat.length === (cat.friends.length - 1));
}
weixin_41568126
乱世@小熊 Minor addendum, don't forget to wrap your await/asyncs in try/catch blocks!!
接近 3 年之前 回复

Using Task, futurize, and a traversable List, you can simply do

async function printFiles() {
  const files = await getFiles();

  List(files).traverse( Task.of, f => readFile( f, 'utf-8'))
    .fork( console.error, console.log)
}

Here is how you'd set this up

import fs from 'fs';
import { futurize } from 'futurize';
import Task from 'data.task';
import { List } from 'immutable-ext';

const future = futurizeP(Task)
const readFile = future(fs.readFile)

Another way to have structured the desired code would be

const printFiles = files => 
  List(files).traverse( Task.of, fn => readFile( fn, 'utf-8'))
    .fork( console.error, console.log)

Or perhaps even more functionally oriented

// 90% of encodings are utf-8, making that use case super easy is prudent

// handy-library.js
export const readFile = f =>
  future(fs.readFile)( f, 'utf-8' )

export const arrayToTaskList = list => taskFn => 
  List(files).traverse( Task.of, taskFn ) 

export const readFiles = files =>
  arrayToTaskList( files, readFile )

export const printFiles = files => 
  readFiles(files).fork( console.error, console.log)

Then from the parent function

async function main() {
  /* awesome code with side-effects before */
  printFiles( await getFiles() );
  /* awesome code with side-effects after */
}

If you really wanted more flexibility in encoding, you could just do this (for fun, I'm using the proposed Pipe Forward operator )

import { curry, flip } from 'ramda'

export const readFile = fs.readFile 
  |> future,
  |> curry,
  |> flip

export const readFileUtf8 = readFile('utf-8')

PS - I didn't try this code on the console, might have some typos... "straight freestyle, off the top of the dome!" as the 90s kids would say. :-p

weixin_41568127
?yb? FWIW, ++1 on this. It's an elegant implementation.
2 年多之前 回复

One important caveat is: The await + for .. of method and the forEach + async way actually have different effect.

Having await inside a real for loop will make sure all async calls are executed one by one. And the forEach + async way will fire off all promises at the same time, which is faster but sometimes overwhelmed(if you do some DB query or visit some web services with volume restrictions and do not want to fire 100,000 calls at a time).

You can also use reduce + promise(less elegant) if you do not use async/await and want to make sure files are read one after another.

files.reduce((lastPromise, file) => 
 lastPromise.then(() => 
   fs.readFile(file, 'utf8')
 ), Promise.resolve()
)

Or you can create a forEachAsync to help but basically use the same for loop underlying.

Array.prototype.forEachAsync = async function(cb){
    for(let x of this){
        await cb(x);
    }
}
csdnceshi74
7*4 You can use Array.prototype.reduce in a way that uses an async function. I've shown an example in my answer: stackoverflow.com/a/49499491/2537258
2 年多之前 回复
csdnceshi58
Didn"t forge Have a look at How to define method in javascript on Array.prototype and Object.prototype so that it doesn't appear in for in loop. Also you probably should use the same iteration as native forEach - accessing indices instead of relying on iterability - and pass the index to the callback.
2 年多之前 回复
共12条数据 1 尾页
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问
相关内容推荐