fs: allow correct handling of burst in fs-events with AsyncIterator

[pipobscure]

This addresses a bug in fs.watch when used as an AsyncIterator.

The issue is that when consuming the AsyncIteractor returned by fs.watch it yields a value. When using that value and in turn awaiting an asynchrounous action any events happening in the meantime will go missing. The reason is that between exiting the watch function by yielding and reentering it through the next round, the promise inside watch is already resolved. So any events generated will be duplicate resolutions of that promise and therefore ignored.

for await (let found of fs.watch('my dir')) {
  console.log(found);
  await sleep(10000); // any events happening during these 10seconds will  be missed;
}

More reaslistically than this minimal example is when the found files are actually read via await fs.readFile. Then there will be a lag. If the file events happening are happening close together, then the second one will be missed.

To fix this issue I added a queue to the watch function that new file events get pushed onto. The promise is no longer resolved with a value, but is simply the gate gating whether or not there are any events in the queue. The iterator awaits the promise and then yields the items from the queue so long as there are any. When the queue is empty and the watch is still running, then a new promise is created and awaited upon. This whould eliminate the problem entirely and one can now go asynchronous in the loop as long as one wants without missing events.

Verified that the added test fails with v24.0.0 and passes after the fix.

Based on feedback the queuing was made configurable with the maxQueue (default 2048) option determining the maximum size of the queue and the overflow option deciding to either ignore the issue or throw an Error (default: 'ignore'). To effectively get back the previous behavior one would have to pass { maxQueue: 1, overflow: 'ignore' }.

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.