reasoning is that polling with large amount of operations will be rarely
done with std.Io.Threaded. However this still provides the opportunity
to provide concurrency for any real world use cases that need it.
This commit shows a proof-of-concept direction for std.Io.VTable to go,
which is to have general support for batching, timeouts, and
non-blocking.
I'm not sure if this is a good idea or not so I'm putting it up for
scrutiny.
This commit introduces `std.Io.operate`, `std.Io.Operation`, and
implements it experimentally for `FileReadStreaming`.
In `std.Io.Threaded`, the implementation is based on poll().
This commit shows how it can be used in `std.process.run` to collect
both stdout and stderr in a single-threaded program using
`std.Threaded.Io`.
It also demonstrates how to upgrade code that was previously using
`std.Io.poll` (*not* integrated with the interface!) using concurrency.
This may not be ideal since it makes the build runner no longer support
single-threaded mode. There is still a needed abstraction for
conveniently reading multiple File streams concurrently without
io.concurrent, but this commit demonstrates that such an API can be
built on top of the new `std.Io.operate` functionality.
When `NtReadFile` returns `SUCCESS`, the APC routine still runs when
next alertable, which was previously clobbering an out of scope `done`.
Instead of adding an extra syscall to the success path, avoid all APC
side effects, allowing instant completions to return immediately.
As mlugg pointed out those race when a thread finishes an operation just
after it is canceled and then that thread to picks up another task,
resulting in these fields being potentially overwritten.
This updates fileReadStreaming on Windows to handle being alerted, and
then manage its own cancelation of the file I/O.
For now, let us refrain from putting the sync mode into the Io.File
struct, and document that to do concurrent batch operations, any Windows
file handles must be in asynchronous mode. The consequences for
violating this requirement is neither illegal behavior, nor an error,
but that concurrency is lost. In other words, deadlock might occur. This
prevents the addition of flags field.
partial revert of 2faf14200f58ee72ec3a13e894d765f59e6483a9
This tracks whether it is a file opened in synchronous mode, or
something that supports APC.
This will be needed in order to know whether concurrent batch operations
on the file should return error.ConcurrencyUnavailable, or use APC to
complete the batch.
This patch also switches to using NtCreateFile directly in
std.Io.Threaded for dirCreateFile, as well as NtReadFile for
fileReadStreaming, making it handle files opened in synchronous mode as
well as files opened in asynchronous mode.
Most places where `undefined` was previously (intentionally) passed across
function calls now use `Air.Inst.Ref.none` instead to ensure that these
`undefined` references don't accidentally outlive the `switch` logic they
belong to.
The call to `rebase` in `discardIndirect` and `discardDirect` was inappropriate. As `rebase` expects the `capacity` parameter to exclude the sliding window, this call was asking for ANOTHER `d.window_len` bytes. This was impossible to fulfill with a buffer smaller than 2*`d.window_len`, and caused [#25764](https://github.com/ziglang/zig/issues/25764).
This PR adds a basic test to do a discard (which does trigger [#25764](https://github.com/ziglang/zig/issues/25764)), and rebases only as much as is required to make the discard succeed ([or no rebase at all](https://github.com/ziglang/zig/issues/25764#issuecomment-3484716253)). That means: ideally rebase to fit `limit`, or if the buffer is too small, as much as possible.
I must say, `discardDirect` does not make much sense to me, but I replaced it anyway. `rebaseForDiscard` works fine with `d.reader.buffer.len == 0`. Let me know if anything should be changed.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/30891
Reviewed-by: Andrew Kelley <andrew@ziglang.org>
Co-authored-by: mercenary <mercenary@noreply.codeberg.org>
Co-committed-by: mercenary <mercenary@noreply.codeberg.org>
