gh-77714: Provide an async iterator version of as_completed - #22491

I object against the change to emphasize the async iterator API over the synchronous iterator. Since the two examples are identical except for the added keyword await in the former, it makes more sense to describe both variants as equivalent, and explain the difference carefully. The advantage of using the async version should be clear to those who need it. The disadvantage (no support in 3.12 or before) should also be clear.

I'll do that. This was biased towards deprecating the older format, knowing it could mean replacing this hybrid class with a simpler and faster async generator down the line.

I recommend against capitalizing Tasks and Futures -- depending on what loop.create_future() and loop.create_task(), they may or may not return an actual instance or a duck type equivalent. If you really mean the specific classes they should at least be put in code font, and probably cross-linked.

You're right. We use is_future to decide, which duck-types on presence of ._asyncio_future_blocking. Will lower-case these.

Maybe we can expand the example to include a meaningful result (e.g. some data read from a URL?), so there is even more clarity about how to obtain it? Then update the second example too. Or, even better (but more code) show an example where we actually need a table mapping tasks to results (which can't be done using the synchronous iterator).

I've finished the other requested doc changes, but still thinking about this one. I tried to come up with a toy example originally, but they always ended up overly verbose. With URLs, we could invent an example with aiohttp or httpx. If we want to stick to standard libraries, we could instead do something DNS-related (e.g. happy eyeballs IPv4 vs IPv6 race). Open to ideas.

Sleep sort.

from asyncio import as_completed, sleep, run
import random

async def sleep_and_return(x):
    await sleep(x)
    return x

async def sleep_sort(data):
    for coro in as_completed(sleep_and_return(x) for x in data):
        res = await coro
        print(res)

run(sleep_sort([random.random() for i in range(10)]))

The same can be used also with asynchronous iteration, but the variant that requires keeping identity of futures:

from asyncio import as_completed, sleep, ensure_future, run
import random

def sleeping_future(x):
    fut = ensure_future(sleep(x))
    fut.x = x
    return fut

async def sleep_sort(data):
    async for fut in as_completed(sleeping_future(x) for x in data):
        await fut
        print(fut.x)

run(sleep_sort([random.random() for i in range(10)]))

Again, don't capitalize task or future.

I prefer docstrings that start with a brief one-line summary followed blank line, then you can put what you have here.

Also, this sentence is very long. Try rewriting as several shorter sentences.

Is this still a problem? I don't see how asyncio/queues.py would import asyncio/tasks.py. The original code that had this is a decade old, likely the problem was solved in another way?

I still recommend moving this to the top level imports.

Maybe this should be a comment, not a docstring? (Also the use of present tense "Waits ..." makes me queasy -- IIRC we have some kind of guideline against it, preferring the imperative mood instead?)

(Also if you remove the resolve argument as I suggested above this becomes much simpler.)

I'd prefer a single-line summary here as well. And probably more equal treatment of both forms, as I recommended in the docs.

Maybe clarify that the iterator is ambidextrous (or polymorphic or whatever :-).

This could be written more compactly:

We need the ipv6_connect variable for use in the identity check below.

It would be nice if this example used the same variable names as the full example above, to emphasize how similar they are (async for -> for).

for earliest_connect in as_completed(tasks):
    reader, writer = await earliest_connect
    # ...

It would be misleading to name the synchronous for variable earliest_connect, because it is not one of ipv4_connect or ipv6_connect. If you expand # ... to the same code as in the asynchronous iteration example, it will not fail, but work incorrectly.

And yet, these two are completely equivalent:

    async for result in as_completed([ipv4_connect, ipv6_connect]):
        r, w = await result

and

    for result in as_completed([ipv4_connect, ipv6_connect]):
        r, w = await result

It's true that result represents something different, but in most cases you just want to await it, and then it's the same.

Maybe we need two examples -- one showing that if you just await the result, there's no difference; another showing that only async for can identify the task that first completed?

True. And note that in the asynchronous iteration example it could not necessary be await, it could be a synchronous operation, like result() or result.get(). await is only used to mimic the old code.

Oh, the result is already 'done' in that case? I had not realized that subtlety. Worth pointing out in the docs then!

The latter sentence seems to be missing something (at least a verb)?

I still recommend moving this to the top level imports.

As I suggested in the full docs.

Ditto.

We need the ipv6_connect variable for use in the identity check below.

It would be misleading to name the synchronous for variable earliest_connect, because it is not one of ipv4_connect or ipv6_connect. If you expand # ... to the same code as in the asynchronous iteration example, it will not fail, but work incorrectly.