Skip to content

Instantly share code, notes, and snippets.

@felixpk
Last active November 8, 2019 16:39
Show Gist options
  • Select an option

  • Save felixpk/4487c99042bb7e6611fe6a5de87ce621 to your computer and use it in GitHub Desktop.

Select an option

Save felixpk/4487c99042bb7e6611fe6a5de87ce621 to your computer and use it in GitHub Desktop.
import multiprocessing as mp
from concurrent import futures
from concurrent.futures import ProcessPoolExecutor
def foo(doc):
pass
with ProcessPoolExecutor(max_workers=4, mp_context=mp.get_context('spawn')) as executor:
tasks = [executor.submit(foo, doc) for doc in all_docs]
with tqdm(total=docs_count) as pbar:
for _ in futures.as_completed(tasks):
pbar.update()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment