Sep 19, 2019

Python ThreadPoolExecutor, map

import urllib.request 
from concurrent.futures import ThreadPoolExecutor
import threading

urls = [
  'http://www.python.org', 
  'http://www.python.org/about/',
  'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',
  'http://www.python.org/doc/',
  'http://www.python.org/download/',
  'http://www.python.org/getit/',
  'http://www.python.org/community/',
  'https://wiki.python.org/moin/',
]

def fun(url):
  print(url, threading.current_thread())
  r = urllib.request.urlopen(url)
  return r

# make the Pool of workers
pool = ThreadPoolExecutor(4) 

results = pool.map(fun, urls)
real_results = list(results)
print('----')
print(real_results)


Output:
http://www.python.org <Thread(ThreadPoolExecutor-0_0, started daemon 139989036611328)>
http://www.python.org/about/ <Thread(ThreadPoolExecutor-0_1, started daemon 139988956083968)>
http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html <Thread(ThreadPoolExecutor-0_2, started daemon 139988947691264)>
http://www.python.org/doc/ <Thread(ThreadPoolExecutor-0_3, started daemon 139988939298560)>
http://www.python.org/download/ <Thread(ThreadPoolExecutor-0_1, started daemon 139988956083968)>
http://www.python.org/getit/ <Thread(ThreadPoolExecutor-0_3, started daemon 139988939298560)>
http://www.python.org/community/ <Thread(ThreadPoolExecutor-0_0, started daemon 139989036611328)>
https://wiki.python.org/moin/ <Thread(ThreadPoolExecutor-0_3, started daemon 139988939298560)>
----
[<http.client.HTTPResponse object at 0x7f51be3d7910>, <http.client.HTTPResponse object at 0x7f51be3c5f90>, <http.client.HTTPResponse object at 0x7f51be3d73d0>, <http.client.HTTPResponse object at 0x7f51be3d77d0>, <http.client.HTTPResponse object at 0x7f51be3b3b90>, <http.client.HTTPResponse object at 0x7f51be3c5610>, <http.client.HTTPResponse object at 0x7f51be3e1c90>, <http.client.HTTPResponse object at 0x7f51be3d7250>]

No comments:

Post a Comment