Showing posts with label python_asyncIO. Show all posts
Showing posts with label python_asyncIO. Show all posts

May 19, 2020

Python 2 Vs 3


Python 2Python 3
input() may store as int, string
raw_input() stores str always
input() function was fixed in Python 3 so that it always stores the user inputs as str
print "Hi"
print("Hi")
print("Hi")
3/2 ==> floor(1.5) => 1 (defaults to floor), return int3/2 ==> 1.5
Strings default stores as AsciiStrings default stores as unicode

Unicode is a superset of ASCII and hence, can encode more characters including foreign ones.
sorted(employees.items(), key=lambda(x,y): y['age'])sorted(employees.items(), key=lambda x: x[1]['age'])
AsyncIO
Fstrings
It is recommended to use __future__ imports it if you are planning Python 3.x support for your code
xrange() - Lazy evaluationrange() - Lazy evaluation
except NameError, err:except NameError as err:
my_generator = (letter for letter in 'abcdefg')

next(my_generator)
my_generator.next()
my_generator = (letter for letter in 'abcdefg')

next(my_generator)
print 'Python', python_version()

i = 1
print 'before: i =', i
print 'comprehension: ', [i for i in range(5)]
print 'after: i =', i

Python 2.7.6
before: i = 1
comprehension: [0, 1, 2, 3, 4]
after: i = 4
Python 3.x for-loop variables don’t leak into the global namespace anymore!

print ('Python', python_version())
i = 1
print 'before: i =', i
print 'comprehension: ', [i for i in range(5)]
print 'after: i =', i

Python 3.4.1
before: i = 1
comprehension: [0, 1, 2, 3, 4]
after: i = 1
print range(3)
print type(range(3))

[0, 1, 2]
<type 'list'>
print range(3)
print type(range(3))
print(list(range(3)))

range(0, 3)
<class 'range'>
[0, 1, 2]
round(15.5) # 16.0
round(16.5) # 17.0
Bankers rounding
round(15.5) # 16
round(16.5) # 16

Mar 29, 2020

Python AsyncIO Example

import asyncio
import aiohttp
import time

async def crawl_one_url(url, session):
    get_request = session.get(url)
    print(url)
    res = await get_request
    txt = await res.text()
    get_request.close()
    return txt


async def crawl_urls(urls_to_crawl):
    session = aiohttp.ClientSession()

    work_to_do = list()
    for url in urls_to_crawl:
        work_to_do.append(crawl_one_url(url, session))
    print(*work_to_do)
    res = await asyncio.gather(*work_to_do)
    # print(res)
    await session.close()
    return res


def main():
    t0 = time.time()
    urls_to_crawl = list()
    urls_to_crawl.append('http://blog.prabhathkota.com/search/label/python')
    urls_to_crawl.append('http://blog.prabhathkota.com/search/label/perl')
    urls_to_crawl.append('http://blog.prabhathkota.com/search/label/unix')
    urls_to_crawl.append('http://blog.prabhathkota.com/search/label/aws')
    urls_to_crawl.append('http://blog.prabhathkota.com/search/label/java')
    asyncio.run(crawl_urls(urls_to_crawl))
    elapsed = time.time() - t0
    print(f"{len(urls_to_crawl)} URLS downloaded in {elapsed:.2f}")


if __name__ == '__main__':
    main()


Output:
<coroutine object crawl_one_url at 0x7f5fe36181c0> <coroutine object crawl_on
e_url at 0x7f5fe3618240> <coroutine object crawl_one_url at 0x7f5fe36182c0> <
coroutine object crawl_one_url at 0x7f5fe3618340> <coroutine object crawl_one
_url at 0x7f5fe36183c0>
http://blog.prabhathkota.com/search/label/python
http://blog.prabhathkota.com/search/label/perl
http://blog.prabhathkota.com/search/label/unix
http://blog.prabhathkota.com/search/label/aws
http://blog.prabhathkota.com/search/label/java
5 URLS downloaded in 0.48

Nov 11, 2019

Python AsyncIO

Ref: https://realpython.com/async-io-python/
  • Threading Vs Multi-Processing
    • Threading is better for I/O based tasks
    • Multi-Processing is better for CPU based tasks
    • What’s important to know about threading is that it’s better for IO-bound tasks.
  • Concurrency Vs Parallelism
    • Concurrency is when two tasks can start, run, and complete in overlapping time periods. e.g., Threading, AsyncIO
    • Parallelism is when tasks literally run at the same time, eg. multi-processing. 
  • While a CPU-bound task is characterised by the computer’s cores continually working hard from start to finish, an IO-bound job is dominated by a lot of waiting on input/output to complete.
  • Preemptive multitasking Vs Cooperative multitasking
    • OS preempts a thread forcing it to give up the use of CPU (E.g., Threading)
    • Cooperative multitasking on the other hand, the running process voluntarily gives up the CPU to other processes E.g., (AsyncIO)
  • Coroutine Vs Method/Function/Subroutine
    • Method or Function returns a value and don't remember the state between invocations
    • A coroutine is a special function that can give up control to its caller without losing its state
  • Coroutine Vs Generator
    • Generator yield back value to invoker
    • Coroutine yields control to another coroutine and can resume execution from point it gave the control
    • A generator can't accept arguments once it is started where as a coroutine can accept arguments once it started
  • AsyncIO is a single-threaded, single-process design: it uses cooperative multitasking
  • AsyncIO gives a feeling of concurrency despite using a single thread in a single process
  • Coroutines (a central feature of async IO) can be scheduled concurrently, but they are not inherently concurrent.
  • Asynchronous routines are able to “pause” while waiting on their ultimate result and let other routines run in the meantime.


import asyncio
import time

async def count_func():
    print("Line One")
    await asyncio.sleep(1) # await non-blocking call
    print("Line Two")


async def main():
    await asyncio.gather(count_func(), count_func(), count_func())


if __name__ == "__main__":
    t1 = time.time()
    asyncio.run(main())
    elapsed = time.time() - t1

    #This is supposed to take more than 3 secs
    print(f"{__file__} executed in {elapsed:0.2f} seconds.")


Output:
#####
Line One
Line One
Line One
Line Two
Line Two
Line Two
main.py executed in 1.10 seconds.