| Python 2 | Python 3 |
| input() may store as int, string raw_input() stores str always | input() function was fixed in Python 3 so that it always stores the user inputs as str |
| print "Hi" print("Hi") | print("Hi") |
| 3/2 ==> floor(1.5) => 1 (defaults to floor), return int | 3/2 ==> 1.5 |
| Strings default stores as Ascii | Strings default stores as unicode Unicode is a superset of ASCII and hence, can encode more characters including foreign ones. |
| sorted(employees.items(), key=lambda(x,y): y['age']) | sorted(employees.items(), key=lambda x: x[1]['age']) |
| AsyncIO | |
| Fstrings | |
| It is recommended to use __future__ imports it if you are planning Python 3.x support for your code | |
| xrange() - Lazy evaluation | range() - Lazy evaluation |
| except NameError, err: | except NameError as err: |
| my_generator = (letter for letter in 'abcdefg') next(my_generator) my_generator.next() | my_generator = (letter for letter in 'abcdefg') next(my_generator) |
| print 'Python', python_version() i = 1 print 'before: i =', i print 'comprehension: ', [i for i in range(5)] print 'after: i =', i Python 2.7.6 before: i = 1 comprehension: [0, 1, 2, 3, 4] after: i = 4 | Python 3.x for-loop variables don’t leak into the global namespace anymore! print ('Python', python_version()) i = 1 print 'before: i =', i print 'comprehension: ', [i for i in range(5)] print 'after: i =', i Python 3.4.1 before: i = 1 comprehension: [0, 1, 2, 3, 4] after: i = 1 |
| print range(3) print type(range(3)) [0, 1, 2] <type 'list'> | print range(3) print type(range(3)) print(list(range(3))) range(0, 3) <class 'range'> [0, 1, 2] |
| round(15.5) # 16.0 round(16.5) # 17.0 | Bankers rounding round(15.5) # 16 round(16.5) # 16 |
PySpark, BigData, SQL, Hive, AWS, Python, Unix/Linux, Shortcuts, Examples, Scripts, Perl
Showing posts with label python_asyncIO. Show all posts
Showing posts with label python_asyncIO. Show all posts
May 19, 2020
Python 2 Vs 3
Labels:
ascii,
fstrings,
input,
python,
python_2_vs_3,
python_advanced,
python_asyncIO,
python_basics,
python_interview_questions,
python_lambda,
python_sorted,
Python3,
range,
raw_input,
unicode,
xrange
Mar 29, 2020
Python AsyncIO Example
import asyncio
import aiohttp
import time
async def crawl_one_url(url, session):
get_request = session.get(url)
print(url)
res = await get_request
txt = await res.text()
get_request.close()
return txt
async def crawl_urls(urls_to_crawl):
session = aiohttp.ClientSession()
work_to_do = list()
for url in urls_to_crawl:
work_to_do.append(crawl_one_url(url, session))
print(*work_to_do)
res = await asyncio.gather(*work_to_do)
# print(res)
await session.close()
return res
def main():
t0 = time.time()
urls_to_crawl = list()
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/python')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/perl')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/unix')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/aws')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/java')
asyncio.run(crawl_urls(urls_to_crawl))
elapsed = time.time() - t0
print(f"{len(urls_to_crawl)} URLS downloaded in {elapsed:.2f}")
if __name__ == '__main__':
main()
Output:
<coroutine object crawl_one_url at 0x7f5fe36181c0> <coroutine object crawl_on
e_url at 0x7f5fe3618240> <coroutine object crawl_one_url at 0x7f5fe36182c0> <
coroutine object crawl_one_url at 0x7f5fe3618340> <coroutine object crawl_one
_url at 0x7f5fe36183c0>
http://blog.prabhathkota.com/search/label/python
http://blog.prabhathkota.com/search/label/perl
http://blog.prabhathkota.com/search/label/unix
http://blog.prabhathkota.com/search/label/aws
http://blog.prabhathkota.com/search/label/java
5 URLS downloaded in 0.48
import aiohttp
import time
async def crawl_one_url(url, session):
get_request = session.get(url)
print(url)
res = await get_request
txt = await res.text()
get_request.close()
return txt
async def crawl_urls(urls_to_crawl):
session = aiohttp.ClientSession()
work_to_do = list()
for url in urls_to_crawl:
work_to_do.append(crawl_one_url(url, session))
print(*work_to_do)
res = await asyncio.gather(*work_to_do)
# print(res)
await session.close()
return res
def main():
t0 = time.time()
urls_to_crawl = list()
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/python')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/perl')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/unix')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/aws')
urls_to_crawl.append('http://blog.prabhathkota.com/search/label/java')
asyncio.run(crawl_urls(urls_to_crawl))
elapsed = time.time() - t0
print(f"{len(urls_to_crawl)} URLS downloaded in {elapsed:.2f}")
if __name__ == '__main__':
main()
Output:
<coroutine object crawl_one_url at 0x7f5fe36181c0> <coroutine object crawl_on
e_url at 0x7f5fe3618240> <coroutine object crawl_one_url at 0x7f5fe36182c0> <
coroutine object crawl_one_url at 0x7f5fe3618340> <coroutine object crawl_one
_url at 0x7f5fe36183c0>
http://blog.prabhathkota.com/search/label/python
http://blog.prabhathkota.com/search/label/perl
http://blog.prabhathkota.com/search/label/unix
http://blog.prabhathkota.com/search/label/aws
http://blog.prabhathkota.com/search/label/java
5 URLS downloaded in 0.48
Nov 11, 2019
Python AsyncIO
Ref: https://realpython.com/async-io-python/
- Threading Vs Multi-Processing
- Threading is better for I/O based tasks
- Multi-Processing is better for CPU based tasks
- What’s important to know about threading is that it’s better for IO-bound tasks.
- Concurrency Vs Parallelism
- Concurrency is when two tasks can start, run, and complete in overlapping time periods. e.g., Threading, AsyncIO
- Parallelism is when tasks literally run at the same time, eg. multi-processing.
- While a CPU-bound task is characterised by the computer’s cores continually working hard from start to finish, an IO-bound job is dominated by a lot of waiting on input/output to complete.
- Preemptive multitasking Vs Cooperative multitasking
- OS preempts a thread forcing it to give up the use of CPU (E.g., Threading)
- Cooperative multitasking on the other hand, the running process voluntarily gives up the CPU to other processes E.g., (AsyncIO)
- Coroutine Vs Method/Function/Subroutine
- Method or Function returns a value and don't remember the state between invocations
- A coroutine is a special function that can give up control to its caller without losing its state
- Coroutine Vs Generator
- Generator yield back value to invoker
- Coroutine yields control to another coroutine and can resume execution from point it gave the control
- A generator can't accept arguments once it is started where as a coroutine can accept arguments once it started
- AsyncIO is a single-threaded, single-process design: it uses cooperative multitasking
- AsyncIO gives a feeling of concurrency despite using a single thread in a single process
- Coroutines (a central feature of async IO) can be scheduled concurrently, but they are not inherently concurrent.
- Asynchronous routines are able to “pause” while waiting on their ultimate result and let other routines run in the meantime.
import asyncio
import time
async def count_func():
print("Line One")
await asyncio.sleep(1) # await non-blocking call
print("Line Two")
async def main():
await asyncio.gather(count_func(), count_func(), count_func())
if __name__ == "__main__":
t1 = time.time()
asyncio.run(main())
elapsed = time.time() - t1
#This is supposed to take more than 3 secs
print(f"{__file__} executed in {elapsed:0.2f} seconds.")
Output:
#####
Line One
Line One
Line One
Line Two
Line Two
Line Two
main.py executed in 1.10 seconds.
Subscribe to:
Posts (Atom)