Prabhath Kota

Jun 23, 2018

Learning Dictionary in Python

#!/usr/local/bin/python2.7

'''
#Learning Dictionary in Python
File name: dict_all_operations.py
Author: Prabhath Kota
Date: June 22, 2018
Python Version: 2.7
'''

'''
Python Dictionaries has key value pairs
They are not ordered
They don't have any index
Keys are unique
A key can be any type of objects, for example, a number, string in Python dictionary
Values can be accessed by using key rather than the index
We can fetch the values by using key
Case sensitive

[ 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values', 'viewitems', 'viewkeys', 'viewvalues']
'''
from copy import deepcopy
import operator

employees = {1000: {'name': 'Sahasra','country': 'India', 'age': 25}, \
1001: {'name': 'Peter','country': 'US', 'age': 21}, \
1002: {'name': 'John','country': 'US', 'age': 36}, \
1003: {'name': 'Sarayu','country': 'India', 'age': 30},\
1004: {'name': 'Akio','country': 'Japan', 'age': 60}, \
1005: {'name': 'Anand','country': 'India', 'age': 50}, \
1006: {'name': 'Vidya','country': 'India', 'age': 32}, \
1007: {'name': 'Salma','country': 'Bangladesh', 'age': 23},}
print employees

#{1000: {'country': 'India', 'age': 25, 'name': 'Sahasra'}, 1001: {'country': 'US', 'age': 21, 'name': 'Peter'}, 1002: {'country': 'US', 'age': 36, 'name': 'John'}, 1003: {'country': 'India', 'age': 30, 'name': 'Sarayu'}, 1004: {'country': 'Japan', 'age': 60, 'name': 'Akio'}, 1005: {'country': 'India', 'age': 50, 'name': 'Anand'}, 1006: {'country': 'India', 'age': 32, 'name': 'Vidya'}, 1007: {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}}

print '---------------Dictionary Keys---------------'
print employees.keys() #Get employee Ids
#[1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

print '---------------Dictionary Values---------------'
print employees.values()
#[{'country': 'India', 'age': 25, 'name': 'Sahasra'}, {'country': 'US', 'age': 21, 'name': 'Peter'}, {'country': 'US', 'age': 36, 'name': 'John'}, {'country': 'India', 'age': 30, 'name': 'Sarayu'}, {'country': 'Japan', 'age': 60, 'name': 'Akio'}, {'country': 'India', 'age': 50, 'name': 'Anand'}, {'country': 'India', 'age': 32, 'name': 'Vidya'}, {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}]

print '---------------Dictionary Access---------------'
print employees[1002]
#{'country': 'US', 'age': 36, 'name': 'John'}
employee_obj = employees[1002]
print employee_obj['name']
#John
print employee_obj['country']
#US
print employee_obj['age']
#36
#print employee_obj['salary'] #This will throw error
print employee_obj.get('salary') #Get will return None atleast, but won't throw any error

print '---------------Dictionary Check if key exists---------------'
print 1002 in employees
#True
print 2002 in employees.keys()
#False
print employees.has_key('dept')
#None

print '---------------Dictionary Add/update---------------'
print employees
employees[1008] = {'name': 'Sriya','country': 'India', 'age': 42}

print employees.keys()
#[1008, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

employees.update({1009: {'name': 'Ahmad','country': 'Bangladesh', 'age': 60}})
print employees.keys()
#[1008, 1009, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

print '---------------Dictionary Delete entry---------------'
print employees.keys()
#[1008, 1009, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

del employees[1009]

print employees.keys()
#[1008, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

print employees[1002]
#{'country': 'US', 'age': 36, 'name': 'John'}

del employees[1002]['age']

print employees[1002]
#{'country': 'US', 'name': 'John'}

print employees[1002].pop('country') #You need to specify key name
#US

print employees[1002]
#{'name': 'John'}

print '---'
print employees[1002].popitem() #It randomly removed any one key
#('name', 'John')

print '---'
print employees[1002]
#{}

print '---'
print len(employees)
#9
del employees[1002]

print len(employees)
#8

print '---------------Dictionary copy by ref---------------'
print employees[1008]['age']
employees_1 = employees
employees[1008]['age'] = 45
print employees[1008]['age']
print employees_1[1008]['age']
#42
#45
#45

print '---------------Dictionary shallow copy---------------'
print employees[1008]['age']
employees_1 = employees.copy()
employees[1008]['age'] = 50
print employees[1008]['age']
print employees_1[1008]['age']
#45
#50
#50

print '---------------Dictionary deep copy---------------'
print employees[1008]['age']
employees_1 = deepcopy(employees)
employees[1008]['age'] = 55
print employees[1008]['age']
print employees_1[1008]['age']
#50
#55
#50

print '---------------Dictionary loop---------------'
for key,val in employees.items():
print key, ' ----> ', val
for key in employees:
print key, ' ----> ', employees[key]

#1008 ----> {'country': 'India', 'age': 55, 'name': 'Sriya'}
#1000 ----> {'country': 'India', 'age': 25, 'name': 'Sahasra'}
#1001 ----> {'country': 'US', 'age': 21, 'name': 'Peter'}
#1003 ----> {'country': 'India', 'age': 30, 'name': 'Sarayu'}
#1004 ----> {'country': 'Japan', 'age': 60, 'name': 'Akio'}
#1005 ----> {'country': 'India', 'age': 50, 'name': 'Anand'}
#1006 ----> {'country': 'India', 'age': 32, 'name': 'Vidya'}
#1007 ----> {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}

print '---------------Dictionary sort by key---------------'
for key in sorted(employees):
print key, ' ----> ', employees[key]

#1000 ----> {'country': 'India', 'age': 25, 'name': 'Sahasra'}
#1001 ----> {'country': 'US', 'age': 21, 'name': 'Peter'}
#1003 ----> {'country': 'India', 'age': 30, 'name': 'Sarayu'}
#1004 ----> {'country': 'Japan', 'age': 60, 'name': 'Akio'}
#1005 ----> {'country': 'India', 'age': 50, 'name': 'Anand'}
#1006 ----> {'country': 'India', 'age': 32, 'name': 'Vidya'}
#1007 ----> {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}
#1008 ----> {'country': 'India', 'age': 55, 'name': 'Sriya'}

print '---------------Dictionary sort by value ---------------'
try:
#Dictionary of Dictionaries - sort of specific value
print sorted(employees.items(), key=lambda(x,y): y['age'])
#[(1001, {'country': 'US', 'age': 21, 'name': 'Peter'}), (1007, {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}), (1000, {'country': 'India', 'age': 25, 'name': 'Sahasra'}), (1003, {'country': 'India', 'age': 30, 'name': 'Sarayu'}), (1006, {'country': 'India', 'age': 32, 'name': 'Vidya'}), (1005, {'country': 'India', 'age': 50, 'name': 'Anand'}), (1008, {'country': 'India', 'age': 55, 'name': 'Sriya'}), (1004, {'country': 'Japan', 'age': 60, 'name': 'Akio'})]

#Based on Value
x = {1: 2000, 3: 8000, 4: 2500, 2: 9000, 9:10000, 7:9500}
print x
#{1: 2000, 2: 9000, 3: 8000, 4: 2500, 7: 9500, 9: 10000}

sorted_x = sorted(x.items(), key=operator.itemgetter(1))
print sorted_x
#[(1, 2000), (4, 2500), (3, 8000), (2, 9000), (7, 9500), (9, 10000)]

sorted_x = sorted(x.items(), key=lambda(x,y): y)
print sorted_x
#[(1, 2000), (4, 2500), (3, 8000), (2, 9000), (7, 9500), (9, 10000)]
except Exception, e:
print e

Jun 22, 2018

Learning Tuples in Python

#!/usr/local/bin/python2.7

'''
#Learning Tuples
File name: tuple_all_operations.py
Author: Prabhath Kota
Date: June 22, 2018
Python Version: 2.7
'''

'''
Tuples are immutable,means not changable
Tuples have no append or extend method.
Elements in Tuples cannot be removed from a tuple.
You can only find elements in a tuple, since this doesn not change the tuple.
You can also use the in operator to check if an element exists in the tuple.
Tuples are faster than lists. If you have a constant set of values, use a tuple instead of a list.
It makes your code safer if you 'write-protect' data that does not need to be changed.
'''

freedom_fighters = ("Lala Lajipati Rai", "Dadabhai Naoroji", "Rajendra Prasad", "Sarojini Naidu", "Dadabhai Naoroji", "Lal Bahadur Shastri")
print freedom_fighters
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')

print '---------------index---------------'
print freedom_fighters[0]
print freedom_fighters[-1]
print freedom_fighters[2:]
print freedom_fighters[:-1]
print freedom_fighters[2:4]

#Lala Lajipati Rai
#Lal Bahadur Shastri
#('Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji')
#('Rajendra Prasad', 'Sarojini Naidu')

print '---------------copy---------------'
print freedom_fighters
freedom_fighters_1 = freedom_fighters
freedom_fighters_1 += ("Bhagat Singh",)
print freedom_fighters
print freedom_fighters_1

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Bhagat Singh')

print '---------------List Comprehension ---------------'
print [i + ' India saluets you' for i in freedom_fighters]
genrator_obj = (i + ' India saluets you' for i in freedom_fighters)
print genrator_obj
for each in genrator_obj:
print each

#['Lala Lajipati Rai India saluets you', 'Dadabhai Naoroji India saluets you', 'Rajendra Prasad India saluets you', 'Sarojini Naidu India saluets you', 'Dadabhai Naoroji India saluets you', 'Lal Bahadur Shastri India saluets you']
#<generator object <genexpr> at 0x0000000001DC41B0>
#Lala Lajipati Rai India saluets you
#Dadabhai Naoroji India saluets you
#Rajendra Prasad India saluets you
#Sarojini Naidu India saluets you
#Dadabhai Naoroji India saluets you
#Lal Bahadur Shastri India saluets you

print '---------------multiply---------------'
freedom_fighters2 = freedom_fighters * 2
print freedom_fighters2
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')

print '---------------check if item exits---------------'
print freedom_fighters
print 'Bhagat Singh' in freedom_fighters
print 'Sarojini Naidu' in freedom_fighters

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#False
#True

print '---------------Remove Duplicates---------------'
freedom_fighters3 = freedom_fighters * 2
print freedom_fighters3
#print set(freedom_fighters3)
freedom_fighters_no_dup =[]
[freedom_fighters_no_dup.append(x) for x in freedom_fighters if x not in freedom_fighters_no_dup]
print tuple(freedom_fighters_no_dup)
print tuple(set(freedom_fighters3)) #set - unordered collection data type, this kills the original order

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Lal Bahadur Shastri')
#('Rajendra Prasad', 'Sarojini Naidu', 'Lala Lajipati Rai', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')

print '---------------Convert to list---------------'
print freedom_fighters
print list(freedom_fighters)
print tuple(list(freedom_fighters))

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')

print '---------------length of array---------------'
print freedom_fighters
print len(freedom_fighters)

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#6

print '---------------Default multile values assingn---------------'
#Python Default assigns to tuple
freedom_fighters_2 = 'Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Lala Lajipati Rai'
print freedom_fighters_2
print type(freedom_fighters_2)

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Lala Lajipati Rai')
#<type 'tuple'>

print '---------------Default multile values assingn---------------'
print freedom_fighters
print freedom_fighters.count('Lala Lajipati Rai')
freedom_fighters += ('Lala Lajipati Rai',)
print freedom_fighters.index('Sarojini Naidu')
print freedom_fighters
print freedom_fighters.count('Lala Lajipati Rai')

#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri')
#1
#3
#('Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Lala Lajipati Rai')
#2

Html5 - How to disable video download

Html5 - How disable video download

Use controls controlsList="nodownload" to disable download
Use oncontextmenu="return false;" to avoid right click and they by avoid save as option

Sample Snippet

<video width="512" height="380" controls controlsList="nodownload" oncontextmenu="return false;">
<source data-src="mov_bbb.ogg" type="video/mp4">
</video>

Learning lists - all operations

#!/usr/local/bin/python2.7

'''
File name: list_all_operations.py
Author: Prabhath Kota
Date: June 22, 2018
Python Version: 2.7
'''

freedom_fighters = ["Lala Lajipati Rai", "Dadabhai Naoroji", "Rajendra Prasad", "Sarojini Naidu", "Dadabhai Naoroji", "Lal Bahadur Shastri"]
print freedom_fighters
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '---------------index---------------'
print freedom_fighters[0]
print freedom_fighters[-1]
print freedom_fighters[2:]
print freedom_fighters[:-1]
print freedom_fighters[2:4]

#Lala Lajipati Rai
#Lal Bahadur Shastri
#['Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji']
#['Rajendra Prasad', 'Sarojini Naidu']

print '---------------insert in index---------------'
freedom_fighters[2] = 'Subhash Chandra Bose'
print freedom_fighters
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '---------------Add Element append vs extend---------------'
print freedom_fighters
#Append - appends as a single item
freedom_fighters.append('Subhash Chandra Bose')
print freedom_fighters
freedom_fighters.pop()
freedom_fighters.append(['Subhash Chandra Bose', 'Lal Bahadur Shastri'])
print freedom_fighters
freedom_fighters.pop()
#Extend - it adds multiple values
freedom_fighters.extend(['Subhash Chandra Bose', 'Lal Bahadur Shastri'])
print freedom_fighters
freedom_fighters.pop()
freedom_fighters.pop()
print freedom_fighters

#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Subhash Chandra Bose', ['Subhash Chandra Bose', 'Lal Bahadur Shastri']]
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Subhash Chandra Bose', 'Subhash Chandra Bose', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '---------------Remove Element by value---------------'
print freedom_fighters
freedom_fighters.remove('Subhash Chandra Bose')
print freedom_fighters

#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '-----------delete by index---------------------'
print freedom_fighters
del freedom_fighters[-1] #removes index element
del freedom_fighters[1] #removes index element
print freedom_fighters

#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji']

print '---------------By Ref---------------'
print freedom_fighters

#copy by Reference
freedom_fighters_1 = freedom_fighters
freedom_fighters_1.append('Bhagat Singh')
print freedom_fighters
print freedom_fighters_1

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']

print '---------------By Value -----------'
#copy by value
freedom_fighters_2 = freedom_fighters[:]
freedom_fighters_2.append('Sardar Vallabhbhai Patel')
print freedom_fighters
print freedom_fighters_2

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh', 'Sardar Vallabhbhai Patel']

print '---------------List Comprehension ---------------'

#This gives list
print [i + ' - India Salutes you' for i in freedom_fighters]

#tuple - gives generator object
generator_obj = (i + ' - India Salutes you' for i in freedom_fighters)
print generator_obj
for each in generator_obj:
print each

#['Lala Lajipati Rai - India Salutes you', 'Sarojini Naidu - India Salutes you', 'Dadabhai Naoroji - India Salutes you', 'Bhagat Singh - India Salutes you']
#<generator object <genexpr> at 0x0000000001EB41B0>
#Lala Lajipati Rai - India Salutes you
#Sarojini Naidu - India Salutes you
#Dadabhai Naoroji - India Salutes you
#Bhagat Singh - India Salutes you

print '---------------Sorted()---------------'
print freedom_fighters
print sorted(freedom_fighters)
print freedom_fighters

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']

print '---------------list.reverse() ---------------'
print freedom_fighters
print freedom_fighters.reverse() #Returns None but reverse itself
print freedom_fighters

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#None
#['Bhagat Singh', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Lala Lajipati Rai']

print '---------------list.sort() ---------------'
print freedom_fighters
print freedom_fighters.sort() #Returns None but sorts itself
print freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Lala Lajipati Rai']
#None
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']

print '---------------multiply---------------'
freedom_fighters2 = freedom_fighters * 2
print freedom_fighters2
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']

print '---------------check if item exits---------------'
print freedom_fighters
print 'Bhagat Singh' in freedom_fighters
print 'Sardar Vallabhbhai Patel' in freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']
#True
#False

print '---------------Remove Duplicates---------------'
print freedom_fighters
freedom_fighters.append('Mahatma Gandhi')
freedom_fighters.append('Mahatma Gandhi')
print freedom_fighters
freedom_fighters_no_dup = []
[freedom_fighters_no_dup.append(x) for x in freedom_fighters if x not in freedom_fighters_no_dup]
print freedom_fighters_no_dup
print list(set(freedom_fighters)) #set - unordered collection data type, this kills the original order

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi']
#['Mahatma Gandhi', 'Bhagat Singh', 'Lala Lajipati Rai', 'Dadabhai Naoroji', 'Sarojini Naidu']

print '---------------Convert to tuple---------------'
print freedom_fighters
print tuple(freedom_fighters)
print list(tuple(freedom_fighters))

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#('Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi')
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']

print '---------------length of array---------------'
print freedom_fighters
print len(freedom_fighters)

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#6

print '---------------Find count of Element---------------'
freedom_fighters.append('Bhagat Singh')
print freedom_fighters
print freedom_fighters.count('Bhagat Singh')

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Bhagat Singh']
#2

print '---------------Pop---------------'
#pop() method removes and returns the last item if index is not provided
print freedom_fighters
freedom_fighters.pop()
print freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Bhagat Singh']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']

print '---------------Insert---------------'
#When you insert(-3, 0) you are inserting 0 before index -3.
#When it says "before" it has nothing to do with the direction of travel.
print freedom_fighters
freedom_fighters.insert(0,'Sarojini Naidu')
freedom_fighters.insert(len(freedom_fighters),'Sarojini Naidu')
print freedom_fighters
freedom_fighters.insert(-1,'Bhagat Singh')
print freedom_fighters
del freedom_fighters[0]
freedom_fighters.pop()
freedom_fighters.pop()
print freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#['Sarojini Naidu', 'Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Sarojini Naidu']
#['Sarojini Naidu', 'Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Bhagat Singh', 'Sarojini Naidu']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']

print '---------------Other Functions---------------'
#sorted(freedom_fighters)
#print max(freedom_fighters)
#print min(freedom_fighters)
#print sum(list1)

Jun 10, 2018

AWS Cognito SDK using JavaScript

Here is my first script for signup on AWS Cognito using Javascript SDK

AWS Cognito SDK:

https://github.com/amazon-archives/amazon-cognito-identity-js/tree/master/dist

Script:

<!DOCTYPE html>
<html>
<head lang="en">
<meta charset="UTF-8">
</head>
<body>
<script type='text/javascript' src="aws-cognito-sdk.js"></script>
<script type='text/javascript' src="amazon-cognito-identity.js"></script>
<script>
var data = {
UserPoolId: 'us-east-XXXXXXXX', // Insert your user pool id
ClientId: 'XXXXXXXXXX' // Insert your app client id
};
var userPool = new AmazonCognitoIdentity.CognitoUserPool(data);
</script>
<fieldset>
<legend>Cognito Sign Up User Demo</legend>
User name: <input type="text" id="username" placeholder="Enter user name...">
<br>
<br>
Password: <input type="text" id="password" placeholder="Enter password...">
<br>
<br>
<div style="width:500px;">
<button id="signupUser">Sign Up User</button>
</div>
<ul id="signupUserResults"></ul>
</fieldset>
<script>
var attributeList = [];
document.getElementById('signupUser').addEventListener('click', function () {
userPool.signUp(document.getElementById('username').value, document.getElementById('password').value,
attributeList, null,
function (err, result) {
if (err) {
alert(err);
return;
}
document.getElementById('signupUserResults').innerHTML = "Results: " + JSON.stringify(
result.user, null, 2);
cognitoUser = result.user;
console.log(cognitoUser);
});
});
</script
</body>
</html>

AWS re:Invent 2014 | (SEC304) Bring Your Own Identities – Federating Acc...

Apr 21, 2018

How to Detect User Idle Time or Inactivity in Acess logs

How to Detect User Idle Time or Inactivity in Acess logs
Requirement:

Read access log
Find the top most idle time(s) between the requests

Script

import itertools
import datetime
import logging

fo = open("access_log_time", "r+")
print "Name of the file: ", fo.name

lst = fo.readlines()
print len(lst)

def diff_date(x, y):
diff=0
try:
x = x.strip()
y = y.strip()
d1 = datetime.datetime.strptime(x, '%d/%b/%Y:%H:%M:%S')
d2 = datetime.datetime.strptime(y, '%d/%b/%Y:%H:%M:%S')
diff = (d2 - d1).total_seconds()
print '-------'
print diff
print x
print y
except Exception, e:
logging.error(e)
return int(diff)

#zip Vs izip
#zip computes all the list at once, izip computes the elements only when requested.
#One important difference is that 'zip' returns an actual list, 'izip' returns an 'izip #object', which is not a list and does not support list-specific features

res= [diff_date(x,y) for x, y in itertools.izip (lst, lst[1:])]
print sorted(res, reverse=True)
#print res

View number of requests by time from Apache access log

Overall requests in an hour

grep "18/Apr/2018:11" /var/log/httpd/access_log | wc -l

Overall requests in a minute

grep "18/Apr/2018:11:05" /var/log/httpd/access_log | wc -l

Overall requests in a minute

grep "18/Apr/2018:11:05:10" /var/log/httpd/access_log | wc -l

Overall requests by sec in an hour (group by sec)

grep "18/Apr/2018:11" /var/log/httpd/access_log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'

Find Sum of RSS memory in ps command in linux/unix

In KB

ps aux | awk 'BEGIN {sum=0} {sum +=$6} END {print sum}'

In MB

ps aux | awk 'BEGIN {sum=0} {sum +=$6} END {print sum/1024}'

In GB

ps aux | awk 'BEGIN {sum=0} {sum +=$6} END {print sum/1024/1024}'

How to find free memory available

cat /proc/meminfo
using free command

In KB

free

In MB

free -m

In GB

free -g

I observed

Sum of RSS memory in ps less than memory actually used
Total used memory a lot higher than sum of RSS
Reason

The Linux kernel will use available memory for disk caching

Apr 9, 2018

AWS ELB

Classic ELB
    Supported Protocols
        HTTP, HTTPS (Secure HTTP), SSL (Secure TCP) and TCP protocols
    TCP Ports
        [EC2-VPC] 1-65535
        [EC2-Classic] 25, 80, 443, 465, 587, 1024-65535
    IPv6 support
        Each Classic Load Balancer has an associated IPv4, IPv6, and dualstack (both IPv4 and IPv6) DNS name.
        IPv6 is not supported in VPC. You can use an Application Load Balancer for native IPv6 support in VPC.
    Cross-Zone Load Balancing
        This option distributes traffic evenly across all your back-end instances in all Availability Zones.
        This reduces to maintain equivalent no of instances in each enabled AZ (But it is recommended to maintain to same no of instances in each AZ for higher fault tolerance)
        This option is enabled by default in AWS console
        This option is disabled by default in AWS API/CLI
    Can I privately access Elastic Load Balancing APIs from my Amazon Virtual Private Cloud (VPC) without using public IPs?
        Yes, you can privately access Elastic Load Balancing APIs from your Amazon Virtual Private Cloud (VPC) by creating VPC Endpoints


Application ELB
    Supported Protocols
        HTTP, HTTPS (Secure HTTP)
    TCP ports
        1-65535
    Can I convert my Classic Load Balancer to an Application Load Balancer (and vice versa)? - No
    Can I migrate to Application Load Balancer from Classic Load Balancer? - Yes
    Can I use an Application Load Balancer as a Layer-4 load balancer?
        No. If you need Layer-4 features, you should use Network Load Balancer.
    Is IPv6 supported with an Application Load Balancer? - Yes
    Can I associate multiple certificates for the same domain to a secure listener?
        Yes, you can associate multiple certificates for the same domain to a secure listener. For example, you can associate
        (a) ECDSA and RSA certificates
        (b) Certificates with different key sizes (e.g. 2K and 4K) for SSL/TLS certificates
        (c) Single-Domain, Multi-Domain (SAN) and Wildcard certificates

Network ELB
    Can I create a TCP (Layer 4) listener for my Network Load Balancer?
        Yes. Network Load Balancers support only TCP (Layer 4) listeners.
    Network Load Balancer Vs TCP listener on a Classic Load Balancer?
        Network Load Balancer preserves the source IP of the client which in the Classic Load Balancer is not preserved.
        Customers can use proxy protocol with Classic Load Balancer to get the source IP.
        Network Load Balancer automatically provides a static IP per Availability Zone to the load balancer and also enables assigning an Elastic IP to the load balancer per Availability Zone. This is not supported with Classic Load Balancer.
        Classic Load Balancer provides SSL termination that is not available with Network Load Balancer.

Apr 8, 2018

AWS EBS

EBS Encryption support
EBS Encryption is supported in all EBS volume types
But not all EC2 instances support encryption

EBS Volume/Snapshot - Encryption keys
KMS - AWS Key management service
CMKs - Customer master keys
When you encrypt first EBS volume, AWS KMS creates default CMS key
After that, each newly encrypted volume is encrypted with a unique/separate AES256 bits encryption key

Sharing EBS Snapshots
Only account owner can create volume from snapshots
Encrypted snapshots cannot be shared, only un-encrypted are meant to be shared
But an Encrypted snapshot can be shared to a selected AWS account id, by making them private (with cross-account permissions)
Account A - with Key 1, has shared encrypted snapshot with Account B, Account B owner needs to create a copy of snapshot with his own key(Key 2) & create volumes
AWS will not allow you to share snapshots encrypted using default CMK key ****
Snapshots are stored in S3 (unknown location to us)
S3 SSE protects snapshot data in transit

Copy Snapshots
Copy snapshot to another to encrypt or to another region
user defined tags are NOT copied from original
5 copy requests per account in parallel

Instance Backed AMI vs EBS Backed AMI
When you create EC2 AMI, AWS automatically EBS volume(s) & Intance root volume snapshots are created
When snapshots attached to AMI, you cannot delete snapshots, you need to de-register AMI & delete snapshots
1) Instance Backed AMI
You need to register AMI (in Amazon market place)
You need to specify S3 bucket for storing snapshots
2) EBS Backed AMI
Registration of AMI happens automatically
You don't need to specify S3 bucket
When creating EMI, stop instance to ensure data consistancy and integrity
EBS Snapshots are point in time
How to ensure EBS data consistancy (stop EC2 instance is best)
1) Pause I/O operations on EBS from EC2
2) Unmount EBS volumes, create snapshot & mount again

RAID in EBS (Redundant array of independent disks)
Increase I/O performance/throughput of EC2
using EBS optimized EC2 instances
use RAID array of EBS volumes
RAID array is collection of multiple EBS volumes
Make sure EC2 max bandwidth >= Total I/O of EBS volume (or RAID)
RAID is not meant for root/boot volumes of EC2
RAID 0 - stripping
Fastest of all RAID types
Distributing data to be written over array of disks in parallel (without redundancy) - faster
If data fails in one volume, whole array of EBS volumes gets corrupted
RAID 1 - Mirroring
Cares about redundancy but not faster
Not able achieve I/O performance/throughput
RAID 10 - It has benefits of both RAID0 & RAID1

EBS Practice scenarios

non-encrypt to encrypt
copy ebs to another AZ
make ebs volume public/private
Encryption types
customer specific encryption
play in CMK & other encrypt patterns - customer managed keys
copy snapshot to another region
copy snapshot to another AWS account
attach to EC2 & mount
increase/decrease EBS volume size
Root volume encrypt (work around)
Do I need to turn off EC2 while taking backup?

Apr 7, 2018

How to encrypt an existing ebs volume of an EC2 instance

How to encrypt an existing ebs volume of an EC2 instance ?

Take snapshot of existing ebs volume
copy snapshot of above to new encrypted snapshot (using encryption)
create volume from the encrypted snapshot (in the same us-east-1a/us-east-1b/us-east-1c availability zone as of EC2)
stop ec2 instance
detach existing unencrypted ebs volume
attach new encrypted ebs volume to the ec2 instance
start the instance again

Assume you have an non-encrypted EBS volume attached to EC2 instance
Create an EBS volume with encrypt option
Attach encrypted EBS volume to EC2 (in addition to the existing non-encrypted EBS volume)
Now EC2, 2 EBS volumes are under a single AZ say us-east-1a
1st EBS volume mounted to /opt/ebs1 -> non-encrypted EBS volume
2nd EBS volume mounted to /opt/ebs2 -> Encrypted EBS volume
Now copy content from non-encrypted EBS volume to Encrypted EBS volume
Once done, detach non-encrypted EBS volume from EC2 instance

Note:

EBS volumes are limited to a specific availability zone
Snapshots are limited to a specific region
EC2 and EBS volumes attached to the EC2 instance must be in the same AZ
However you can copy snapshots across regions

How to make an Amazon EBS Volume Available for Use - mount

How to make an Amazon EBS Volume Available for Use
Please find the steps to mount EBS volume to Ec2 instance

Attach to EC2 intance
Login to EC2 instance
lsblk - gives list of volumes

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 10G 0 disk

sudo file -s /dev/xvdf #for new volumes (it shows as data)

/dev/xvdf: data

sudo file -s /dev/xvda1 #for existing

/dev/xvda1: Linux rev 1.0 ext4 filesystem data, UUID=XXXXX-XXXX-XXXXX-XXXX-XXXXXX (needs journal recovery) (extents) (large files) (huge files)

cd ~
mkdir ebsvolume
sudo mkfs -t ext4 /dev/xvdf
sudo mount /dev/xvdf ebsvolume
df -h
lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 10G 0 disk /root/ebsvolume

Note:

EC2 instance, EBS volumes must be in a same AZ

Apr 5, 2018

python list vs tuple

t1 = [1,2,3,4]
t2 = t1
t2.append(5)
print 'T1: ' + str(t1)
print 'T2: ' + str(t2)

print '------------------'

t1 = (1,2,3,4)
t2 = t1
t2 = t1 + (5,)
print 'T1: ' + str(t1)
print 'T2: ' + str(t2)

Output:
T1: [1, 2, 3, 4, 5]
T2: [1, 2, 3, 4, 5]
------------------
T1: (1, 2, 3, 4)
T2: (1, 2, 3, 4, 5)

Reason:
Tuples are immutable and not supposed to be changed
Lists are mutable and are supposed to be changed