Learning Dictionary in Python

#!/usr/local/bin/python2.7

'''
#Learning Dictionary in Python
    File name: dict_all_operations.py
    Author: Prabhath Kota
    Date: June 22, 2018
    Python Version: 2.7
'''

'''
Python Dictionaries has key value pairs
They are not ordered
They don't have any index
Keys are unique
A key can be any type of objects, for example, a number, string in Python dictionary
Values can be accessed by using key rather than the index 
We can fetch the values by using key
Case sensitive

[ 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values', 'viewitems', 'viewkeys', 'viewvalues']
'''
from copy import deepcopy
import operator

employees = {1000: {'name': 'Sahasra','country': 'India', 'age': 25}, \
1001: {'name': 'Peter','country': 'US', 'age': 21}, \
1002: {'name': 'John','country': 'US', 'age': 36}, \
1003: {'name': 'Sarayu','country': 'India', 'age': 30},\
1004: {'name': 'Akio','country': 'Japan', 'age': 60}, \
1005: {'name': 'Anand','country': 'India', 'age': 50}, \
1006: {'name': 'Vidya','country': 'India', 'age': 32}, \
1007: {'name': 'Salma','country': 'Bangladesh', 'age': 23},}
print employees

#{1000: {'country': 'India', 'age': 25, 'name': 'Sahasra'}, 1001: {'country': 'US', 'age': 21, 'name': 'Peter'}, 1002: {'country': 'US', 'age': 36, 'name': 'John'}, 1003: {'country': 'India', 'age': 30, 'name': 'Sarayu'}, 1004: {'country': 'Japan', 'age': 60, 'name': 'Akio'}, 1005: {'country': 'India', 'age': 50, 'name': 'Anand'}, 1006: {'country': 'India', 'age': 32, 'name': 'Vidya'}, 1007: {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}}

print '---------------Dictionary Keys---------------'
print employees.keys() #Get employee Ids
#[1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

print '---------------Dictionary Values---------------'
print employees.values()
#[{'country': 'India', 'age': 25, 'name': 'Sahasra'}, {'country': 'US', 'age': 21, 'name': 'Peter'}, {'country': 'US', 'age': 36, 'name': 'John'}, {'country': 'India', 'age': 30, 'name': 'Sarayu'}, {'country': 'Japan', 'age': 60, 'name': 'Akio'}, {'country': 'India', 'age': 50, 'name': 'Anand'}, {'country': 'India', 'age': 32, 'name': 'Vidya'}, {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}]

print '---------------Dictionary Access---------------'
print employees[1002]
#{'country': 'US', 'age': 36, 'name': 'John'}
employee_obj = employees[1002]
print employee_obj['name']
#John
print employee_obj['country']
#US
print employee_obj['age']
#36
#print employee_obj['salary'] #This will throw error
print employee_obj.get('salary') #Get will return None atleast, but won't throw any error

print '---------------Dictionary Check if key exists---------------'
print 1002 in employees
#True
print 2002 in employees.keys()
#False
print employees.has_key('dept')
#None

print '---------------Dictionary Add/update---------------'
print employees
employees[1008] = {'name': 'Sriya','country': 'India', 'age': 42}

print employees.keys()
#[1008, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

employees.update({1009: {'name': 'Ahmad','country': 'Bangladesh', 'age': 60}})
print employees.keys()
#[1008, 1009, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

print '---------------Dictionary Delete entry---------------'
print employees.keys()
#[1008, 1009, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

del employees[1009]

print employees.keys()
#[1008, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007]

print employees[1002]
#{'country': 'US', 'age': 36, 'name': 'John'}

del employees[1002]['age']

print employees[1002]
#{'country': 'US', 'name': 'John'}

print employees[1002].pop('country') #You need to specify key name
#US

print employees[1002]
#{'name': 'John'}

print '---'
print employees[1002].popitem() #It randomly removed any one key
#('name', 'John')

print '---'
print employees[1002]
#{}

print '---'
print len(employees)
#9
del employees[1002]

print len(employees)
#8

print '---------------Dictionary copy by ref---------------'
print employees[1008]['age']
employees_1 = employees
employees[1008]['age'] = 45
print employees[1008]['age']
print employees_1[1008]['age']
#42
#45
#45

print '---------------Dictionary shallow copy---------------'
print employees[1008]['age']
employees_1 = employees.copy()
employees[1008]['age'] = 50
print employees[1008]['age']
print employees_1[1008]['age']
#45
#50
#50

print '---------------Dictionary deep copy---------------'
print employees[1008]['age']
employees_1 = deepcopy(employees)
employees[1008]['age'] = 55
print employees[1008]['age']
print employees_1[1008]['age']
#50
#55
#50

print '---------------Dictionary loop---------------'
for key,val in employees.items():
    print key, ' ----> ', val
for key in employees:
    print key, ' ----> ', employees[key]

#1008  ---->  {'country': 'India', 'age': 55, 'name': 'Sriya'}
#1000  ---->  {'country': 'India', 'age': 25, 'name': 'Sahasra'}
#1001  ---->  {'country': 'US', 'age': 21, 'name': 'Peter'}
#1003  ---->  {'country': 'India', 'age': 30, 'name': 'Sarayu'}
#1004  ---->  {'country': 'Japan', 'age': 60, 'name': 'Akio'}
#1005  ---->  {'country': 'India', 'age': 50, 'name': 'Anand'}
#1006  ---->  {'country': 'India', 'age': 32, 'name': 'Vidya'}
#1007  ---->  {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}


print '---------------Dictionary sort by key---------------'
for key in sorted(employees):
    print key, ' ----> ', employees[key]

#1000  ---->  {'country': 'India', 'age': 25, 'name': 'Sahasra'}
#1001  ---->  {'country': 'US', 'age': 21, 'name': 'Peter'}
#1003  ---->  {'country': 'India', 'age': 30, 'name': 'Sarayu'}
#1004  ---->  {'country': 'Japan', 'age': 60, 'name': 'Akio'}
#1005  ---->  {'country': 'India', 'age': 50, 'name': 'Anand'}
#1006  ---->  {'country': 'India', 'age': 32, 'name': 'Vidya'}
#1007  ---->  {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}
#1008  ---->  {'country': 'India', 'age': 55, 'name': 'Sriya'}

print '---------------Dictionary sort by value ---------------'
try:
   #Dictionary of Dictionaries - sort of specific value    
   print sorted(employees.items(), key=lambda(x,y): y['age'])
   #[(1001, {'country': 'US', 'age': 21, 'name': 'Peter'}), (1007, {'country': 'Bangladesh', 'age': 23, 'name': 'Salma'}), (1000, {'country': 'India', 'age': 25, 'name': 'Sahasra'}), (1003, {'country': 'India', 'age': 30, 'name': 'Sarayu'}), (1006, {'country': 'India', 'age': 32, 'name': 'Vidya'}), (1005, {'country': 'India', 'age': 50, 'name': 'Anand'}), (1008, {'country': 'India', 'age': 55, 'name': 'Sriya'}), (1004, {'country': 'Japan', 'age': 60, 'name': 'Akio'})]

   #Based on Value
   x = {1: 2000, 3: 8000, 4: 2500, 2: 9000, 9:10000, 7:9500}
   print x
   #{1: 2000, 2: 9000, 3: 8000, 4: 2500, 7: 9500, 9: 10000}

   sorted_x = sorted(x.items(), key=operator.itemgetter(1))
   print sorted_x
   #[(1, 2000), (4, 2500), (3, 8000), (2, 9000), (7, 9500), (9, 10000)]

   sorted_x = sorted(x.items(), key=lambda(x,y): y)
   print sorted_x
   #[(1, 2000), (4, 2500), (3, 8000), (2, 9000), (7, 9500), (9, 10000)]
except Exception, e:
   print e    

Learning lists - all operations

#!/usr/local/bin/python2.7

'''
    File name: list_all_operations.py
    Author: Prabhath Kota
    Date: June 22, 2018
    Python Version: 2.7
'''

freedom_fighters = ["Lala Lajipati Rai", "Dadabhai Naoroji", "Rajendra Prasad", "Sarojini Naidu", "Dadabhai Naoroji", "Lal Bahadur Shastri"]
print freedom_fighters
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '---------------index---------------'
print freedom_fighters[0]
print freedom_fighters[-1]
print freedom_fighters[2:]
print freedom_fighters[:-1]
print freedom_fighters[2:4]

#Lala Lajipati Rai
#Lal Bahadur Shastri
#['Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Rajendra Prasad', 'Sarojini Naidu', 'Dadabhai Naoroji']
#['Rajendra Prasad', 'Sarojini Naidu']

print '---------------insert in index---------------'
freedom_fighters[2] = 'Subhash Chandra Bose'
print freedom_fighters
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '---------------Add Element append vs extend---------------'
print freedom_fighters
#Append - appends as a single item
freedom_fighters.append('Subhash Chandra Bose')
print freedom_fighters
freedom_fighters.pop()
freedom_fighters.append(['Subhash Chandra Bose', 'Lal Bahadur Shastri'])
print freedom_fighters
freedom_fighters.pop()
#Extend - it adds multiple values
freedom_fighters.extend(['Subhash Chandra Bose', 'Lal Bahadur Shastri'])
print freedom_fighters
freedom_fighters.pop()
freedom_fighters.pop()
print freedom_fighters

#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Subhash Chandra Bose', ['Subhash Chandra Bose', 'Lal Bahadur Shastri']]
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri', 'Subhash Chandra Bose', 'Subhash Chandra Bose', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '---------------Remove Element by value---------------'
print freedom_fighters
freedom_fighters.remove('Subhash Chandra Bose')
print freedom_fighters

#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Subhash Chandra Bose', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']

print '-----------delete by index---------------------'
print freedom_fighters
del freedom_fighters[-1] #removes index element
del freedom_fighters[1] #removes index element
print freedom_fighters

#['Lala Lajipati Rai', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Lal Bahadur Shastri']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji']

print '---------------By Ref---------------'
print freedom_fighters

#copy by Reference
freedom_fighters_1 = freedom_fighters
freedom_fighters_1.append('Bhagat Singh')
print freedom_fighters
print freedom_fighters_1

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']

print '---------------By Value -----------'
#copy by value
freedom_fighters_2 = freedom_fighters[:]
freedom_fighters_2.append('Sardar Vallabhbhai Patel')
print freedom_fighters
print freedom_fighters_2

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh', 'Sardar Vallabhbhai Patel']

print '---------------List Comprehension ---------------'

#This gives list
print [i + ' - India Salutes you' for i in freedom_fighters]

#tuple - gives generator object
generator_obj = (i + ' - India Salutes you' for i in freedom_fighters)
print generator_obj
for each in generator_obj:
print each

#['Lala Lajipati Rai - India Salutes you', 'Sarojini Naidu - India Salutes you', 'Dadabhai Naoroji - India Salutes you', 'Bhagat Singh - India Salutes you']
#<generator object <genexpr> at 0x0000000001EB41B0>
#Lala Lajipati Rai - India Salutes you
#Sarojini Naidu - India Salutes you
#Dadabhai Naoroji - India Salutes you
#Bhagat Singh - India Salutes you

print '---------------Sorted()---------------'
print freedom_fighters
print sorted(freedom_fighters)
print freedom_fighters

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']
#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']

print '---------------list.reverse() ---------------'
print freedom_fighters
print freedom_fighters.reverse() #Returns None but reverse itself
print freedom_fighters

#['Lala Lajipati Rai', 'Sarojini Naidu', 'Dadabhai Naoroji', 'Bhagat Singh']
#None
#['Bhagat Singh', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Lala Lajipati Rai']

print '---------------list.sort() ---------------'
print freedom_fighters
print freedom_fighters.sort() #Returns None but sorts itself
print freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Sarojini Naidu', 'Lala Lajipati Rai']
#None
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']

print '---------------multiply---------------'
freedom_fighters2 = freedom_fighters * 2
print freedom_fighters2
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']

print '---------------check if item exits---------------'
print freedom_fighters
print 'Bhagat Singh' in freedom_fighters
print 'Sardar Vallabhbhai Patel' in freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']
#True
#False

print '---------------Remove Duplicates---------------'
print freedom_fighters
freedom_fighters.append('Mahatma Gandhi')
freedom_fighters.append('Mahatma Gandhi')
print freedom_fighters
freedom_fighters_no_dup = []
[freedom_fighters_no_dup.append(x) for x in freedom_fighters if x not in freedom_fighters_no_dup]
print freedom_fighters_no_dup
print list(set(freedom_fighters)) #set - unordered collection data type, this kills the original order

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi']
#['Mahatma Gandhi', 'Bhagat Singh', 'Lala Lajipati Rai', 'Dadabhai Naoroji', 'Sarojini Naidu']

print '---------------Convert to tuple---------------'
print freedom_fighters
print tuple(freedom_fighters)
print list(tuple(freedom_fighters))

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#('Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi')
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']

print '---------------length of array---------------'
print freedom_fighters
print len(freedom_fighters) 

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#6

print '---------------Find count of Element---------------'
freedom_fighters.append('Bhagat Singh')
print freedom_fighters
print freedom_fighters.count('Bhagat Singh')

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Bhagat Singh']
#2

print '---------------Pop---------------'
#pop() method removes and returns the last item if index is not provided
print freedom_fighters
freedom_fighters.pop()
print freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Bhagat Singh']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']

print '---------------Insert---------------'
#When you insert(-3, 0) you are inserting 0 before index -3. 
#When it says "before" it has nothing to do with the direction of travel.
print freedom_fighters
freedom_fighters.insert(0,'Sarojini Naidu')
freedom_fighters.insert(len(freedom_fighters),'Sarojini Naidu')
print freedom_fighters
freedom_fighters.insert(-1,'Bhagat Singh')
print freedom_fighters
del freedom_fighters[0]
freedom_fighters.pop()
freedom_fighters.pop()
print freedom_fighters

#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']
#['Sarojini Naidu', 'Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Sarojini Naidu']
#['Sarojini Naidu', 'Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi', 'Bhagat Singh', 'Sarojini Naidu']
#['Bhagat Singh', 'Dadabhai Naoroji', 'Lala Lajipati Rai', 'Sarojini Naidu', 'Mahatma Gandhi', 'Mahatma Gandhi']

print '---------------Other Functions---------------'
#sorted(freedom_fighters)
#print max(freedom_fighters)
#print min(freedom_fighters)
#print sum(list1)

Perl Remove Special Characters From File

While reading some kind of log files as mentioned below, we need to get rid of these special characters.

Because of these special characters, it makes the Developers job tough :
To parse the content of a file
To convert the special characters from the file

Let us explain how can we get rid of these special characters with a simple example

test.log

^[[1;31mTest 1^[[0m
^[[1;31mTest 2^[[0m
^[[1;31mTest 3^[[0m
^[[1;31mTest 4^[[0m
^[[1;31mTest 5^[[0m  

In the above test.log file, first of all, what is displayed as ^[ is not ^ and [
But it is the ASCII ESC character, produced by Esc or Ctrl[ (the ^ notation means the Ctrl key).

We can use the following regular expression :

s/\e\[[\d;]*[a-zA-Z]//g;

Note: 
\e represents escape character in the above regular expression (substituting instead of ^[ )
We can shorten from [a-zA-Z] to just [mK], based on the requirement
You can make use of the above regular expression while parsing the file as well (line by line)

In case if you want to backup file (test.log.bak) instead of changing in the original file (test.log) then use the following :
perl -pi.bak -e 's/\e\[[\d;]*[a-zA-Z]//g' test.log

The following will remove the special chars in test.log
perl -pi -e 's/\e\[[\d;]*[a-zA-Z]//g' test.log

Output after removing

Test 1
Test 2
Test 3
Test 4
Test 5
  

' tr ' or ' y '

Removing the duplicate characters from the string:

#!F:\Perl\bin\perl -w
use strict;


#Removing the duplicate characters ('c' , 'd') but not ('e') from the string
my $val = 'abcccdddddeeeeeeeeeeeeecccccc';
print "\n Given String:", $val;

$val =~ y/cd//s; # 'y' is nothing but 'tr'

print "\nAfter :$val\n";