Apr 5, 2018

get last billing date of previous month

relm.py
import datetime
from dateutil import relativedelta
import sys

try:
   input_date = sys.argv[1]
   input_date = datetime.datetime.strptime(input_date, "%Y-%m-%d").date()
   #This gives always last date of the month - relativedelta.relativedelta(day=31)
   prev_month = input_date - relativedelta.relativedelta(months=1) - relativedelta.relativedelta(day=31)
   print prev_month
except Exception, e:
   print e


Output:
python relm.py 2018-02-27 #2018-01-31
python relm.py 2018-12-28 #2018-11-30
python relm.py 2018-01-28 #2017-12-31
python relm.py 2018-02-31  #day is out of range for month

Jul 20, 2015

Jquery Examples

MAP
$.map([10,20,30], function(n,i) { return n+i;});   //[10, 21, 32]

Jquery Reverse Order of  Child Elements (ul & li):
var list = $('ul');
var listItems = list.children('li');
list.append(listItems.get().reverse());

$('ul li')[0]                       //<li>AAA</li>
$('ul li')[0].innerHTML  //AAA


$("ul li").eq(7).css("border", "1px solid #000000");  //Applies style to 8th element

Apply Style to Even & Odd rows:
$("ul  li:even").addClass("even");
$("ul  li:odd").addClass("odd");
(or)

$("ul li").each(function(i) {
if (i % 2 == 1)
{
    $(this).addClass("odd");    //$(this).css("background-color", 'red')
}
else
{
    $(this).addClass("even");   //$(this).css("background-color", 'yellow')
}
});

if ($('#elem').is(':hidden')) {
   // Do something conditionally
}

$('p:visible').hide();  // Hiding only elements that are currently visible
$('ul').siblings().length   //Siblings of <ul> element

To Get Immediate Children
<a href="/category">Category</a>
<ul id="nav">
<li><a href="#anchor1">Anchor 1</a></li>
<li><a href="#anchor2">Anchor 2</a></li>
<li><span><a href="#anchor3">Anchor 3</a></span></li>
</ul>

$('li > a').length => 2 (only in immediate direct children)
$('li a').length => 3 (all anchor tags )

$('li').children()
or
$('li > *')
or
$('li').find('> *')

Stop Event Bubbling:
return false; => event.stopPropagation(); + event.preventDefault();
event.stopImmediatePropagation(); => stops immediately without executing handlers of same element.
event.stopPropagation(); => stops executing handlers of parent elements, but executes that of same element

end()
You can return to the previous set of DOM elements that were selected before using a destructive method.

   <p>text 1 </p>
    <p class="test_class">text 2
        <span>AAA</span>
    </p>
    <p>text 3 </p>
   
    <script type="text/JavaScript"
    src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
    <script type="text/JavaScript">
        console.log($('p').filter('.test_class').length); // 1
        console.log($('p').filter('.test_class').end().length); // 3
        console.log($('p').filter('.test_class').find('span').end().end().length); // 3
    </script>

filter() vs find()
find() will operates the children of the current set
filter() will only operate on the current set of elements.

On the same level: (FILTER)
$('div').filter('.x').length
or
$('div.x').text()

Inner level search (FIND)
$('div').find('.x').length
or
$('div .x').text()

Jquery - Save File:
$('#ajax-loader').show();
            $.post("/test/return/data/", snddata,
                  function callbackHandler(data, textstatus){
                    $('#ajax-loader').hide();
                    if(data.status == 1){
                        str = data.encrypted_data
                        saveData(str, "download_file_name")
                    }
                    else{
                       alert(data.msg);
                    }
                 },
                 "json"
            );

var saveData = (function () {
        var a = document.createElement("a");
        document.body.appendChild(a);
        a.style = "display: none";
        return function (data, fileName) {
            var json = data,
                blob = new Blob([json], {type: "octet/stream"}),
                url = window.URL.createObjectURL(blob);
            a.href = url;
            a.download = fileName;
            a.click();
            window.URL.revokeObjectURL(url);
        };
      }());


&amp issues while passing from template to server
str = ''
document.location = '/test/url/' + encodeURIComponent(str);

!important
!important (It will override the CSS definition in header classes)
<style type="text/css">
   .bst_popover{
     max-width: 500px !important;
   }
</style>

Choose a specific parent
$(this).parents("tr:first");
$(this).closest("tr");


Jun 25, 2015

Python Spawn New Process SubProcess don't wait

Spawn New Process:
You can spawn a new process within an another process
You can make use of subprocess to start a new process

proc = Popen([python process_doc.py], shell=True, stdin=None, stdout=None, stderr=None, close_fds=True)

close_fds makes the parent process file handles inaccessible for the child

Once it hits Popen, it startes daemon process and coninues execution (parallely daemon process will coninue executing)

Example:
.....
print "Before Daemon Process to Start"
proc = Popen([python process_doc.py], shell=True, stdin=None, stdout=None, stderr=None, close_fds=True)
print "After Daemon Process to Start"

.....

In the above example, the parent process don't wait for child process (process_doc.py) to wait till it executes completely. It spawns new daemon process and continues.
This daemon process executes separately, finish the task and dies.

Run command in Shell:
cmd = 'python /tmp/test.py'
subprocess.call(cmd, shell=True) #Synchronous
subprocess.Popen(cmd, shell=True, executable='/bin/bash') #Asynchronous ****

Python Asynchronous


Real Time Scenario:

In general, we process data synchrnously by default
To improve performance, we can separate data processing separately (if no dependency) by spawing a new process and use multi-threading or multi-tasking

E.g.,
Suppose, you have a webpage, on submit you have to upload many images/documents to remote location (Synchronously one by one it takes more time). When hundreds of requests keep coming, its difficult for the server to handle 

How can we achieve better results in Asynchronous way:
Once you select images/documents to upload and submit
Spawn a new process to start the processing of uploading images (this processing can be multi-threaded, multi-processing)
Keep updating the progress of the status to Database (1 of 5 completed, 2 of 5 completed etc..)
Browser keep checking for the status of the Database
Once the status is SUCCESS, initimate the user with Success message

Spawn New Process:
You can spawn a new process within an another process
You can make use of subprocess to start a new process

proc = Popen([python process_doc.py], shell=True, stdin=None, stdout=None, stderr=None, close_fds=True)

close_fds makes the parent process file handles inaccessible for the child

Once it hits Popen, it startes daemon process and coninues execution (parallely daemon process will coninue executing)

JQuery/JavaScript to keep checking Server for status:
function runner() {
     setTimeout(function() {
        $.ajax({
             url : "AJAX_POST_URL",
             type: "POST",
            data : formData,
            success: function(data, textStatus) {
                    if (data.status != 'Success') {
                        runner()
                    } else if  (data.status == 'Success') {
                        alert("Successfully Uploaded") 
                    }      
            },
        });
    }, time);
 }

runner();
 

Jun 24, 2015

Python/Django Html to Text Conversion

Html to Text Conversion in Python (Using BeautifulSoup)

import urllib
url = "<url>"
html = urllib.urlopen(url).read()
soup = BeautifulSoup(html)

# kill all script and style elements
for script in soup(["script", "style"]):
    script.extract()    # rip it out

# get text
text = soup.get_text()

print text

Html to Text Conversion in Django (strip_tags)

from django.utils.html import strip_tags
import urllib

url = "<url>"

html = urllib.urlopen(url).read()
text = strip_tags(html)

print text


Jun 14, 2015

Django access remote application from your system

Assume you have two systems with Django applications installed

Suppose
System1 IP is : 192.10.10.111
System2 IP is : 192.10.10.222 

Suppose you are using System1, and you want to also access the System2 application (for Testing)
Instead of going to System2, you can very well configure host settings to access System2 machine from your own Desktop/Machine

Say the application name (ServerName) for System1 is app111.com
Say the application name (ServerName) for System2 is app222.com

Configure Hosts File in Windows System (192.10.10.111):

The Hosts file in Windows is located at the following location:
C:\Windows\System32\drivers\etc

Suppose your system IP is: 192.10.10.111

Add the following to the Hosts File:
192.10.10.111    app111.com
192.10.10.222   app222.com

When you access app222.com in your browser, it then points to 192.10.10.222 for accessing the application.

Similarly you can make use of hosts file to access the remote applications at your ease.

You might also be interested in reading:
Django Application with Remote Database


Django Application with remote database

Assume you want to access remote database on your Django application

Suppose
Your system IP is             : 192.10.10.111  (or) localhost
Your friends system IP is : 192.10.10.222 

Run your Django application with local database:
DATABASES = {
    'default': {
        'HOST': '192.10.10.111 ',    #(or) localhost
        'ENGINE': 'django.db.backends.mysql',
        'NAME': '111_test_db' ,
        'USER': '111_user',
        'PASSWORD': 'XXX'
    }
}

Run your Django application with remote database:
DATABASES = {
    'default': {
        'HOST': '192.10.10.222',
        'ENGINE': 'django.db.backends.mysql',
        'NAME': '222_test_db',
        'USER':'222_user',
        'PASSWORD':'YYY'
    }
}

When accessing remote database for yout django application, you need Grant Permissions on Remote Database for initiating HOST user

E.g., In the remote database '222_test_db'  (Login)
grant all privileges on *.* to 111_user@192.10.10.111 identified by 'XXX' with grant option;

The above command gives/grants enough permissions to the 111_user to access 222_test_db, else "Permission Denied" error is shown.


Reload Apache:
service httpd restart

How to run multiple sites on one Apache

Assuming we have two sites to configure in Apache
www.example1.com
www.example2.com

/etc/httpd/conf.d/example1_http.conf
<VirtualHost *>
        ServerAdmin webmaster@example1.com
        ServerName  www.example1.com
        ServerAlias example1.com

        # Indexes + Directory Root.
        DirectoryIndex index.html
        DocumentRoot /home/www/www.example1.com/htdocs/
.....
</VirtualHost>

/etc/httpd/conf.d/example2_http.conf
<VirtualHost *>
        ServerAdmin webmaster@example2.com
        ServerName  www.example2.com
        ServerAlias example2.com

        # Indexes + Directory Root.
        DirectoryIndex index.html
        DocumentRoot /home/www/www.example2.com/htdocs/
.....
</VirtualHost>

Configure Hosts File in Windows:

The Hosts file in Windows is located at the following location:
C:\Windows\System32\drivers\etc

Suppose your system IP is: 192.55.44.55

Add the following to the Hosts File:
192.55.44.55 example1.com
192.55.44.55 example2.com
192.55.44.55 www.example1.com
192.55.44.55 www.example2.com

Restart Apache

Now your Apache runs the two multiple sites

You can access example1.com and example2.com respectively

Access Default Site when Server Name/Alias mismatches

When you have multiple sites (multiple conf files) in conf.d, you can create a default conf file

If Apache finds difficulty in finding the site, it defaults to
The following defaults to www.example1.com in case of any conflicts

/etc/httpd/conf.d/aaa_http.conf
<VirtualHost *>
        ServerAdmin webmaster@example1.com
        ServerName  www.example1.com
        ServerAlias example1.com

        # Indexes + Directory Root.
        DirectoryIndex index.html
        DocumentRoot /home/www/www.example1.com/htdocs/
.....
</VirtualHost>

How to simulate the conflict state where Apache not able to find the correct site conf file

For two different sites, give the same ServerName
/etc/httpd/conf.d/test1_http.conf
          ServerName  www.test.com
/etc/httpd/conf.d/test2_http.conf
          ServerName  www.test.com

If you try to access www.test.com, Apache gets CONFUSED to pick which sites conf file (whether to choose test1_http.conf or test2_http.conf)

So Apache picks the top conf file (as per naming order) defined above aaa_http.conf
So you are re-directed to www.example1.com (as defined in aaa_http.conf  

 

Load Testing Apache with AB (Apache Bench)

ab - Apache HTTP server benchmarking tool

  • ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. 
  • It is designed to give you an impression of how your current Apache installation performs. 
  • This especially shows you how many requests per second your Apache installation is capable of serving.

-n => No.of requests
Number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.

-c => Concurrency
Number of multiple requests to perform at a time. Default is one request at a time.

-k => Keep Alive
Enable the HTTP KeepAlive feature, i.e., perform multiple requests within one HTTP session. Default is no KeepAlive.
KeepAlive header, which asks the web server to not shut down the connection after each request is done, but to instead keep reusing it.

-q => Supress messages
When processing more than 150 requests, ab outputs a progress count on stderr every 10% or 100 requests or so. The -q flag will suppress these messages.

 -r => Don't exit on socket receive errors.

E.g.,
 ab -n 1000 -c 100 http://test.xyz.com/

Sample Output:
Benchmarking test.xyz.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests

Server Software:        Apache/2.2.15
Server Hostname:        test.xyz.com
Server Port:            80

Document Path:          /
Document Length:        29089 bytes

Concurrency Level:      450
Time taken for tests:   9.537570 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      29537664 bytes
HTML transferred:       29295116 bytes
Requests per second:    104.85 [#/sec] (mean)
Time per request:       4291.906 [ms] (mean)
Time per request:       9.538 [ms] (mean, across all concurrent requests)
Transfer rate:          3024.36 [Kbytes/sec] received


AWS Route 53 Configuration

Using CLI53 Tool - Export from GoDaddy & Import Zone File to Route53



cli53 is a Python script for managing Amazon Route 53.
It's freaking awesome as it can import, export and help you debug your Route 53 setup.

Ref:

1) Godadd'y DNS Manager you can export your domain's setup:
export file <todoist.com.zone>

2) pip install cli53

3) Simply add $ORIGIN yourdomain.com. at the top of the Godaddy's export file
from Step 1
$ORIGIN yourdomain.com.
e.g., $ORIGIN todoist.com.

4) Import data with cli53
- Create a hosted Zone : cli53 create todoist.com
- Import your Godaddy export : cli53 import todoist.com --file <todoist.com.zone>

5)  Use the excellent "Interstate53" service to recheck that everything is setup correctly
     using a web interface.
      Or
      just use "cli53 rrlist command"
     e.g, cli53 rrlist todoist.com

6) Debug and re-check everything
For GoDaddy:
dig -t any @PDNS01.DOMAINCONTROL.COM todoist.com

For Route53:
dig -t any @ns-158.awsdns-19.com todoist.com

7) Update your domain's nameservers
- The last step is to update your domain's nameservers to point to Route 53

8) Transfer your domain from GoDaddy

1. How to unlock your domain with GoDaddy:
- Login to your GoDaddy account
- Next to Domains click on Manage
- Select the domain(s) to unlock and click Lock
- Select Off radio-button to have the domain(s) unlocked and click Save
2. How to obtain EPP/Authorization code from GoDaddy:
- Login to your GoDaddy account
- Next to Domains click on Launch
- Click on the domain you need EPP code
- Click Email my code in the Authorization Code field - send

3. You would also need to disable privacy protection service for the domain (if it's enabled).
- Go to the Domains By Proxy website and login to your account (the login details are not the same as for your GoDaddy account. Look for the email sent by support@domainsbyproxy.com for the login details)
- Select a check box next to the domain name(s) you need to disable privacy protection for
- Click Cancel Selected, OR click on the cancellation icon next to the domain name
- In "Confirm" window click OK

4. How to accept the transfers at GoDaddy.
- Log in to your Account Manager
- Next to Domains, click Manage,
- From the Domains menu, select Transfers
- Click on Pending Transfers Out, select the domain name(s) you are transferring from GoDaddy and click on Accept/ Decline above
- Select Accept and click OK. The request will be processed within 15 minutes.


Using Route53 Console UI - Export from GoDaddy & Import Zone File to Route53

  • Get a zone file from the DNS service provider that is currently servicing the domain. The process and terminology vary from one service provider to another. Refer to your provider's interface and documentation for information about exporting or saving your records in a zone file or a BIND file.

If the process isn't obvious, try asking your current DNS provider's customer support for your records list or zone file information.
  • Click Create Hosted Zone.
  • Enter the name of your domain and, optionally, a comment. Note that the comment can't be edited later.
  • Click Create.
  • On the Hosted Zones page, double-click the name of your new hosted zone.
  • Click Import Zone File.
  • In the Import Zone File pane, paste the contents of your zone file into the Zone File text box.
  • Click Import.

Jun 20, 2014

Django Example Project - Student Application

About Django Student Application
This student application achieves a reporting tool for subject and marks of the student
Based on the subject filtering, this application generates a brief report on student marks (Beautiful Pie Chart)

Download Application

Prerequisites
Python
Django
MySQL
HighCharts
Jqury

JavaScript Modules Used
Highcharts
Jquery

Steps to install Student Application
django-admin.py startproject StudentWebsite
python manage.py startapp StudentApp
python manage.py validate
python manage.py sqlall StudentApp

MySQL Database Details
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'studentsdb',                     
        'USER': 'XXXX',
        'PASSWORD': 'XXXX',
        'HOST': '127.0.0.1',                     
        'PORT': '3306',                
    }
}

Insert MySQL sample data
Note: You can either add the data from Admin page (one by one) or Load/Import sample "studentapp.sql" into database

Database Models Used:
Student
Subject
StudentMarks

How to start Django Development Server
Django develeopment server works on default port 8000
python manage.py runserver 8000

Access Home Page

http://127.0.0.1:8000/home/


How Application Looks
Select the desired subject

After selecting the subject as Physics - Generates the report as above































models.py

#-*- encoding=UTF-8 -*-
from django.db import models
from django.core.exceptions import ValidationError
import re

class Student(models.Model):
    firstname = models.CharField(max_length=30, null=False, blank=False)
    lastname = models.CharField(max_length=30, null=False, blank=False)

    def clean(self):
        if re.match(r'^\s+$', self.firstname):
            raise ValidationError('Firstname should contain some characters, only spaces not allowed')
        if re.match(r'^\s+$', self.lastname):
            raise ValidationError('Lastname should contain some characters, only spaces not allowed') 
 
    def __unicode__(self):
        return u'%s %s' % (self.firstname, self.lastname)

    class Meta:
        unique_together = (("firstname", "lastname"),)
        ordering = ['firstname']

class Subject(models.Model):
    name = models.CharField(max_length=10, unique=True, null=False, blank=False)
    
    def clean(self):
        if re.match(r'^\s+$', self.name):
            raise ValidationError('Subject should contain some characters, only spaces not allowed')
    
    def __unicode__(self):
        return u'%s' % (self.name)

class StudentMarks(models.Model):
    student = models.ForeignKey(Student)
    subject = models.ForeignKey(Subject)
    marks   = models.IntegerField(max_length=3, null=False, blank=False)

    class Meta:
        unique_together = (("student", "subject"),)  

urls.py

from django.conf.urls import patterns, include, url

# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'StudentWebsite.views.home', name='home'),
    # url(r'^StudentWebsite/', include('StudentWebsite.foo.urls')),

    # Uncomment the admin/doc line below to enable admin documentation:
    url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

    # Uncomment the next line to enable the admin:
    url(r'^admin/', include(admin.site.urls)),
)

urlpatterns += patterns('StudentWebsite.StudentApp.views',
 #url(r'^$', 'home'),
 url(r'^home/$', 'subjectView', name="home"),
    url(r'^getMarksDetailsView/(?P\w+)/$', 'getMarksDetailsView', name="getMarksDetailsView"),
)


views.py

#-*- encoding=UTF-8 -*-
from django.http import HttpResponse, HttpResponseRedirect, Http404
#from django.http import *
from django.shortcuts import render, render_to_response
from django.core.urlresolvers import reverse
from StudentWebsite.StudentApp.models import Subject, Student, StudentMarks
from django.db.models import Avg, Count

def subjectView(request):
    subjectList = Subject.objects.values('pk', 'name')
    side_bar = {'Home':'home', 'Admin' : 'admin:index'}
    return render(request, 'StudentsMain.html', {'subjects' : subjectList, 'side_bar' : side_bar})

def getMarksDetailsView(request, subjectName):
    subjectStudentDetails = StudentMarks.objects.filter(subject__name=subjectName).values('student__firstname', 'student__lastname', 'subject__name', 'marks')
    avgSubjectMarks = Subject.objects.filter(name=subjectName).annotate(average_rating=Avg('studentmarks__marks'))[0].average_rating
    side_bar = {'Home':'home', 'Admin' : 'admin:index'}
    ranges = [ '0-40', '41-60', '61-80', '81-90', '91-100']
    marksRange = {}
    for eachRange in ranges:
        count = StudentMarks.objects.filter(subject__name=subjectName).filter(marks__range=(eachRange.split('-'))).count()
        marksRange[eachRange] = count 
    return render(request, 'StudentSubjectDetails.html', {'subjectStudentDetails' : subjectStudentDetails, 'avgSubjectMarks' : avgSubjectMarks, 'side_bar' : side_bar, 'marksRange' : marksRange, 'subjectName' : subjectName.title() })


admin.py

from django.contrib import admin
from StudentWebsite.StudentApp.models import Student, Subject, StudentMarks

#This is just for dispaly order of columns for Author
class StudentAdmin(admin.ModelAdmin):
    list_display = ('firstname', 'lastname')
    search_fields = ('firstname', 'lastname')
    ordering = ('-firstname',)

class SubjectAdmin(admin.ModelAdmin):
    list_display = ('name',)
    list_filter = ('name',)
    ordering = ('-name',)
    fields = ('name',)

class StudentMarksAdmin(admin.ModelAdmin):
    list_display = ('student', 'subject', 'marks')
    list_filter = ('student', 'subject', 'marks')
    ordering = ('-student',)

admin.site.register(Student, StudentAdmin) #StudentAdmin is just for display in Admin page
admin.site.register(Subject, SubjectAdmin) #SubjectAdmin is just for display in Admin page
admin.site.register(StudentMarks, StudentMarksAdmin)