Prabhath Kota: Django

Showing posts with label Django. Show all posts

Feb 12, 2019

Python csv write

from users.models import *
import sys

up = UserProfile.objects.all()
print up.count()

header = "Name, User Type, Email, Phone, City, Country, TimeZone, Created Date\n"

fname = "/tmp/user_data.csv"
with open(fname, 'w') as FW:
FW.write(header)
for each in up:
try:
name = each.name
name = name.encode('utf-8').strip()
dts = each.created_date
dts = dts.strftime("%d %b %Y") #dts.strftime("%b %d %Y %I:%M %p")
str1 = "%s,%s,%s,%s,%s,%s,%s,%s\n" % (str(name), str(each.usertype), str(each.email_id), str(each.phone), str(each.city), str(each.country), str(each.timezone), str(dts))
#print str1
FW.write(str1)
except Exception, e:
print '---- Error: ' + str(e)

print fname

Output:
Name, User Type, Email, Phone, City, Country, TimeZone, Created Date
User2,Student,user2@abc.com,None,Bangalore,IN,Asia/Kolkata,16 Mar 2018
User3,Student,user3@gmail.com,None,Pune,IN,Asia/Kolkata,28 Mar 2018
....

Feb 5, 2019

Django update UTC time to MySQL

from datetime import datetime
from django.utils.timezone import utc

sch = Schedule.objects.get(id=25)
sch.start_time = datetime.utcnow().replace(tzinfo=utc)
sch.save()

Sep 1, 2018

S3 Direct Upload Python Django

Using Pyhton Boto API, we can interact with Amazon S3 servers (for GET, PUT, POST etc)
We can also directly upload files to Amazon S3 from the client browser using Browser Uploads to S3 using HTML POST Forms.

Ref:
https://aws.amazon.com/articles/Java/1434

Prerequisites:
S3_BUCKET
S3_KEY
S3_SECRET
S3_URL

Note:
As per Amazon API, we need to encode policy format to base64 and further generate signature with SHA1
Both Policy and Signature need to be posted
Expiration : You can define the expiration time
acl: public-read/private
$key: upload path startes with
Check the POST S3 Url in HTML
Once the document is successfully uploaded to S3, S3 URL is shown on the page

Django Pyhton View:

import base64
import hmac, hashlib
import re

def direct_s3_upload(request):
  my_bucket = None
  policy_document = None
  policy_base_64 = signature = cors_xml = ""
  try:
    policy_document = '{"expiration": "2020-01-01T00:00:00Z", \
                       "conditions": [ \
                         {"bucket": "%s"}, \
                         {"acl": "public-read"}, \
                         ["starts-with", "$key", "uploads/"], \
                         ["content-length-range", 0, 524288000] \
                       ] \
                    }' % (<S3_BUCKET>)

    whitespace = re.compile(r'\s+')
    policy_document = whitespace.sub('', policy_document)
    policy_base_64 = base64.b64encode(whitespace.sub('', policy_document))
    signature = base64.b64encode(hmac.new(S3_SECRET, policy_base_64, hashlib.sha1).digest()) 
    dic1 = { 
           'MY_BUCKET_NAME': <S3_BUCKET>,
           'MY_AWS_KEY_ID': <S3_KEY>, 
           'MY_POLICY' : policy_base_64,
           'MY_SIGNATURE' : signature,
        }
  except:
    write_exception("direct_s3_upload")
  return render_to_response('test_s3_upload.html', context_instance=RequestContext(request, dic1))

HTML: (test_s3_upload.html)

<html> 
  <head>
    <title>S3 POST Form</title> 
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <script>
      var bucketName = '{{MY_BUCKET_NAME}}';
      var AWSKeyId   = '{{MY_AWS_KEY_ID}}';
      var policy     = '{{MY_POLICY}}';
      var signature  = '{{MY_SIGNATURE}}';

      function S3ToolsClass() {
        var _handle_progress = null;
        var _handle_success  = null;
        var _handle_error    = null;
        var _file_name       = null;

        this.uploadFile = function(file, progress, success, error) {
          _handle_progress = progress;
          _handle_success  = success;
          _handle_error    = error;
          _file_name       = file.name;

          console.log(file.name)
          var fd = new FormData();
          fd.append('key', "uploads/" + file.name);
          fd.append('AWSAccessKeyId', AWSKeyId);
          fd.append('acl', 'public-read');
          fd.append('policy', policy);
          fd.append('signature', signature);
          fd.append("file",file);

          var xhr = new XMLHttpRequest({mozSystem: true});
          xhr.upload.addEventListener("progress", uploadProgress, false);
          xhr.addEventListener("load", uploadComplete, false);
          xhr.addEventListener("error", uploadFailed, false);
          xhr.addEventListener("abort", uploadCanceled, false);
          xhr.open('POST', 'https://s3.amazonaws.com/' + bucketName + '/');

          xhr.send(fd);
        }

        function uploadProgress(evt) {
          if (evt.lengthComputable) {
            var percentComplete = Math.round(evt.loaded * 100 / evt.total);
            _handle_progress(percentComplete);
          }
        }

        function uploadComplete(evt) {
          if (evt.target.responseText == "") {
            console.log("Upload complete - success") 
            _handle_success(_file_name);
          } else {
            console.log("Upload complete - not success") 
            _handle_error(evt.target.responseText);
          }
        }

        function uploadFailed(evt) {
          console.log("upload Failed")
          _handle_error("There was an error attempting to upload the file." + evt);
        }

        function uploadCanceled(evt) {
          console.log("upload cancelled")
          _handle_error("The upload has been canceled by the user or the browser dropped the connection.");
        }
      }
      var S3Tools = new S3ToolsClass();
      
      function uploadFile() {
        var file = document.getElementById('file').files[0];
        S3Tools.uploadFile(file, handleProgress, handleSuccess, handleError);
      }

      function handleProgress(percentComplete) {
        document.getElementById('progressNumber').innerHTML = percentComplete.toString() + '%';
      }

      function handleSuccess(fileName) {
        document.getElementById('progressNumber').innerHTML = 'Done!';
        document.getElementById('resultant_s3_url').innerHTML = 'https://s3.amazonaws.com/' + bucketName + '/uploads/' + fileName;
      }

      function handleError(message) {
        document.getElementById('progressNumber').innerHTML = 'Error: ' + message;
      }
    </script>
  </head>
  <body>
    <form id="form" enctype="multipart/form-data" method="post">
      <div class="row">
        1. Select a File<br>
        <input type="file" name="file" id="file"/>
      </div>
      <br>
      <div class="row">
        2. Upload File<br>
        <input type="button" onclick="uploadFile()" value="Upload" />
        <br/>
        <span id="progressNumber"></span>
      </div>
    </div>
    <br>
    <div class="row">
      3. Result S3 Url
      <br>
      <div id="resultant_s3_url"> </div>

    </div>
  </body>
</html>

Thank you for reading this article.

Jun 24, 2015

Python/Django Html to Text Conversion

Html to Text Conversion in Python (Using BeautifulSoup)

import urllib
url = "<url>"
html = urllib.urlopen(url).read()
soup = BeautifulSoup(html)

# kill all script and style elements
for script in soup(["script", "style"]):
script.extract() # rip it out

# get text
text = soup.get_text()
print text

Html to Text Conversion in Django (strip_tags)

from django.utils.html import strip_tags
import urllib

url = "<url>"
html = urllib.urlopen(url).read()
text = strip_tags(html)
print text

Jun 14, 2015

Django access remote application from your system

Assume you have two systems with Django applications installed

Suppose
System1 IP is : 192.10.10.111
System2 IP is : 192.10.10.222

Suppose you are using System1, and you want to also access the System2 application (for Testing)
Instead of going to System2, you can very well configure host settings to access System2 machine from your own Desktop/Machine

Say the application name (ServerName) for System1 is app111.com
Say the application name (ServerName) for System2 is app222.com

Configure Hosts File in Windows System (192.10.10.111):

The Hosts file in Windows is located at the following location:
C:\Windows\System32\drivers\etc

Suppose your system IP is: 192.10.10.111

Add the following to the Hosts File:
192.10.10.111 app111.com
192.10.10.222 app222.com

When you access app222.com in your browser, it then points to 192.10.10.222 for accessing the application.

Similarly you can make use of hosts file to access the remote applications at your ease.

You might also be interested in reading:
Django Application with Remote Database

Django Application with remote database

Assume you want to access remote database on your Django application

Suppose
Your system IP is             : 192.10.10.111 (or) localhost
Your friends system IP is : 192.10.10.222

Run your Django application with local database:
DATABASES = {
    'default': {
        'HOST': '192.10.10.111 ',    #(or) localhost
        'ENGINE': 'django.db.backends.mysql',
        'NAME': '111_test_db' ,
        'USER': '111_user',
        'PASSWORD': 'XXX'
    }
}

Run your Django application with remote database:
DATABASES = {
    'default': {
        'HOST': '192.10.10.222',
        'ENGINE': 'django.db.backends.mysql',
        'NAME': '222_test_db',
        'USER':'222_user',
        'PASSWORD':'YYY'
    }
}

When accessing remote database for yout django application, you need Grant Permissions on Remote Database for initiating HOST user

E.g., In the remote database '222_test_db' (Login)
grant all privileges on *.* to 111_user@192.10.10.111 identified by 'XXX' with grant option;

The above command gives/grants enough permissions to the 111_user to access 222_test_db, else "Permission Denied" error is shown.

Reload Apache:
service httpd restart

Jun 20, 2014

Django Example Project - Student Application

About Django Student Application

This student application achieves a reporting tool for subject and marks of the student

Based on the subject filtering, this application generates a brief report on student marks (Beautiful Pie Chart)

Download Application

https://github.com/prabhathkota/Django-Student-Application

Prerequisites

Python

Django

MySQL

HighCharts

Jqury

JavaScript Modules Used

Highcharts

http://code.highcharts.com/modules/exporting.js

http://code.highcharts.com/highcharts.js

Jquery

http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js

Steps to install Student Application

django-admin.py startproject StudentWebsite

python manage.py startapp StudentApp

python manage.py validate

python manage.py sqlall StudentApp

MySQL Database Details

DATABASES = {

'default': {

'ENGINE': 'django.db.backends.mysql',

'NAME': 'studentsdb',

'USER': 'XXXX',

'PASSWORD': 'XXXX',

'HOST': '127.0.0.1',

'PORT': '3306',

}

Insert MySQL sample data

Note: You can either add the data from Admin page (one by one) or Load/Import sample "studentapp.sql" into database

https://github.com/prabhathkota/Django-Student-Application/blob/master/studentapp.sql

Database Models Used:

Student

Subject

StudentMarks

How to start Django Development Server

Django develeopment server works on default port 8000

python manage.py runserver 8000

Access Home Page

http://127.0.0.1:8000/home/

How Application Looks

Select the desired subject

After selecting the subject as Physics - Generates the report as above

models.py

#-*- encoding=UTF-8 -*-
from django.db import models
from django.core.exceptions import ValidationError
import re

class Student(models.Model):
    firstname = models.CharField(max_length=30, null=False, blank=False)
    lastname = models.CharField(max_length=30, null=False, blank=False)

    def clean(self):
        if re.match(r'^\s+$', self.firstname):
            raise ValidationError('Firstname should contain some characters, only spaces not allowed')
        if re.match(r'^\s+$', self.lastname):
            raise ValidationError('Lastname should contain some characters, only spaces not allowed') 
 
    def __unicode__(self):
        return u'%s %s' % (self.firstname, self.lastname)

    class Meta:
        unique_together = (("firstname", "lastname"),)
        ordering = ['firstname']

class Subject(models.Model):
    name = models.CharField(max_length=10, unique=True, null=False, blank=False)
    
    def clean(self):
        if re.match(r'^\s+$', self.name):
            raise ValidationError('Subject should contain some characters, only spaces not allowed')
    
    def __unicode__(self):
        return u'%s' % (self.name)

class StudentMarks(models.Model):
    student = models.ForeignKey(Student)
    subject = models.ForeignKey(Subject)
    marks   = models.IntegerField(max_length=3, null=False, blank=False)

    class Meta:
        unique_together = (("student", "subject"),)

urls.py

from django.conf.urls import patterns, include, url

# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'StudentWebsite.views.home', name='home'),
    # url(r'^StudentWebsite/', include('StudentWebsite.foo.urls')),

    # Uncomment the admin/doc line below to enable admin documentation:
    url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

    # Uncomment the next line to enable the admin:
    url(r'^admin/', include(admin.site.urls)),
)

urlpatterns += patterns('StudentWebsite.StudentApp.views',
 #url(r'^$', 'home'),
 url(r'^home/$', 'subjectView', name="home"),
    url(r'^getMarksDetailsView/(?P\w+)/$', 'getMarksDetailsView', name="getMarksDetailsView"),
)

views.py

#-*- encoding=UTF-8 -*-
from django.http import HttpResponse, HttpResponseRedirect, Http404
#from django.http import *
from django.shortcuts import render, render_to_response
from django.core.urlresolvers import reverse
from StudentWebsite.StudentApp.models import Subject, Student, StudentMarks
from django.db.models import Avg, Count

def subjectView(request):
    subjectList = Subject.objects.values('pk', 'name')
    side_bar = {'Home':'home', 'Admin' : 'admin:index'}
    return render(request, 'StudentsMain.html', {'subjects' : subjectList, 'side_bar' : side_bar})

def getMarksDetailsView(request, subjectName):
    subjectStudentDetails = StudentMarks.objects.filter(subject__name=subjectName).values('student__firstname', 'student__lastname', 'subject__name', 'marks')
    avgSubjectMarks = Subject.objects.filter(name=subjectName).annotate(average_rating=Avg('studentmarks__marks'))[0].average_rating
    side_bar = {'Home':'home', 'Admin' : 'admin:index'}
    ranges = [ '0-40', '41-60', '61-80', '81-90', '91-100']
    marksRange = {}
    for eachRange in ranges:
        count = StudentMarks.objects.filter(subject__name=subjectName).filter(marks__range=(eachRange.split('-'))).count()
        marksRange[eachRange] = count 
    return render(request, 'StudentSubjectDetails.html', {'subjectStudentDetails' : subjectStudentDetails, 'avgSubjectMarks' : avgSubjectMarks, 'side_bar' : side_bar, 'marksRange' : marksRange, 'subjectName' : subjectName.title() })

admin.py

from django.contrib import admin
from StudentWebsite.StudentApp.models import Student, Subject, StudentMarks

#This is just for dispaly order of columns for Author
class StudentAdmin(admin.ModelAdmin):
    list_display = ('firstname', 'lastname')
    search_fields = ('firstname', 'lastname')
    ordering = ('-firstname',)

class SubjectAdmin(admin.ModelAdmin):
    list_display = ('name',)
    list_filter = ('name',)
    ordering = ('-name',)
    fields = ('name',)

class StudentMarksAdmin(admin.ModelAdmin):
    list_display = ('student', 'subject', 'marks')
    list_filter = ('student', 'subject', 'marks')
    ordering = ('-student',)

admin.site.register(Student, StudentAdmin) #StudentAdmin is just for display in Admin page
admin.site.register(Subject, SubjectAdmin) #SubjectAdmin is just for display in Admin page
admin.site.register(StudentMarks, StudentMarksAdmin)

Feb 26, 2014

Django Tutorial - Part I

Reference:
Django Project / Django Book

Django web framework / Python web development with Django

Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
The tage line for Django is "The web framework for perfectionists with deadlines"
Django makes it easier to build Web apps more quickly and with less code, mainly used for web development

Main components of The Django framework

Object-relational mapper
Define your data models entirely in Python. You get a rich, dynamic database-access API for free, but you can still write SQL if required.
If your web application is data extensive, better write your SQL instead of object-relational mapper

Automatic admin interface
It comes with Admin interface, you need to enable it in the web framework
There by saving extra effort in creating the admin interface from the scratch

Elegant URL design
Very Flexible URL design using loose coupling
URL definitions and the view functions they call are loosely coupled; that is, the decision of what the URL should be for a given function, and the implementation of the function itself, reside in two separate places. This lets you switch out one piece without affecting the other.
Map the URLs and view methods, if you want to change the URL pattern later, you can do it very easily by changing urls.py
It makes use of regular expression for mapping the URL patterns
It generates dynamic URLs as well

Template system
Use Django's powerful, extensible and designer-friendly template language to separate design, content and Python code.

Cache system
Hook into memcached or other cache frameworks for super performance — caching is as granular as you need.

Internationalization
Django has full support for multi-language applications, letting you specify translation strings and providing hooks for language-specific functionality.

How to Install Django

Download Django (https://www.djangoproject.com/download/) and unzip

django-admin.py should be on your system path if you installed Django via its setup.py utility.
on Windows, you need to update your PATH environment variable.

python setup.py install
python -c 'import sys, pprint; pprint.pprint(sys.path)' #this locate site-packages direcotry

How to check Django Installed

>>> import django
>>> django.VERSION
(1, 4, 2, 'final', 0)

How to create a project

django-admin.py startproject <proj_name>
E.g., django-admin.py startproject mysite

Django Project Structure

The startproject command creates a directory containing five files:
mysite/
manage.py
mysite/
__init__.py
settings.py
urls.py
wsgi.py

How to access Django from shell

python manage.py shell

Django Development Server

Django comes with its own development server

Difference between Django Development Server Vs Web Development Server like Apache is as mentioned below :

Advantage:
Comes as part of Django itself
You can develop your site rapidly without having to deal with configuring your production server (e.g., Apache) until you are ready for production
Django Development server watches your code and automatically reloads it. making it easy for you to change code without restarting server

Disadvantage:
Not suitable for production environment
You have to deploy your code to Apache when moving to production

Django Development Server suitable in development environment
Apache web development server suitable in production environment

How to start Django Development Server

Django develeopment server works on default port 8000
python manage.py runserver 8000

How to test home page on Web Page

Using Django Development Server
http://127.0.0.1:8000/

Using Apache Server (If you configure Apache)
http://127.0.0.1/mysite/

How to configure Apache with Django

In Apache httpd.conf
WSGScriptAlias /mysite "C:/django_workspace/mysite/mysite/wsgi.py"

Thanks for reading this article. I will discuss more examples in the coming sessions, stay tuned :-)