Loop through Directory and read the files in Perl

Now, we discuss about how to loop through a directory and read all the files in Perl
I have a directory with name "humanists" which has different files as mentioned below :

humanists :
abraham_lincon.txt
alfred_nobel.txt
mahatma_gandhi.txt
mother_teresa.txt
winston_churchill.txt

All the files has details in the following format
Name: <>
Born: <>
Died: <>

Now we have to loop thorugh all the files in the directory and get the information from these files and create a hash object.

Lets dive into this example. Please drop me a comment if you have any doubts on the same.


read_humanists.pl

#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper;

my $dir_path = "/home/prabhath/prabhath_test/test/humanists";

my %humanists_list;

opendir(IN_DIR, $dir_path) or die $!; 
open (LOG_FILE, "> read_humanists.log") or die $!; 

unless ( -d $dir_path) {
   print "\n Path not present: " . $dir_path;
} else {
   print "\n Path is present: " . $dir_path;
}


while (my $file = readdir(IN_DIR)) {
    next if $file =~ /^\./;
    
    unless (-f "$dir_path/$file") {
        print "\n -W- Not a File, Ignoring : " . "$dir_path/$file" . "\n";
        next;
    }   

    open(IN_FILE, "< $dir_path/$file")
                    ||  die "\n Cant open file for reading: " . "$dir_path/$file";

    my @lines = ;
    close(IN_FILE);

    my @newlines;
    foreach my $each_line (@lines) {
        if ($each_line =~ /^Name:/) {
           my ($name) = $each_line =~ /Name:\s(.*)$/;
           $humanists_list{$file}{"name"} = $name;
        }

        if ($each_line =~ /^Born:/) {
           my ($dob) = $each_line =~ /Born:\s(.*)$/;
           $humanists_list{$file}{"date_of_birth"} = $dob;
        }

        if ($each_line =~ /^Died:/) {
           my ($dod) = $each_line =~ /Died:\s(.*)$/;
           $humanists_list{$file}{"died"} = $dod;
        }
    }   
} 

Output:

          'abraham_lincon.txt' => {
                                    'died' => 'April 15, 1865',
                                    'date_of_birth' => 'February 12, 1809',
                                    'name' => 'Abraham Lincon'
                                  },
          'winston_churchill.txt' => {
                                       'died' => 'January 24, 1965',
                                       'date_of_birth' => 'November 30, 1874',
                                       'name' => 'Winston Churchill'
                                     },
          'alfred_nobel.txt' => {
                                  'died' => 'December 10, 1896',
                                  'date_of_birth' => 'October 21, 1833',
                                  'name' => 'Alfred Nobel'
                                },
          'mother_teresa.txt' => {
                                   'died' => 'September 5, 1997',
                                   'date_of_birth' => 'August 26, 1910',
                                   'name' => 'Mother Teresa'
                                 },
          'mahatma_gandhi.txt' => {
                                    'died' => 'January 30, 1948',
                                    'date_of_birth' => 'October 2, 1869',
                                    'name' => 'Mahatma Gandhi'
                                  }
  

Please refer to other topics on Unix like :
Unix Delete Duplicated Lines in a File
Unix Unique Lines in a File
Unix Grep Examples
Unix Cut Command Examples
Search a Directory in Unix
Unix For Loop
pushd & popd in Unix
Find Size of Directory
Word Count

Please refer to other topics on AWK like :
Awk Examples
Print First Two Columns of File
Print Last Two Columns of File

Please refer to other topics on Dict like :
Dict in Python
Dict keys and values in Python

Please refer to other topics on List like :
List in Python
Append to list in Python
Delete the last name from the list in Python
Remove an element from List in Python
Check an element exists in an list in Python
Python Filter Vs Map Vs List Comprehension

Please refer to other topics on File Concepts like :
Print File Content in Python
Print File in Reverse Order in Python

Please refer to Regular Expressions Concepts :
Brief on Regular Expressions
Greedy Operators in Regular Expressions in Perl
Modifiers in Regular Expressions in Perl
Capturing concept in Regular Expressions in Perl
Capture Pre Match ,Post Match, Exact match in Regular Expressions in Perl
Non Capturing Paranthesis in Regular Expressions in Perl
Substitute nth occurance in Regular Expressions in Perl
All Topics in Regular Expressions in Perl

You might also wish to read other topics like :
Python Class and Object Example
Inheritance in Python
Packages in Python
Exceptions in Python
How to remove duplicate lines from a file in Perl
How to remove duplicate lines from a file in Pyhton

How to create excel report in Perl

We can generate Microsoft Excel (XLS) report using Perl

Note:
Please install module Spreadsheet::WriteExcel before running the program

We can create worksheets
We can set styles like color/font/size etc for the cells

#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper;
use Spreadsheet::WriteExcel;


my ($xls_label, $workbook, $worksheet, $format_header, $format_row);

$xls_label = "test.xls";

$workbook = Spreadsheet::WriteExcel->new($xls_label);

$worksheet = $workbook->add_worksheet();

$format_header = $workbook->add_format(); # Add a format
$format_header->set_bold();
$format_header->set_color('purple');
$format_header->set_align('center');
$format_header->set_size(12);

$format_row = $workbook->add_format(); # Add a format
$format_header->set_bold();
$format_row->set_align('center');

my $row = 0;
my $column = 0;

$worksheet->write($row, 0, 'S.No', $format_header);
$worksheet->write($row, 1, 'Humanists', $format_header);

my @humanists_arr = ("Mother Teresa", "Mahatma Gandhi", "Abraham Lincoln", "Winston Churchil", "Alfred Nobel");

foreach my $each_humanist (@humanists_arr) {
   $row++;
   $worksheet->write($row, 0, $row, $format_row);
   $worksheet->write($row, 1, $each_humanist, $format_row);
}

1;

Output :

S.No Humanists
1 Mother Teresa
2 Mahatma Gandhi
3 Abraham Lincoln
4 Winston Churchil
5 Alfred Nobel  

Please refer to other topics on Unix like :
Unix Delete Duplicated Lines in a File
Unix Unique Lines in a File
Unix Grep Examples
Unix Cut Command Examples
Search a Directory in Unix
Unix For Loop
pushd & popd in Unix
Find Size of Directory
Word Count

Please refer to other topics on AWK like :
Awk Examples
Print First Two Columns of File
Print Last Two Columns of File

Please refer to other topics on Dict like :
Dict in Python
Dict keys and values in Python

Please refer to other topics on List like :
List in Python
Append to list in Python
Delete the last name from the list in Python
Remove an element from List in Python
Check an element exists in an list in Python
Python Filter Vs Map Vs List Comprehension

Please refer to other topics on File Concepts like :
Print File Content in Python
Print File in Reverse Order in Python

Please refer to Regular Expressions Concepts :
Brief on Regular Expressions
Greedy Operators in Regular Expressions in Perl
Modifiers in Regular Expressions in Perl
Capturing concept in Regular Expressions in Perl
Capture Pre Match ,Post Match, Exact match in Regular Expressions in Perl
Non Capturing Paranthesis in Regular Expressions in Perl
Substitute nth occurance in Regular Expressions in Perl
All Topics in Regular Expressions in Perl

You might also wish to read other topics like :
Python Class and Object Example
Inheritance in Python
Packages in Python
Exceptions in Python
How to remove duplicate lines from a file in Perl
How to remove duplicate lines from a file in Pyhton

Unix Cut Comamnd

CUT command has lot of advantages in grep/cut the fields/chars from the files easily in no time

We can use CUT command in the following ways:
1) Print Characters by Position (chars)
2) Print Characters by Range (range a-b)
3) Print the fields using the delimiter

Now, we discuss the above mentioned briefly. Let's have fun with some simple examples.
In all the below examples, we use same.txt as input as mentioned below :

sample.txt
Andy,100,Ney york,USA
Arjun,200,New Delhi,India
Venkatesh,300,Chennai,India
Andy,400,Boston,USA
John,500,Chicago,USA


1) Using Cut Print Characters by Position

This command prints the fourth character in each line of the file

]# cut -c4 sample.txt
y
u
k
y
n

This command prints the fourth and sixth character in each line.

]# cut -c4,6 sample.txt
y1
u,
kt
y4
n5

2) Using Cut Print Characters by Range

This command prints the characters from fourth position to the eighth position in each line

]# cut -c4-8 sample.txt
y,100
un,20
kates
y,400
n,500

This command prints the first four characters in a line

]# cut -c-4 sample.txt
Andy
Arju
Venk
Andy
John

This command prints the characters from fourth position to the end

]# cut -c4- sample.txt
y,100,Ney york,USA
un,200,New Delhi,India
katesh,300,Chennai,India
y,400,Boston,USA
n,500,Chicago,USA

This command omits the start and end positions, then the cut command prints the entire line.

]# cut -c- sample.txt
Andy,100,Ney york,USA
Arjun,200,New Delhi,India
Venkatesh,300,Chennai,India
Andy,400,Boston,USA
John,500,Chicago,USA

3) Print the fields using the delimiter
-d  option in cut command can be used to specify the delimiter 
-f  option is used to specify the field position

This command prints the first field of comma separated line

]# cut -d',' -f1 sample.txt
Andy
Arjun
Venkatesh
Andy
John

This command prints the first and second field in each comma separated line

]# cut -d',' -f1,2 sample.txt
Andy,100
Arjun,200
Venkatesh,300
Andy,400
John,500

This command prints the first, second and third field (range 1-3) in each comma separated line

]# cut -d',' -f1-3 sample.txt
Andy,100,Ney york
Arjun,200,New Delhi
Venkatesh,300,Chennai
Andy,400,Boston
John,500,Chicago

This command prints the first two fields in each comma separated line

]# cut -d',' -f-2 sample.txt
Andy,100
Arjun,200
Venkatesh,300
Andy,400
John,500

This command prints from first field to last field in each comma separated line

]# cut -d',' -f1- sample.txt
Andy,100,Ney york,USA
Arjun,200,New Delhi,India
Venkatesh,300,Chennai,India
Andy,400,Boston,USA
John,500,Chicago,USA 

Please refer to other topics on Unix like :
Unix Delete Duplicated Lines in a File
Unix Unique Lines in a File
Unix Grep Examples
Search a Directory in Unix
Unix For Loop
pushd & popd in Unix
Find Size of Directory
Word Count

Please refer to other topics on AWK like :
Awk Examples
Print First Two Columns of File
Print Last Two Columns of File

Please refer to other topics on Dict like :
Dict in Python
Dict keys and values in Python

Please refer to other topics on List like :
List in Python
Append to list in Python
Delete the last name from the list in Python
Remove an element from List in Python
Check an element exists in an list in Python
Python Filter Vs Map Vs List Comprehension

Please refer to other topics on File Concepts like :
Print File Content in Python
Print File in Reverse Order in Python

Please refer to Regular Expressions Concepts :
Brief on Regular Expressions
Greedy Operators in Regular Expressions in Perl
Modifiers in Regular Expressions in Perl
Capturing concept in Regular Expressions in Perl
Capture Pre Match ,Post Match, Exact match in Regular Expressions in Perl
Non Capturing Paranthesis in Regular Expressions in Perl
Substitute nth occurance in Regular Expressions in Perl
All Topics in Regular Expressions in Perl

You might also wish to read other topics like :
Python Class and Object Example
Inheritance in Python
Packages in Python
Exceptions in Python
How to remove duplicate lines from a file in Perl
How to remove duplicate lines from a file in Pyhton