Latex Project Report Template

Hi,

I have made an Unofficial BE Project Report Template in latex for Pune University - PICT College.

Note :
1) For BTech CSE NIT Calicut Template, & Bharati Vidyapeeth’s College of Engineering for Women, Pune University Latex Template, check the references below.

2) This template is for preliminary report. If you are looking for Final BE Project Report check this post.

To use it in an easy way, you can go to this site. It's an online latex web application. Although I did it on my Fedora KDE 15 Desktop, but many people don't have Linux or the software, so this will work no matter what OS you have as long as you have the net connection.

1. Go to this site.
2. Sign up for an Account by clicking on Sign up on top right.
3. Choose Free account.
4. Enter all details and ready your account.
5. Once you are done, Select New Project
6. Choose the name project
7. Go to my Github repo here and download the repo as a zip file.
8. Once you have downloaded the file.Go back to the Scribtex and click upload file.
9. Select the file and select check box - This is a zip file and should be extracted
10. Upload the file.
11. Once uploaded, you will see a new folder being created with something like firesofmay-Latex-Template-XXXXX - rename that folder by hovering over it to project.
12. Above is a very important step Since I have added the path as project/file.tex in all the files, hence the change is required.
13. Go to settings of your project and in compiler settings enter project/report.  
14. report.tex is the main file (like main function in a c program). So we are telling the web app this is the file to start with.
15. Click save. Select Files, select project folder, you should see all the files being uploaded. It has one README file, one jpg (pict-logo.jpg) and a few tex files.

Now Modify the files as you need to for your report.
Few Notes :-

% is an equivalent to // in C i.e. % is a comment specifier till the end of line.

I have added comments to make it easy to understand what is going on.

\\ is an equivalent to \n in C. Note at some places you can't add \n just like that.

Start with report.tex file
report.tex includes these files in this order

1. title page - First Page- title.tex
2. cover page - Second Page - cover.tex
3. Index Page - Automatically Generated
4. List of Figures - Automatically Generated
5. Introduction page - Intro to your project - introduction.tex
6. Literature Survey page - All your IEEE survey - literature-survey.tex
7. Project Statement page - Problem Statement, Explanantion - project-statement.tex
8. System Requirements page - Use Case, DFD, Activity Diagram - sys-req.tex
9. Scheduling and Planning page - Your plans - planning.tex
10. References page - Your References - ref.tex

You can modify and add anything anywhere you like in above files.

report.tex does not need any modification in start. But read it to get an idea of what is going on. Once done reading report.tex, start with files in the above order and modify them and save them and continue with the next file.

Once you are done, Either go to main project folder and select "View as PDF" or goto any file and click on compile. It will generate the PDF file for you.

Iterate over your work, and your latex report should be ready in no time! :)

Note : It generates a clickable Table of contents and list of figures, references which is highlighted sometimes. but when you do print preview, it is not printed with a highlighted view, so don't worry if you see that, else ignore this tip.

Please Read the Documentation of Latex here or you can download the book from here. It's a free book by wiki community.

Major part of the latex has been done in this template, but every project has its own requirement, so modify as it suits you.

I don't know many things in latex myself. So instead of asking me, Google it, Read The documentation, or ask at the #latex IRC channel, that's how I have been able to figure out my problems and create this template for you guys :).

Also, note that to add more pictures, upload the file and update the picture place holders.

Also if you see something like this:-

\fontsize{12.6468mm}{1mm} \textbf{Project Group No - <<Group Number Here>>}\\[2cm]

Here, <<>> is a place holder for you to remove and add your own content.

So if your group number is lets say 21, it will become

\fontsize{12.6468mm}{1mm} \textbf{Project Group No - 21}\\[2cm]

Note Remove the << and >> also!!!

You'll see <<Some text>> at many places in tex files.
I have added them for you to know here's a place to start modifying!!

\\[2cm] means give newline and the next line has a gap of 2cm.

Use the documentation to search for exactly what you need.

NOTE FOR THOSE WHO ARE USING THIS TEMPLATE OFFLINE:-
Also, if you are going to use this inside a desktop environment let's say Linux and use the latex tool on your OS instead of this site, you have to update all  the paths specified from project/file to file

If something needs to be fixed, please let me know by leaving a message on my github account

Hope it helps.
Enjoy :)

References :-
I have used this guy's template and modified it to suit Pune Universities Latex Template.Thanks Kartik :)

Update 2: Kartik added a link back to this post on his blog post :)

Update 3: Avani Shah of Bharati Vidyapeeth’s College of Engineering for Women, Pune University also made a Latex template which is tailor made for her college. Here's the link.

Note - I am not responsible if your guide does not accept your report for any reason. Please consult your Guide before you use this template ;)

HTTP, Web Scraping and Python - Part 2

In Part 1, I talked about User Agents. Today we'll try to see what I said, is it actually true? i.e. Do servers really see that user agent value? Do they really identify you with it?


Last time we proposed this as a hypothesis, today we'll see if it's a fact or not? ;)

We'll do a little experiment. For this You'll need firefox. So swtich to firefox, if you haven't.

Now, Go to this url. It's an addon which allows you to switch User Agents. Just download and restart firefox.

Oye. Stop. Go and download that Plugin before you move on! Such a lazy person you are! :) Just kidding ;) (but you'll really learn a lot more if you do this)

Now Go to 

Tools > Default User Agent > Edit User Agents.

Select New > New User Agents.

Now fill in random crap in each text field. Yes you heard me. Fill all crap!! Utter nonsense, or write poetry.  At least Change the User Agent field.


Okay. Done? Select Okay. Okay.
Now go again to 
Tools > Default User Agent > And select the user agent you made.


Okay Done?


Now go back to this same page. Notice anything above? It says :- "To try the thousands of add-ons available here, download Mozilla Firefox, a fast, free way to surf the Web!"

Huh?? Download Firefox? But I am in firefox!!! :)
Okay cool.


Now go again to 

Tools > {Your User Agent Name} > And select the default user agent.

Now reload the Page again. Bam! That Banner is gone!!


It shows that the value of the User Agent String does matter, and that servers do read that value to identify who you are! Why is this important to know you'll see in the later part of the series...

Hence, Proved! :)

PS - This Idea came from a problem I had. From past few months my firefox was not being recognized by my gmail and it always use to go to default basic mode rather than standard mode. I tried googling for it, but I didn't understand what was going on, or what to search for! It was a weird problem! I knew about User Agents but it was all theory!! :) No practicals about it, hence it never clicked me, my user agent might be a problem!

While writting the part 1, i noticed the header value of User Agent String which was some rubbish and it just struck me!! I set it back to default and everything was fine again :)

Moral Of this Post - Don't just believe what You read anywhere, no matter who says it, try it out on your own and really test it before you believe it! Ask questions! Challenge what you learn! Learn the same thing in different ways. And you'll get awesome everyday :)

Have Fun! :)

References :-

• Add-On
• Switching Agents in other browsers
• Resetting User Agents in firefox

HTTP, Web Scraping and Python - Part 1

Today we'll see what is this Web Scraping. We'll also learn HTTP protocol but I promise i'll make it more hands on rather than all jargon which you can read online anyway :)

So what is this HTTP thing?

Simply put, it's how our computers (Clients) Talk to Big Computers (Servers) and get the cool stuff done for us.

So when you go to wiki and open a link, internally HTTP requests are made and that gets your page to your screen via browser.


Let's check how its done.
Goto this page and it loads up a wiki page. But what really happened? Here's a sample HTTP request that was made...

----------Request From Client to Server----------

GET /wiki/Python_(programming_language) HTTP/1.1
Host: en.wikipedia.org
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://en.wikipedia.org/wiki/Python
Cookie: clicktracking-session=QgVKVqIpsfsgsgszgvwBCASkSOdw2O; mediaWiki.user.bucket:ext.articleFeedback-tracking=8%3Aignore; mediaWiki.user.bucket:ext.articleFeedback-options=8%3Ashow

----------End of Request From Client to Server----------

----------Response From Server to Client----------

HTTP/1.0 200 OK
Date: Mon, 10 Oct 2011 12:44:46 GMT
Server: Apache
X-Content-Type-Options: nosniff
Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
Content-Language: en
Vary: Accept-Encoding,Cookie
Last-Modified: Sun, 09 Oct 2011 05:01:32 GMT
Content-Encoding: gzip
Content-Length: 47407
Content-Type: text/html; charset=UTF-8
Age: 10932
X-Cache: HIT from sq66.wikimedia.org, MISS from sq65.wikimedia.org
X-Cache-Lookup: HIT from sq66.wikimedia.org:3128, MISS from sq65.wikimedia.org:80
Connection: keep-alive

----------End of Response From Server to Client----------

I have highlighted the important part of the request and response.

HTTP has several kinds of request out of which one is GET request. 

That's why we had this line below :-
GET /wiki/Python_(programming_language) HTTP/1.1




It says "Oye  you Big Computer (Server) GET me this python page whose address is /wiki/Python_(programming_language) and I am talking in language HTTP whose version is 1.1 and the website that i want is mentioned in Host Parameter (Below)"
Host: en.wikipedia.org 

You ever wondered how the websites know what browser you are using, what operating system you are using....? What happens is, when your browser makes a request, it adds to the request what is called as headers (Even the host parameter is part of header), One of the parameter - User Agent, specifies where the request is coming from. So here I am using Linux with Firefox version 7.0.1 (keeping it simple here)

User-Agent: Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 

I'll cover other things later in request (Cookies and stuff). In response we get

HTTP/1.0 200 OK
Server: Apache


Above simply means "Hey...Nice to see you again....Everything is fine :), i have done my job and btw I am Apache" 

(Ignore if you don't know what is apache)

So Let's fire this up in python and see how we can do this on our own....

Note Anything in the line (except the first line) with a # is a comment. I have made it Bold+Italic to highlight it. 

-----------http_Get_Request_1.py---------------  

#!/usr/bin/env python

#Python library for making HTTP connections/Requests
import httplib

#make connection with the host using http protocol with port 80
connection = httplib.HTTPConnection("en.wikipedia.org",80)

#make a GET request for the resource mentioned
connection.request("GET", "/wiki/Hello_world")

#Get the response and save it resp
resp = connection.getresponse()

#Print the Response and see the textual description of it
print resp.status, resp.reason

#Save the data
data = resp.read()

#write the html data to file
page = open("file.html", "w")
page.write(data)
page.close()

#Close the HTTP Connection

connection.close()

-------End of http_Get_Request_1.py--------------- 



If you run the above code you should get something like this
 
mankaj $ python http_Get_Request_1.py
200 OK
mankaj $ ls
file.html  http_Get_Request_1.py
mankaj $



You can see that you got a new file called file.html. If you open that file you can see that web page. Notice that pictures are missing. Does it give you any clue of how requests are made internally?? Any ideas???

All the requests are not made in one Go! It fetches the skeleton of the web page, and as it encounters (the browser) new links, it makes a seperate request for each picture. You might have known this, but now you saw it yourself :)

Let's wrap it up with one thing you might be wondering! How the hell I saw that HTTP request and response ?? :)

There is something called as LiveHTTPHeader . It's a Firefox Plugin which allows you to see What is happening Internally. (Other browsers might have something like this, just google for it). Just download it. and Restart your Firefox browser. Go to that wiki page. Once you are there. Go to Tools >  Live HTTP Headers. And reload the page. You can see all the requests made. Go to the top and you can see the first request made and later internal requests made to load the whole of page. Don't get confused. It's just to give you a taste of what is about to come in the next set of tutorials :)

Have Fun!! :)

Update : Read Part 2 here

References :-

• Atul Alex Cherian's Script (which I modified and used)
• Siddhant's PyCon Slides
• HTTP Definitive Guide (Amazing Book)
• Python For Newbies Video Series (Good if you don't know python at all!!)

Simple App on Google App Engine using Python

Hi,
Today we'll try to make a simple App on Google App Engine in Python which just says 'YourName Rulz!!' and we'll upload it on Google App Engine :)

To get what I am saying, goto this page and check it out :)


That's what we are going to make today. Isn't that cool! You are about to write your name in history! Well on the web and as long as the sever lasts but it still rocks! :)

Once you are done with this tutorial, you can try and solve this activity at reliscore - 
Google AppEngine using Python to see how much you really understood :)

Now what is Google App Engine? Simply, its a way to deploy your web apps and let google's infrastucture do all the hard work for you! It will do all the cloud computing *cool* stuff for you!

So Let's get started.

Go to this url and download the app engine for your OS. I am using Fedora 15 KDE for it. And I recommend you do start using linux if you haven't. Here's a link to my friend's blog post on Getting Started with linux . Its a tutorial for the absolute newbies.

Okay Back to the game. So now that you have downloaded it. For Windows/Mac please check the installations note.

For linux you just have to unzip the folder somewhere, lets say /home/yourusername/workspace/gapp

Fire up your terminals and move into the directory using cd command.
just in case you don't know



$ cd /home/yourusername/workspace/gapp


create a new directory called hello
and inside that directory
write a file called hello.py
and in that file
write these contents


------------hello.py------------
print 'Content-Type: text/plain'
print ''
print 'yourname Rulz'
------------X--X---------------


save it.
Note1 : Make sure you have an empty line After Content-Type else you will have issues.

Note2: Replace yourname with YOUR name :p ... Incase you haven't figured that out ;)

open another file and call it app.yaml
and write these contents

------------app.yaml------------
application: helloworld
version: 1
runtime: python
api_version: 1

handlers:
- url: /.*
  script: hello.py
------------X--X---------------





Note: Make sure url and script are aligned. I had issues when they were not aligned.

Basically, this is like the app configuration file. Simply put, this tells that the application name is helloworld (we will change this later, why we will see that), the version is 1, you are using python as a language. And the handlers just says simply right now is that whatever sub url you give, all will be handled by the script hello.py given by us. If you don't get it, its okay...

and save it.

to test it run this command

$ /home/yourusername/gapp/dev_appserver.py /homeyourusername/hello/ 

(Note: You don't need to give full path, but to keep it simple and universal, I am writing full path. You can write it whatever way you like as long as its correct.)

Once it works, open your favourite browser and check it aout! :)


http://localhost:8080/




Ta ra! :)

"Hey, but thats not deployed on the web" you say! Aye. That's Next ;)

Okay go to this site and login with your google account.

Once you are done, choose create application. Choose a nice "Application Identifier" as thats what you will be sharing it with others. In my case it was firesofmay

Fill in the Application title and let other things be default and click "Create Application".

Now go back to the hello folder and open up the app.yaml file and modify the
"application: helloworld" to whatever your application identifier was. In my case it was
"application: firesofmay"

Once you are done issue this command :)


$ /home/yourusername/workspace/gapp/appcfg.py update /home/yourusername/workspace/helloworld/




If you have done everything correctly you should see something like this as output...

-------------------------
Application: firesofmay; version: 1
Host: appengine.google.com

Starting update of app: firesofmay, version: 1
Scanning files on local disk.
Cloning 2 application files.
Compilation starting.
Compilation completed.
Starting deployment.
Checking if deployment succeeded.
Deployment successful.
Checking if updated app version is serving.
Will check again in 1 seconds.
Checking if updated app version is serving.
Will check again in 1 seconds.
Checking if updated app version is serving.
Completed update of app: firesofmay, version: 1
-------------------------

Open up your URL and check it out... isn't that super cool!! :)
It's time for you to go to your facebook and post some cool stuff online and show it off to your friends!

Here are the references :-

• Google App Engine - Intro
• Google App Engine - Basics
• Google App Engine - Hello
• Google App Engine - Uploading
• Google's Python Class
• Google AppEngine using Python - Problem Challenge @ Reliscore

I want to thank Navin Kabra, for being a great support and starting such a cool website - reliscore for programmers like you and me who love coding real world problems. And its a Kick Ass Website for those who wanna show off and get job for it too ;) Now Go and sharpen your coding skills on that website.

Cya :)

How to fix Wifi on fedora 15

Add free repos and non free repos

Fedora 13, 14 and 15:

su -c 'yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm'

yum install broadcom-wl akmod-wl 

sudo akmods
sudo modprobe wl

Xooglers

A PDF archive of the blog Xooglers which has posts of X Google employees on there experience of being in Google.
Worth reading it.
Cheers.

Link to download from.

Sed and Awk - From a Pen Testers Perspective

Today we are going to see what sed and a little bit of awk are and how we can use it in Pen Testing. I am using Backtrack 4 in vmware.

Recently a hacker group called Anonymous attacked rootkit.com and uploaded the whole database of the site online. Its a huge database. With some 80,000+ users and many other information in it.

Lets say we are the Pen Testers. And we just got hold of that huge database. Inside the database there are many tables, but the one that is of your interest is called - people and it has many fields, but you are interested in only the user name and its hashed password. We want to extract all the user name and its hash in this format

userid1:password1

userid2:password2

....

You certainly can't do this manually. And it doesn't suit us doing manual work, Ahmm .. CTRL-C -> CTRL-V :).

We like to do things fast and smart.... right ;)
Sed and Awk (and grep too) to the rescue!

So what is this... Sed

Sed is a Stream Editor in which we feed some text, and it processes them line by line and performs some commands which manipulate the text in the way we want. For example We can replace all " " to ":" or replace all the occurrences of the string "hello" to "hi" and many awesome stuffs.
Hold ...Hold, before you say "  big deal. That I can do with Replace All command in my Notepad"  (yeah even I thought the same before I learned Sed)

OK, Lets start the Magic Show.

Here's the link, download the gzip file rootkit_com_mysqlbackup_02_06_11.gz , and paste it in any folder in your Linux machine.

Once you have downloaded the file, rename the file so that its small

root@bt:~/blog# mv rootkit_com_mysqlbackup_02_06_11.gz database.gz

now we need to decompress the file.

root@bt:~/blog# gunzip database.gz


Now that you have decompressed it, you should have a file called database (without any .gz)

OK. Just to get a feel of how BIG the database is just do the Cat command on the database file, go have some coke, sleep and come back. :)
......
......
No, don't worry, it is possible to extract fields from this huge file in a very clever yet elegant way. Hold On, magic is about to begin.

OK. First we need to know what we are dealing with.
Open database file in Vim.
We will search for all the Create Table Statements. Do

/CREATE TABLE [ENTER]

and keep pressing "n" for next occurrence of the given string. You will notice that there are so many tables in this database. Keep going till you hit the people database. Saw? OK. Now keep going down (using down arrow key) slowly and keep noticing the fields (yeah there are many fields in this table). You will hit the insert into table and notice that the insert into line span for multiple lines.

INSERT INTO `people` VALUES (1,'admin','51a42fa118e77f95f70d4efff4395f8d','rootkit sysop','hoglund@rootkit.com',10,0,'','','','','','',0,'http://www.rootkit.com/usericons/admin.jpg','',1296966693,'213.243.145.60',1296705113,1283501911,1296457930,1294556469,1294942812,0,0,'','','','','',-1,'P'),(2,'aaronh','7cb8b36d','Aaron Heady','hackdoctor@aol.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/aaronh.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''),(3,'abcdef','e80

 ..........................................................................and so on

To be sure that one line is really spanning multiple lines do the following command inside Vim

[ESC] :set nu

And it should show you the line number at which insert into is. If you are at first insert into
(1,'admin','51a42fa118e77f95f70d4efff4395f8d','rootkit...
then the line should be 425.

Anyway the point was it is spanning multiple lines. It is important to know this. Why? Because cut command cuts per line.

What is this Cut command you ask? We will see that later in detail. The mist is about to clear. Forget Cut for now.

Now that you know what lines you want to work on i.e. INSERT INTO `people`
(Note around people is not single quotes but backtick, which lies above the tab key)

To select only particular lines from this file we will use grep command.
Grep command takes a file as input and one or more strings to match. The lines that are matched are returned from that file. There are many more features of grep (check out this command -> #man grep)

ok quit from the Vim
To quit Vim
[ESC] :q!

Oh just press CTRL-L to clear the screen. :) if you are wondering how to clear so much clutter on the screen.

OK Do,

root@bt:~/blog# grep "INSERT INTO \`people\`" database

You will see a huge amount of output even now, but don't worry we have extracted out only the INSERT INTO `people` statements.

Wait why we put those \ in front of Backticks "`" ?
Because backticks are special characters and we want linux to treat them as normal characters. To make any special character normal character we put "\" in front of them.

Ok. Now Very Important part.
Each insert into statement is inserting many values within.
(1,'admin',...),(2, 'aaronh',....) etc


To simplify things we are going to put each row of value in seperate line.
How we are going to do this? By asking sed to substitute "),(" with a newline. Why "),(" ? Because that is where your one row is ending and a new one is beginning.

So we do,

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g

Ok. Is that too much? :) Don't worry i'll explain.

1. grep "INSERT INTO \`people\`" database 
• What we did before.
2. |   
• Pipe Command. What it does is, it passes the output of one command as input to the other command. So here, the selected text from the grep command is passed as input to the sed command. Simple!
3. sed s/\)\,\(/\\n/g
• here s stands for substitute.
• what to replace is told after the first /
• i.e. to replace string "),("
• we added \ before ) and ( so that its treated as normal characters not special characters.
• 2nd / specifies the string to replace with
• to replace with string is newline, i.e. \n but since \ is a special character we make it normal character buy adding one more \ :)
• 3rd / specifies substitute all the occurrences (g = global) of the, to replace string
• Note Replacing a string and putting a newline is something you cannot do in notepad with replace All :)

Now when you run the command, you yet cant see the output.
Append the command with a head command. By default head command outputs only first ten lines of text file given (and tail command does the opposite)

So just do

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | head  

and this should be the output
.....

INSERT INTO `people` VALUES (1,'admin','51a42fa118e77f95f70d4efff4395f8d','rootkit sysop','hoglund@rootkit.com',10,0,'','','','','','',0,'http://www.rootkit.com/usericons/admin.jpg','',1296966693,'213.243.145.60',1296705113,1283501911,1296457930,1294556469,1294942812,0,0,'','','','','',-1,'P'

2,'aaronh','7cb8b36d','Aaron Heady','hackdoctor@aol.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/aaronh.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

3,'abcdef','e80b5017','Ashish Rungta','ASHTME@YAHOO.COM',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/abcdef.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

4,'abel','cd779e8a','Adi A','adia@opsynet.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/abel.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

5,'abhi0070','a6a7c0ce','kbcack','unknownbuddy@yahoo.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/abhi0070.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

6,'abm','e50624ea','alex murphy','abm@mitre.org',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/abm.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

7,'abraxas','7f9cc44f','Alex Mellor','i_love_g0ats@hotmail.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/abraxas.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

8,'acc_chen','c4025c6f','Jun.chen','johnychen@netease.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/acc_chen.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

9,'access55','e9f5bda6','WHY YOU ASK','access55@manx.net',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/access55.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

10,'accobra','66f363a6','Rob','rlstephe@yahoo.com',1,0,'','','','','','',0,'http://www.rootkit.com/usericons/accobra.jpg','',0,'',0,0,0,0,0,0,0,'','0','','','',-1,''

..................

I can see a faint smile on your face now :) Don't worry you will be having a strong urge to show off at the end of this tutorial. Just a few steps more.

Ok. Before we extract out the user and  password. You need to know what is cut command. The cut command works on field seperators.

To understand how cut works before we contd, lets take an example. Do

root@bt:~/blog# cat /etc/passwd
.....
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
........

All the fields in this file are separated by ":". Userid is the first one, 2nd password, 3rd uid and so on.

What if you want to extract only user and uid from this file?
This is where cut comes into play. Cut command works on field separators, for each line. By default field separator is space but we can specify any character as field separator.

Do

root@bt:~/blog# cut -d":" -f1,3 /etc/passwd 

........

root:0
daemon:1
bin:2
sys:3

..........

Nice.. Now we are gonna cut each line by comma. Why? Cause each field is seperated by commas. We want user and hash which is 2nd and 3rd field if the field separator is ","

So we do

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | cut -d "," -f2,3 | head

I have put cut command before head. We are simply saying to cut as
field seperator , ( -d "," )
and select only column 2 and 3 ( -f2,3)
and pass it to head so that we get first ten lines only (its easy to see output and be sure that the command is working)

It should give
.........

'admin','51a42fa118e77f95f70d4efff4395f8d'
'aaronh','7cb8b36d'
'abcdef','e80b5017'
'abel','cd779e8a'
'abhi0070','a6a7c0ce'
'abm','e50624ea'
'abraxas','7f9cc44f'
'acc_chen','c4025c6f'
'access55','e9f5bda6'
'accobra','66f363a6'

.....

Sugoi

Now we just need to remove those single quotes.
You know what to do :)

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | cut -d "," -f2,3 | sed s/\'//g | head

i.e. replace all (/g)
single quotes ( /\' as Single Quotes is a special character)
with Nothing ( // )
......
admin,51a42fa118e77f95f70d4efff4395f8d
aaronh,7cb8b36d
abcdef,e80b5017
abel,cd779e8a
abhi0070,a6a7c0ce
abm,e50624ea
abraxas,7f9cc44f
acc_chen,c4025c6f
access55,e9f5bda6
accobra,66f363a6
....

Last thing, and its Game Over, replace "," with ":"

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | cut -d "," -f2,3 | sed s/\'//g | sed s/,/:/g | head

:) Done!

You can now remove the head command to check if its working for all files.

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | cut -d "," -f2,3 | sed s/\'//g | sed s/,/:/g

Awesome!
Wait there's more.
What if you want hash first and user name later? i.e.
password1:user1
password2:user2
....

:)

With sed you can do, but its too complicated.
Meet Sed's Elder brother Awk. Awk is more powerful (and more complicated). Awk is used mainly for data extraction and reporting tool.

So do,

root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | cut -d "," -f2,3 | sed s/\'//g | sed s/,/:/g | awk -F':' '{print $2 ":" $1}' | head

What we did?
-F is like -d in cut command, i.e. specifying field separator.
Each field is put in $1 $2 $3 etc, where $1 is username here and $2 password.

By '{print $2 ":" $1}'
we are asking it to print it in reverse order. Don't give Comma in between $2 and $1 as it will replace the field separator with space.

Output ->
.......
51a42fa118e77f95f70d4efff4395f8d:admin
7cb8b36d:aaronh
e80b5017:abcdef
cd779e8a:abel
a6a7c0ce:abhi0070
e50624ea:abm
7f9cc44f:abraxas
c4025c6f:acc_chen
e9f5bda6:access55
66f363a6:accobra
..........

Now just redirect the final output (without the head command) to a file.

Do,
root@bt:~/blog# grep "INSERT INTO \`people\`" database | sed s/\),\(/\\n/g | cut -d "," -f2,3 | sed s/\'//g | sed s/,/:/g | awk -F':' '{print $2 ":" $1}' > output.txt

With > output.txt we are redirecting the final output of awk command into the file i.e. output.txt rather than on the terminal.

Now just for curiosity, to count the number of users we got we do

root@bt:~/blog# wc output.txt

.....

81450   82345 3145329 output.

 .....
wc  command prints
1) newline
2) word
3) byte
counts for each file

So we have 81450 users in this final output.txt Pheww Thats a lot!



OK.

[ OPTIONAL ]

Super Crisp Command Mode

Above command has Total 6 Commands (excluding > output.txt) i.e.

grep "INSERT INTO \`people\`" database
| sed s/\),\(/\\n/g
| cut -d "," -f2,3
| sed s/\'//g
| sed s/,/:/g
| awk -F',' '{print $2 $1}'


Do you think we can crisp it down to only 2 Commands!!
No, I am not crazy. It is possible.
Why do I want to do it? Just cause we can ;)


Wanna See ? :)
Here's the command


root@bt:~/blog# sed -n /"INSERT INTO \`people\`"/s/\),\(/\\n/pg database | gawk -F\' '{print $4 ":" $2}' | head
.....


[ Head command is for you to see the short version of the output, not required though ]


What is going on? Thats for you to figure it out.


Have Fun. :D


Here are the references that was made during the writing of this tutorial.

1. Sed
2. Awk
3. Basic Linux
4. Grep
5. Anonymous
6. Sed & Awk Book
7. HackingDojo

Cheers!

VIM Config File

Finally I was able to change the vim configuration file in windows (yes it exists in windows too psss - even i didn't know :) )

To configure your configuration file open gvim (graphical vim) and type
(NOTE : Don't Type ESC OK!!)

ESC :e $MYVIMRC

it will edit your vimrc file (configuration file)

Paste the code below
-------------------------------
if has('gui_running')
  set guifont=Lucida_Console:h16
  set noerrorbells
 
  if has('autocmd')
      autocmd GUIEnter * set vb t_vb=
  endif

  set backspace=indent,eol,start
  fixdel

  syntax enable
  set number
  set autoindent
  set tabstop=4

endif


-----------------------------------
ESC :wq

and Volla!! Restart your Vim and you will see the changes.

The above code will fix :-

1. Font Size
2. No More Annoying Beeps (trust me you dont want to hear after some time)
3. Shows Line Number next to each line very helpful.
4. Fixes Your backspace key to work, (originally it doesnt work :o )
5. A little bit of indentation to the line so the lines are more readable.

It took me a few hours to do this, hope you guys save your time. I'll keep updating the code here whenever I find more cooler configuration settings, so keep checking..

Cheers!! :)

HOWTO Install VMWare Tools in Backtrack 4

Hi,
Finally I was able to install VMWare Tools

Here's what you do.

To install VMware Tools in a Linux guest
1. On the host, select VM > Install VMware Tools.
If an earlier version of VMware Tools is installed, the menu item is Update VMware Tools. If the current version is installed, the menu item is Reinstall VMware Tools.

2 On the guest, log in as root.

3 If your Linux distribution does not automatically mount CD-ROMs, mount the VMware Tools virtual CD-ROM image.

a. If necessary, create the /mnt/cdrom directory:
mkdir /mnt/cdrom

b Mount the CD-ROM drive.
for backtrack 4 -> do ls and see if its cdrom3. For me it was cdrom3.

mount /dev/cdrom3 /mnt/cdrom


4 Change to a working directory (for example, /tmp):

cd /tmp

5 If a previous installation exists, delete the previous vmware-tools-distrib directory.
The location of this directory depends on where you placed it during the previous installation. Often it is placed in /tmp/vmware-tools-distrib.


6 Uncompress the installer:

do tar zxpf /mnt/cdrom/V and tab it will automatically show you the actual file.

exact command is
tar zxpf /mnt/cdrom/VMwareTools-<x.x.x>-<yyyy>.tar.gz
The value <x.x.x> is the product version number and <yyyy> is the build number of the product release.

7 If necessary, unmount the CD-ROM image.
If your Linux distribution automatically mounted the CD-ROM, you do not need to unmount the image.
umount /dev/cdrom

8 Run the VMware Tools installer.
cd vmware-tools-distrib
./vmware-install.pl

Just Press Enter to accept the default value. Dont worry about what is being asked for now :)
The configuration file, vmware-config-tools.pl, runs after the installer file finishes running.


And you are done!! :)

Cheers!!