Saturday, December 25, 2010

Linux RAM Drive

While developing software I occasionally have the need for a RAM drive. There are many reasons and situations where this comes in handy. Usually I use it to store temporary data files, that I can read/write to much faster since they now reside in RAM instead of on the hard drive. The reason is physics. Its faster to read and work on files stored in RAM than it is for a seek/read on the hard drive.

The way it works is simple. You create a folder to be your "RAM drive" - this is the directory that will be stored in RAM. Then you mount the directory as a "tmpfs" and specify the amount of system memory this new RAM drive will occupy. If you want to copy a file or work with a data file from a script in the RAM drive, place it in this directory and work with it the same way you would normally.

Here's the commands to create a directory, then mount it as a tmpfs (RAM drive).

mkdir -p /home/user/Desktop/ramdrive
mount -t tmpfs -o size=500M,mode=0777 tmpfs /home/user/Desktop/ramdrive

As you can see, the mounted RAM drive is 500Mb, and set to 777 mode (accessable and executable by anyone) - once the RAM drive is dismounted, all files within it are gone. Make sure to copy them out when you're done using it.

You can also drastically increase the speed at which Apache serves up your web pages using this method.

To test this on your LAMP create the RAM drive inside /var/www and test the speed of loading those pages as compared to files read from the hard disk.

Computer Biology

MySQL Commands

Some useful MySQL commands:

Alter a table to create a fulltext index of the columns specified..
ALTER TABLE tablename ADD FULLTEXT(column1, column2);

Search a fulltext index, using query expansion which increases relevancy but decreases the speed of the query. Further speed up the query by limiting the result set to 10 results.
SELECT * FROM tablename WHERE MATCH(column1,column2,column3) AGAINST('Search Phrase' WITH QUERY EXPANSION) LIMIT 10;

To import data from a text file into a MySQL database, assuming the text file is a list with each new record on a new line. For example:
Number One
Thing number 2
and a Third
Use the following MySQL command:
LOAD DATA INFILE '/home/user/Desktop/file.txt' INTO TABLE tablename (column1);
This import will only work on a database that is already setup with the proper column settings and data type.

Execute MySQL query from a bash shell script:


somevar=`echo "select column1 from testtable where idkey='5'" | mysql -h $IP_ADDR -u $USER_NAME -p$PASSWD $TABLE_NAME `

echo $somevar

Notes on the above bash script: You can replace the MySQL query in the script with any MySQL command.

Dont forget about the ticks ( ` ` ) that encapsulate the commands inside. This is a useful trick that took me a while to research and figure out. I wanted to set a variable to the results of some commands and found that you need to use the ` ` , not the ' (single quote) this tick shares the same key as the tilde key, to the left of the 1 key.

To view any query that may be hung up or taking a long time to execute:

show full processlist

This will display a table of the MySQL processes running, the process ID, and the Status.

If you find a rogue or otherwise not needed query you can kill that process ID using:

kill 1234
Where 1234 is the process ID that you want to end.

More coming soon.

Friday, December 10, 2010

LAMP Server Stack

Setting up a Linux Apache MySQL PHP (LAMP) server requires a few commands to get you up and running.

If you run Ubuntu or Debain - this is how I got started (Ubuntu 10.04) ; Open a terminal and run:
sudo apt-get install php5 php5-mysql mysql-server mysql-client mysql-query-browser apache2

As you can see from the list above, this command will install the following programs:
  • PHP ver 5
  • PHP MySQL Module
  • MySQL Server
  • MySQL Client
  • MySQL Query Browser GUI
  • Apache 2 Web server

During the MySQL install you will be asked to create a password for the MySQL root user. Remember this username/password.

Verify Apache is Operational

When the install finishes; Browse to http://localhost/ and look for the infamous  

"It Works!"

message. If you see this, Apache is up and running properly.

Verify PHP is Operational

Build yourself a file called test.php with the following code in it:
Save the file test.php in /var/www/ - this is the default local directory where Apache serves up your web documents.

Open a browser and enter the address: http://localhost/test.php

This should display a long page containing version information and current configuration of PHP.

This means that the file was executed properly and PHP is working.

Verify MySQL Server is Operational

From the terminal type:
mysql -u root -p
This command runs mysql as user (-u) root with a password of...(-p) You will be prompted to enter a password. This is the root MySQL password created when installing MySQL.

If you correctly enter the password you will be taken to the mysql> command prompt.

MySQL is working. Type: quit

Congratulations, you now have a LAMP server stack at your service!


Saturday, July 3, 2010

FTP - Packet Sniffing and Wireshark Analysis

FTP (File Transfer Protocol) as the name implies, is a protocol to transfer files from one computer to another. The protocol operates on TCP ports 20 for data transfer and 21 for control. The authentication, communication, and file data are all communicated in plain text; meaning *no* encryption is used.

The most common implementation of FTP is establishing a connection and transferring files with a web server. I configured Wireshark to monitor my wireless connection and then made an FTP connection to my web site.

The screenshot above shows the resulting packets that were transmitted. Here is what actually happened:
  • I execute the command to start the connection, and immediately a DNS query goes out to find the IP address of the host name I provided.
  • DNS responds with the proper IP address and TCP takes over to initiate communication with the server using a series of SYN and ACK transmissions. (TCP Three-way handshake)
  • FTP server says "Welcome" and my host transmits the user name to authenticate. (the username is transmitted in plain text)
  • FTP server responds saying the user name is OK but needs a password.
  • My client transmits the password across the Internet **in unencrypted text** readable to anyone
  • FTP server accepted the user name and password and grants the user access permissions.
  • Client sends the command "PWD" - and the server replies by Printing the Working Directory.
  • The communication ends with another series of SYN and ACK transmissions.

Do not use FTP to transmit anything confidential. This simple analysis shows that in 17 captured packets your server and files can be compromised.

The best alternative is FTPS (FTP and SSL) - from the wikipedia page:
FTPS (also known as FTP Secure and FTP-SSL) is an extension to the commonly used File Transfer Protocol (FTP) that adds support for the Transport Layer Security (TLS) and the Secure Sockets Layer (SSL) cryptographic protocols.
FTPS should not be confused with the SSH File Transfer Protocol (SFTP), an incompatible secure file transfer subsystem for the Secure Shell (SSH) protocol. It is also different from Secure FTP, the practice of tunneling FTP through an SSH connection.

Computer Biology