Saeid Zebardast's Blog » howto

How to find IP locations using MySQL

Saeid Zebardast — Mon, 14 Mar 2011 15:18:59 +0000

Introduction
Here we learn how to use MySQL for locating an IP address. At first we create a table that contains the name of the country and its IP address range. Then define a function to find country name of IP Address.

What’s Ip Address?

Quoting from the “IP address” page in Wikipedia:

An Internet Protocol address (IP address) is a usually numerical label assigned to each device (e.g., computer, printer) participating in a computer network that uses the Internet Protocol for communication. An IP address serves two principal functions: host or network interface identification and location addressing. Its role has been characterized as follows: “A name indicates what we seek. An address indicates where it is. A route indicates how to get there.”

Table (database)
I created the ip_location table to save countries and IP Address ranges. Enter the following command:

CREATE TABLE `ip_location` (
  `from_ip` int(15) DEFAULT NULL,
  `to_ip` int(15) DEFAULT NULL,
  `country` varchar(32) DEFAULT NULL,
  KEY `from_ip` (`from_ip`,`country`),
  KEY `to_ip` (`to_ip`,`country`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;

Then you need to import ip_location data. I exported my table with data via mysqldump. You should download it and use mysql command to restore it.
• Download ip_location.sql.zip (165KB)

getIpCountry() function
Now you need to create the function to extract country name:

DELIMITER $$
CREATE FUNCTION getIpCountry(ip varchar(15)) RETURNS varchar(64)
BEGIN
 declare a tinyint unsigned;
 declare b tinyint unsigned;
 declare c tinyint unsigned;
 declare d tinyint unsigned;
 declare total bigint;
 declare result varchar(64);
 select substring_index(ip, '.', 1 ) into a;
 select substring_index(substring_index(ip , '.', 2 ),'.',-1) into b;
 select substring_index(substring_index(ip , '.', -2 ),'.',1) into c;
 select substring_index(ip, '.', -1 ) into d;
 set total := (a*256*256*256) + (b*256*256) + (c*256) + d;
 select SQL_CACHE country into result from ip_location where total between from_ip and to_ip limit 1;
 if (result is null) or (result = '') then
 set result := 'unknown';
 end if;
 return result;
 END$$
DELIMITER ;

And done! You just need to use select command in MySQL cli:

mysql> SELECT getIpCountry('79.175.165.171');
+--------------------------------+
| getIpCountry('79.175.165.171') |
+--------------------------------+
| IRAN (ISLAMIC REPUBLIC OF)     |
+--------------------------------+
1 row in set (0.03 sec)

mysql> SELECT getIpCountry('4.2.2.4');
+-------------------------+
| getIpCountry('4.2.2.4') |
+-------------------------+
| UNITED STATES           |
+-------------------------+
1 row in set (0.00 sec)

Let me know if you have other way

About Saeid Zebardast

I'm Senior software engineer with 5+ years of professional experience includes cross-platform proficiency with considerable knowledge of programming languages especially Java and programming paradigms such as OO and development methodologies. Also I'm MySQL DBA since 2006.

No related posts.

How to get pure content from HTML page in Java via Regex

Saeid Zebardast — Wed, 19 Jan 2011 14:19:00 +0000

Introduction
I’ve written a web crawler while I was developing a search engine a few weeks ago. It extracts the contents and saves them onto the database. The HTML tags aren’t so important to most of the search engines. So, I removed them successfully. To do the same, follow below steps:
1- Remove the script tags and inclusive content:

// htmlContent is full content of page with HTML codes.

String content;
Pattern pattern;

pattern = Pattern.compile(".*?", Pattern.DOTALL | Pattern.CASE_INSENSITIVE);
content = pattern.matcher(htmlContent).replaceAll("");

Note: In dotall mode, the expression . matches any character, including a line terminator. By default this expression does not match line terminators.

2- Remove the style tags and inclusive content:

String content;
Pattern pattern;

pattern = Pattern.compile(".*?", Pattern.DOTALL | Pattern.CASE_INSENSITIVE);
content = pattern.matcher(content).replaceAll("");

3- Remove all HTML tags without inclusive content.

pattern = Pattern.compile("<[^>]*>");
content = pattern.matcher(content).replaceAll("");

4- Replace new lines, tabs and multiple spaces with a single space.

content = content.replaceAll("\n+", " ");
content = content.replaceAll("\t+", " ");
content = content.replaceAll("(  )+", "");

And you have a pure content now

Links
Regular expression
How to Write an HTML Parser in Java
Regular-Expressions.info

About Saeid Zebardast

No related posts.

Fetch full posts content of FeedWordPress feeds

Saeid Zebardast — Mon, 05 Jul 2010 03:57:59 +0000

I use WordPress and FeedWordPress plugin to create a planet. It’s great plugin. Some bloggers don’t show full post content on their feeds. If you like to get the full content of posts, you can contact to blogger and ask his/her to enable full content on the feed or continue to read this article.
I create functions to get full content of posts.

Requirement

PHP with cURL support (Client URL Library)
Permissions to modify theme files
Text editor
basic php programming skills

Step 1 – Where is post content?
It’s easy. just open the web page and see the page source.
For example, open http://zebardast.ir/en/linux-and-unix-bash-shell-aliases/ (Single post with full content) and see the page source.
On the page source you can see the content which is started by below code:

and ended by :

* It’s not ended only by because there is some divs on post content. So I add some html code after which is unique.

Step 2 – Add started and ended code to `Custom Feed Settings`
Open the wordpress administration panel and go to the `Feed and Update Settings` page. Select the feed from drop down menu (Here `Saeid Zebardast’s Blog`).
Add started and ended code to `Custom Feed Settings`:

— Add started and ended code to `Custom Feed Settings`

Step 3 – Fetch full content from source and update post on wordpress
Open functions.php in text editor and add the below codes to the end of it:

 true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => false,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => "ayy.ir spider", // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
    );

    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

function before ($this, $inthat)
{
    return substr($inthat, 0, strpos($inthat, $this));
}; 

function after ($this, $inthat)
{
    if (!is_bool(strpos($inthat, $this)))
    return substr($inthat, strpos($inthat,$this)+strlen($this));
}; 

function multi_between($this, $that, $inthat)
{
   $counter = 0;
   while ($inthat)
   {
      $counter++;
      $elements[$counter] = before($that, $inthat);
      $elements[$counter] = after($this, $elements[$counter]);
      $inthat = after($that, $inthat);
   }
   return $elements;
} 

function strbet($inputStr, $delimeterLeft, $delimeterRight, $debug=false) {
    $posLeft=strpos($inputStr, $delimeterLeft);

    if ( $debug ) {
    	echo $posLeft;
    }

    if ( $posLeft===false ) {
        if ( $debug ) {
            echo "Warning: left delimiter '{$delimeterLeft}' not found";
        }
        return false;
    }
    $posLeft+=strlen($delimeterLeft);
    $posRight=strpos($inputStr, $delimeterRight, $posLeft);
    if ( $posRight===false ) {
        if ( $debug ) {
            echo "Warning: right delimiter '{$delimeterRight}' not found";
        }
        return false;
    }

    if ( $debug ) {
    	echo $posLeft;
    	echo $posRight;
    }

    return substr($inputStr, $posLeft, $posRight-$posLeft);
} 

?>

Close functions.php and open single.php in text editor. Add the below codes after ``:

After it, replace `the_content()` with:

 echo $my_content;

Close text editor and Upload functions.php and single.php to your theme folder. Now go to the single post and see the full content.
Just try it!

External links
• WordPress
• FeedWordPress (Homepage)
• FeedWordPress (WordPress plugin directory)
• Client URL Library

Good luck

About Saeid Zebardast

No related posts.

How to find hardware information from command line on Linux

Saeid Zebardast — Mon, 01 Mar 2010 14:25:26 +0000

— linux

In this post you can see some useful command to find and show hardware information on Linux.

Default commands

ALL devices

dmesg

dmesg will show you the kernel messages which can show you all the devices the kernel has found (hard disks,cdroms,etc)

CPU

# cat /proc/cpuinfo

Memory

# cat /proc/meminfo
$ free

PCI (including usb bridges,agp cards etc)

$ lspci

USB devices (mice,etc)

$ lsusb

Hard drives

# fdisk -l
$ df -h

Additional Command

lshw

lshw is a Linux command which provides details of all the hardware in your PC. The details provided by the lshw command run the gamut of processors, memory, slots, onboard sound, video chipset and more.

Install lshw

Arch:

# pacman -S lshw

Debain, Ubuntu or any of its derivatives:

$ sudo aptitude install lshw

Redhat, fedora, CentOS:

# yum install lshw

Gentoo:

# emerge lshw

Run lshw

# lshw
# lshw -short

To get the output in HTML, you use the -html option as follows:

# lshw -html > hardware-info.html

• See lshw command – List hardware information in Linux

dmidecode

dmidecode command reads the system DMI table to display hardware and BIOS information of the server. Apart from getting current configuration of the system, you can also get information about maximum supported configuration of the system using dmidecode. For example, dmidecode gives both the current RAM on the system and the maximum RAM supported by the system.
dmidecode is installed by default on many linux distribution like debain, ubuntu and fedora.
• See How To Get Hardware Information On Linux Using dmidecode Command

have a good time

About Saeid Zebardast

Synchronizing Google Tools with Kontact

Saeid Zebardast — Mon, 16 Nov 2009 08:34:12 +0000

Hello

Introduction
Using online tools has many advantages. One of the most important of them is the information is accessible from anywhere and at any time.
One need that was occurred after using this tools is Synchronizing information with other applications and devices. such as Synchronizing contacts between Mobile, Web and PC.

* The following guide is tested on Kubuntu 9.10 with KDE 4.3.3 environment.

Synchronizing Google Tools with Kontact
Step 1
Installing akonadi-kde-resource-googledata package. The package is available in Ubuntu 9.10 and Debian repositories.

sudo aptitude install akonadi-kde-resource-googledata

Step 2
After installing akonadi-kde-resource-googledata package, you need to add Google resources to Akonadi. Open Akonadi Console:

akonadiconsole

Click Add on Agents tab. Add a `Akonadi Google Calendar Resource` for Google Calendar and add `Akonadi Google Contacts Resource` for Google Contacts.

— Add Google Resources to Akonadi Console

Enter your username without `@gmail.com` in the next window.

— Enter your username

Step 3
After adding a resources open Kontact application :

kontact

Add Contacts
Go to Contact section and add `Akonadi Google Resource` to address books:

— Add `Akonadi Google Resource` to Address Books

Add Calendar
Go to Calendar section and add `Akonadi Google Resource` to calendars:

— Add `Akonadi Google Resource` to calendars

done

P.S.
Ubuntu 9.10 release party held in Tehran, Iran.

Have a good time

About Saeid Zebardast

No related posts.

How to extract embedded images from .xls, .ods or .odt

Saeid Zebardast — Thu, 14 May 2009 12:50:29 +0000

It’s simple. Just 4 steps:

Save a copy of the file as ODS (OpenOffice.org Spreadsheet).
Change the filetype of the copy to ZIP (for example rename File.xls to File.zip).
Open the ZIP.
In the directory called Pictures will be the images, just as in the original.

About Saeid Zebardast

No related posts.

How to posting to Identi.ca from the CLI

Saeid Zebardast — Mon, 16 Feb 2009 13:30:13 +0000

Identi.ca is an open source social networking and micro-blogging service.
I like Ideni.ca. It’s good service.

I use CLI (command line) everyday. I like to dented from it. It’s simple.
Just install cURL:

$ sudo apt-get install curl

And type:

$ curl -u username:password -d status="message" http://identi.ca/api/statuses/update.xml

You will receive a response containing the XML coding for your post which acts as a confirmation that your post was submitted.

Also you can create a shell file for this
Open a new text document and add the following, save it as identica.sh (or anything ending in sh):

#!/bin/bash
curl -u username:password -d status=″$1" http://identi.ca/api/statuses/update.xml

Make sure you change the chmod to 777 using the following command.

$ chmod 777 /path/to/file/

When you located the file (or add a bash prompt) it makes it simplier to identi.ca the rules. For example you can type the following to command to link to the file.

$ ./path/to/idetica.sh "Message"

About Saeid Zebardast

No related posts.

Install Oracle Instant Client and PHP OCI8 module

Saeid Zebardast — Sun, 10 Aug 2008 04:52:15 +0000

If you want to connect to an Oracle database with PHP, you can use Oracle’s Instant Client and the oci8 module from pear.

Download the Basic and the SDK packages from oracle.com. At the time of this writing, the filenames are instantclient-basic.zip and instantclient-sdk.zip.

Unzip these files in a new directory, e.g. /opt/oracle/instantclient.

Code:

sudo su
mkdir -p /opt/oracle/instantclient
cd /opt/oracle/instantclient
unzip instantclient-basic.zip
unzip instantclient-sdk.zip
echo /opt/oracle/instantclient >> /etc/ld.so.conf
ldconfig

The previous two lines are supposed to create symlinks named libclntsh.so and libocci.so which we will need later. In my case these symlinks were not created by ldconfig, so I created them manually.

Code:

ln -s libclntsh.so.11.1 libclntsh.so
ln -s libocci.so.11.1 libocci.so

In the next step we will download the oci8 module with pear. Pear is in the php-pear package.

Code:

apt-get install php-pear

Also, you need php5-dev and build-essential packages for compiling oci8 module.

Code:

apt-get install php-pear php5-dev build-essential

“Normally” we should be able to just use pecl install oci8 now, but apparently pear is not able to figure out where the instantclient libraries are. So we will just download the oci8 module and build it on our own.

Code:

mkdir -p /usr/local/src
cd /usr/local/src
pecl download oci8
tar xzf oci8-1.3.4.tgz
cd oci8-1.3.4
phpize
./configure --with-oci8=shared,instantclient,/opt/oracle/instantclient
make
make install

The oci8-1.3.4.tgz filename will of course change for newer releases.

To enable the oci8 module in the php.ini (/etc/php5/apache2/php.ini and /etc/php5/cli/php.ini), add a line
Code:

extension=oci8.so

(put this line after the examples starting with ;extension).

Now stop and start Apache. You should see the oci8 module in the output of phpinfo().

Good luck

About Saeid Zebardast

How to install gOS on Ubuntu Gutsy Gibbon

Nested X11 environment session

Saeid Zebardast — Mon, 07 Apr 2008 19:15:28 +0000

Instead of using a full-blown new virtual X for developing software you can use Xephyr to embed your KDE 4 session into your working KDE 3 or other X11 environment.

If you want to get a minimal KDE session up and running, just launch Xephyr (available in Kubuntu as xserver-xephyr; Gentoo users compile x11-base/xorg-server with USE=”kdrive”):

Xephyr :1 -extension GLX &

You can now launch KDE:

export DISPLAY=:1
/path/to/kde4/bin/startkde &

For other X11 environment just change /path/to/kde4 like:

/usr/bin/startx &

/usr/bin/startkde &

/usr/bin/startxfce&

You can use “locate” command to find paths like:

locate startkde

Appendix
• Happy Nowruz 1387 (Iranian new year holiday)

Good Luck

About Saeid Zebardast

No related posts.

Webmin, Installing on Ubuntu Gutsy Gibbon (7.10)

Saeid Zebardast — Tue, 22 Jan 2008 19:48:10 +0000

Hi
I wrote post about Webmin in ITPencil. My Persian friend like it. I decide to write post about Webmin and how to installing it on Ubuntu Gutsy Gibbon (7.10).
It’s very good for newbie.

from webmin.com:

Webmin is a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more. Webmin removes the need to manually edit Unix configuration files like /etc/passwd, and lets you manage a system from the console or remotely.

Installing Webmin On Ubuntu Gutsy Gibbon (7.10)
Webmin has some dependency package.
Install dependencies:

sudo aptitude install bash perl libnet-ssleay-perl openssl \
libauthen-pam-perl libpam-runtime libio-pty-perl libmd5-perl

After install dependencies, you can download and install Webmin. Last version of Webmin is 1.390.
Download Webmin:

sudo wget http://prdownloads.sourceforge.net/webadmin/webmin_1.390_all.deb

Install Webmin:

sudo dpkg -i webmin*.deb

Log in and use it:
Copy this URL into your web browser: https://localhost:10000

Screenshots
• Grub Boot Loader

• Apache Webserver

• System Documentation

• Scheduled Cron jobs

• Software Packages

Check out the screenshots.

External links
• Official Webmin website
• Webmin documentation wiki

Good luck

About Saeid Zebardast