How to find IP locations using MySQL

Introduction
Here we learn how to use MySQL for locating an IP address. At first we create a table that contains the name of the country and its IP address range. Then define a function to find country name of IP Address.

What’s Ip Address?

Quoting from the “IP address” page in Wikipedia:

An Internet Protocol address (IP address) is a usually numerical label assigned to each device (e.g., computer, printer) participating in a computer network that uses the Internet Protocol for communication. An IP address serves two principal functions: host or network interface identification and location addressing. Its role has been characterized as follows: “A name indicates what we seek. An address indicates where it is. A route indicates how to get there.”

Table (database)
I created the ip_location table to save countries and IP Address ranges. Enter the following command:

CREATE TABLE `ip_location` (
  `from_ip` int(15) DEFAULT NULL,
  `to_ip` int(15) DEFAULT NULL,
  `country` varchar(32) DEFAULT NULL,
  KEY `from_ip` (`from_ip`,`country`),
  KEY `to_ip` (`to_ip`,`country`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;

Then you need to import ip_location data. I exported my table with data via mysqldump. You should download it and use mysql command to restore it.
Download ip_location.sql.zip (165KB)

getIpCountry() function
Now you need to create the function to extract country name:

DELIMITER $$
CREATE FUNCTION getIpCountry(ip varchar(15)) RETURNS varchar(64)
BEGIN
 declare a tinyint unsigned;
 declare b tinyint unsigned;
 declare c tinyint unsigned;
 declare d tinyint unsigned;
 declare total bigint;
 declare result varchar(64);
 select substring_index(ip, '.', 1 ) into a;
 select substring_index(substring_index(ip , '.', 2 ),'.',-1) into b;
 select substring_index(substring_index(ip , '.', -2 ),'.',1) into c;
 select substring_index(ip, '.', -1 ) into d;
 set total := (a*256*256*256) + (b*256*256) + (c*256) + d;
 select SQL_CACHE country into result from ip_location where total between from_ip and to_ip limit 1;
 if (result is null) or (result = '') then
 set result := 'unknown';
 end if;
 return result;
 END$$
DELIMITER ;

And done! You just need to use select command in MySQL cli:

mysql> SELECT getIpCountry('79.175.165.171');
+--------------------------------+
| getIpCountry('79.175.165.171') |
+--------------------------------+
| IRAN (ISLAMIC REPUBLIC OF)     |
+--------------------------------+
1 row in set (0.03 sec)

mysql> SELECT getIpCountry('4.2.2.4');
+-------------------------+
| getIpCountry('4.2.2.4') |
+-------------------------+
| UNITED STATES           |
+-------------------------+
1 row in set (0.00 sec)

Let me know if you have other way :)

How to get pure content from HTML page in Java via Regex

Introduction
I’ve written a web crawler while I was developing a search engine a few weeks ago. It extracts the contents and saves them onto the database. The HTML tags aren’t so important to most of the search engines. So, I removed them successfully. To do the same, follow below steps:
1- Remove the script tags and inclusive content:

// htmlContent is full content of page with HTML codes.

String content;
Pattern pattern;

pattern = Pattern.compile("<script.*?>.*?</script>", Pattern.DOTALL | Pattern.CASE_INSENSITIVE);
content = pattern.matcher(htmlContent).replaceAll("");

Note: In dotall mode, the expression <tt>.</tt> matches any character, including a line terminator. By default this expression does not match line terminators.

2- Remove the style tags and inclusive content:

String content;
Pattern pattern;

pattern = Pattern.compile("<style.*?>.*?</style>", Pattern.DOTALL | Pattern.CASE_INSENSITIVE);
content = pattern.matcher(content).replaceAll("");

3- Remove all HTML tags without inclusive content.

pattern = Pattern.compile("<[^>]*>");
content = pattern.matcher(content).replaceAll("");

4- Replace new lines, tabs and multiple spaces with a single space.

content = content.replaceAll("\n+", " ");
content = content.replaceAll("\t+", " ");
content = content.replaceAll("(  )+", "");

And you have a pure content now :)

Links
Regular expression
How to Write an HTML Parser in Java
Regular-Expressions.info

Fetch full posts content of FeedWordPress feeds

Hi :)

I use WordPress and FeedWordPress plugin to create a planet. It’s great plugin. Some bloggers don’t show full post content on their feeds. If you like to get the full content of posts, you can contact to blogger and ask his/her to enable full content on the feed or continue to read this article.
I create functions to get full content of posts.

Requirement

  • PHP with cURL support (Client URL Library)
  • Permissions to modify theme files
  • Text editor
  • basic php programming skills

Step 1 – Where is post content?
It’s easy. just open the web page and see the page source.
For example, open http://zebardast.ir/en/linux-and-unix-bash-shell-aliases/ (Single post with full content) and see the page source.
On the page source you can see the content which is started by below code:

<div  class="postBody">

and ended by :

			</div> 

						<div class="postFooter">

* It’s not ended only by </div> because there is some divs on post content. So I add some html code after </div> which is unique.

Step 2 – Add started and ended code to `Custom Feed Settings`
Open the wordpress administration panel and go to the `Feed and Update Settings` page. Select the feed from drop down menu (Here `Saeid Zebardast’s Blog`).
Add started and ended code to `Custom Feed Settings`:

Add started and ended code to `Custom Feed Settings`

Step 3 – Fetch full content from source and update post on wordpress
Open functions.php in text editor and add the below codes to the end of it:

<?php
function validLink($link) {
    if(preg_match('|^http(s)?://[a-z0-9-]+(.[a-z0-9-]+)*(:[0-9]+)?(/.*)?$|i', $link)) {
        return true;
    } else {
        return false;
    }
}

/**
 * Get a web file (HTML, XHTML, XML, image, etc.) from a URL.  Return an
 * array containing the HTTP server response header fields and content.
 */
function get_web_page( $url )
{
    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => false,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => "ayy.ir spider", // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
    );

    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

function before ($this, $inthat)
{
    return substr($inthat, 0, strpos($inthat, $this));
}; 

function after ($this, $inthat)
{
    if (!is_bool(strpos($inthat, $this)))
    return substr($inthat, strpos($inthat,$this)+strlen($this));
}; 

function multi_between($this, $that, $inthat)
{
   $counter = 0;
   while ($inthat)
   {
      $counter++;
      $elements[$counter] = before($that, $inthat);
      $elements[$counter] = after($this, $elements[$counter]);
      $inthat = after($that, $inthat);
   }
   return $elements;
} 

function strbet($inputStr, $delimeterLeft, $delimeterRight, $debug=false) {
    $posLeft=strpos($inputStr, $delimeterLeft);

    if ( $debug ) {
    	echo $posLeft;
    }

    if ( $posLeft===false ) {
        if ( $debug ) {
            echo "Warning: left delimiter '{$delimeterLeft}' not found";
        }
        return false;
    }
    $posLeft+=strlen($delimeterLeft);
    $posRight=strpos($inputStr, $delimeterRight, $posLeft);
    if ( $posRight===false ) {
        if ( $debug ) {
            echo "Warning: right delimiter '{$delimeterRight}' not found";
        }
        return false;
    }

    if ( $debug ) {
    	echo $posLeft;
    	echo $posRight;
    }

    return substr($inputStr, $posLeft, $posRight-$posLeft);
} 

?>

Close functions.php and open single.php in text editor. Add the below codes after `<?php if (have_posts()) : while (have_posts()) : the_post(); ?>`:

<?php
	 $my_content = get_the_content();
	 if (is_syndicated()) :

      	$syndication_permalink = get_post_meta(get_the_ID(),"syndication_permalink", true);
      	$syndication_source = get_post_meta(get_the_ID(),"syndication_source", true);
      	$syndication_source_uri = get_post_meta(get_the_ID(),"syndication_source_uri", true);

      	if (!validLink($syndication_permalink) && validLink($syndication_source_uri)) {
      		$syndication_permalink = $syndication_source_uri . "/" . $syndication_permalink;
      	}

	 		$post_updated = get_post_meta(get_the_ID(),"post_updated", true);
	 		if (empty($post_updated) || $post_updated == false)  {

		      $start_content = get_feed_meta('start_content');
		      $end_content = get_feed_meta('end_content');

		      if (!empty ($start_content) && !empty($end_content)) {
		      	$result = get_web_page($syndication_permalink);
		      	$my_page = $result['content'];

		      	if (!empty($my_page)) {
		      		$valid_texts = array();
		      		$valid_texts = strbet($my_page, $start_content, $end_content);
				if (is_array($valid_texts)) {
					$valid_texts = $valid_texts[0];
				}

		      		if (!empty($valid_texts)) {
		      			$my_post = array();
		      			$my_post['ID'] = get_the_ID();
		      			$my_post['post_content'] = $valid_texts;
		      			$my_content = $valid_texts;
		      			wp_update_post($my_post);
		      			update_post_meta(get_the_ID(), 'post_updated', true);
		      		}
		      	}
		      }
	 		}

	 endif; //is_syndicated()
  ?>

After it, replace `the_content()` with:

 echo $my_content; 

Close text editor and Upload functions.php and single.php to your theme folder. Now go to the single post and see the full content.
Just try it!

See also
How do I get FeedWordPress to include the full content of posts, instead of just a short summary or excerpt of the text?

External links
WordPress
FeedWordPress (Homepage)
FeedWordPress (WordPress plugin directory)
Client URL Library

Good luck :)

How to find hardware information from command line on Linux

linux

linux

Hi

In this post you can see some useful command to find and show hardware information on Linux.

Default commands

ALL devices

dmesg

dmesg will show you the kernel messages which can show you all the devices the kernel has found (hard disks,cdroms,etc)

CPU

# cat /proc/cpuinfo

Memory

# cat /proc/meminfo
$ free

PCI (including usb bridges,agp cards etc)

$ lspci

USB devices (mice,etc)

$ lsusb

Hard drives

# fdisk -l
$ df -h

Additional Command

lshw

lshw is a Linux command which provides details of all the hardware in your PC. The details provided by the lshw command run the gamut of processors, memory, slots, onboard sound, video chipset and more.

Install lshw

Arch:

# pacman -S lshw

Debain, Ubuntu or any of its derivatives:

$ sudo aptitude install lshw

Redhat, fedora, CentOS:

# yum install lshw

Gentoo:

# emerge lshw
Run lshw
# lshw
# lshw -short

To get the output in HTML, you use the -html option as follows:

# lshw -html > hardware-info.html

• See lshw command – List hardware information in Linux

dmidecode

dmidecode command reads the system DMI table to display hardware and BIOS information of the server. Apart from getting current configuration of the system, you can also get information about maximum supported configuration of the system using dmidecode. For example, dmidecode gives both the current RAM on the system and the maximum RAM supported by the system.
dmidecode is installed by default on many linux distribution like debain, ubuntu and fedora.
• See How To Get Hardware Information On Linux Using dmidecode Command

have a good time :)

Synchronizing Google Tools with Kontact

Hello :)

Introduction
Using online tools has many advantages. One of the most important of them is the information is accessible from anywhere and at any time.
One need that was occurred after using this tools is Synchronizing information with other applications and devices. such as Synchronizing contacts between Mobile, Web and PC.

* The following guide is tested on Kubuntu 9.10 with KDE 4.3.3 environment.

Synchronizing Google Tools with Kontact
Step 1
Installing akonadi-kde-resource-googledata package. The package is available in Ubuntu 9.10 and Debian repositories.

sudo aptitude install akonadi-kde-resource-googledata

Step 2
After installing akonadi-kde-resource-googledata package, you need to add Google resources to Akonadi. Open Akonadi Console:

akonadiconsole

Click Add on Agents tab. Add a `Akonadi Google Calendar Resource` for Google Calendar and add `Akonadi Google Contacts Resource` for Google Contacts.

Add Google Resources to Akonadi Console

Add Google Resources to Akonadi Console

Enter your username without `@gmail.com` in the next window.

Enter your username

Enter your username

Step 3
After adding a resources open Kontact application :

kontact

Add Contacts
Go to Contact section and add `Akonadi Google Resource` to address books:

Add `Akonadi Google Resource` to Address Books

Add `Akonadi Google Resource` to Address Books

Add Calendar
Go to Calendar section and add `Akonadi Google Resource` to calendars:

Add `Akonadi Google Resource` to calendars

Add `Akonadi Google Resource` to calendars

done :P

P.S.
Ubuntu 9.10 release party held in Tehran, Iran.

Have a good time :)