Archive for June, 2007

Use Google as a Spellchecker with Perl

Try googling for “recieve”, and you will receive the following messages above the results:

Did you mean: receive

This feature can be used to check texts for commonly misspelled words, and doing so is especially easy since the correctly spelled version is always highlighted with B and I elements. The following perl code uses LWP to accomplish the task.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

#!/usr/bin/perl
 
use strict;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent('Perl 5');
my $text = 'You will recieve good resullts.';
my @newtext;
foreach my $word ( split / /, $text ) {
    my $req = HTTP::Request->new( GET => 'http://google.com/search?q=' . $word );
    my $res = $ua->request( $req );
    if ( $res->content =~ m/<b><i>(.*?)<\/i><\/b>/i ) {
        push @newtext, $1;
    } else {
        push @newtext, $word;
    }
    sleep 1;
}
print "@newtext\n";

The output of the above script is: you will receive good results. Note, however, that this script is quite useless since abusing the Google servers like this is against their terms of service and there are better spell checkers available.

Generate Slugs from Strings with PHP

In these days of Web 2.0, no site can survive for long without pretty, mod_rewrite-modified permanent links. Everyone uses it. If you look at the URL for this post, for example, it is timjoh.com/generate-slugs-from-strings-with-php. As you might have guessed, I never typed that hyphenated string–WordPress did it all for me. If your PHP script is fed a string that you want a similar slug from, you can use this handy one-liner, with $string being the string that you want to slug.

1

$slug = str_replace( ' ', '-', preg_replace( '/[^a-z0-9- ]/', '', strtolower( $string ) ) );

Or you could use the neater version:

1
2
3

$slug = strtolower( $string ); // lower-case the string
$slug = preg_replace( '/[^a-z0-9- ]/', '', $slug ); // remove all non-alphanumeric characters except for spaces and hyphens
$slug = str_replace( ' ', '-', $slug ); // substitute the spaces with hyphens

This turns the string I'm an U_G_L_Y string! into the slug im-an-ugly-string. Note that it is often useful to shorten the slug; no-one wants a metre-wide URL. It is also important to check so that the slug doesn’t already exist. We can add to the code a loop that appends an ID number to the slug in case it already exists, making it unique.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

$slug = strtolower( $string ); // lower-case the string
$slug = preg_replace( '/[^a-z0-9- ]/', '', $slug ); // remove all non-alphanumeric characters except for spaces and hyphens
$slug = str_replace( ' ', '-', $slug ); // substitute the spaces with hyphens
$i = ''; // start with no appended value
while ( slugExists( $slug . $i ) ) { // if the slug already exists..
    $i++; // increment the appended value
    # it is bad practice to increment a string
    # but in this case, it simplifies the code
}
$slug .= $i; // append the value to the real slug
 
function slugExists( $slug ) {
    # this is just an example function;
    # the real function depends on the layout of your script
    $query = sprintf( 'SELECT post_id FROM posts WHERE post_slug = \'%s\'', // write the MySQL query
        mysql_real_escape_string( $slug ) // escape the string correctly (make sure that magic_quotes_gpc is turned off)
    );
    $result = mysql_query( $query ); // assuming that we are connected to MySQL
    if ( mysql_num_rows( $result ) ) { // if a post with this slug was found..
        return TRUE; // ..this slug exists
    } else { // if not..
        return FALSE; // ..this slug does not exist
    }
}

Now you can slug whatever you want, without fearing collisions. This means that you can even use the slug column as an ID instead of post_id in this example.

Perl Script to Parse IRC Logs for Use in MegaHAL.trn

MegaHAL is an interesting AI engine that has no knowledge of the words; it only reassembles sentences and abuses the ability of the human mind to read order in chaos. If you are into MegaHAL IRC bots, the following Perl code might be handy:

1
2
3
4
5
6

#!/usr/bin/perl
while ( <> ) {
    s/<.*>//;
    s/^[^ ]*://;
    print unless m/^\*/;
}

This is for mIRC-style logs without timestamps, i.e:

<Tim> This is an example.
* Tim explains

Line 3 removes the nickname from each line. If the first word in a line is followed by a colon, the fourth line removes that word and the colon. This is useful since people often highlight each other in this style, and you do not want those nicknames. Line 5 prints the resulting fixed line, unless it starts with a *, i.e. CTCP ACTIONs (”/me”) and server messages are not printed. To use the perl script just pipe the output to the MegaHAL training file:

$ perl logfix.pl SomeNet-somechan.log > megahal.trn

How to Record and Play NetHack Games with ttyrec and ttyplay Live

ttyrec is a great program for recording a terminal. I use it for recording nethack games. Any command is easily recorded with the “-e” argument of ttyrec, which allows you to choose a single command to record instead of the whole shell session. For example, I could type

$ ttyrec -e nethack

In order to start nethack, recording the whole game. Or rather, if I want to save the recorded game to a file other than the default “ttyrec”:

$ ttyrec game00 -e nethack

However, I don’t want to nethack. My name is Tim, and I want to nethack -u Tim. Simply adding -u Tim to the previous code does not work, since “-u” is interpreted as another switch for ttyrec. Quotation marks solve the problem:

$ ttyrec game00 -e "nethack -u Tim"

You can easily make a bash script out of this. The following is my playnethack, which is run instead of the regular “nethack”.

#!/bin/bash
echo -n "Name?"
read pname
ttyrec ~/nethack/rec/$pname-`date +%Y%m%d%H%M%S`.tty -e "nethack -u $pname"

Note that the filename here is dynamic. The text within backticks (”`”) is executed as a command, and the command is then substituted with its actual output. date +%Y%m%d%H%M%S returns the current date and time in a format such as 20070613233807. Thus, if I ran the “playnethack” command and set my name to “Tim”, my complete game would be recorded into the file “~/nethack/rec/Tim-20070613233807.tty”, probiding the ~/nethack/rec/ directory existed.

The sister command of ttyrec is ttyplay, which plays the recorded .tty files. Simply do:

$ clear
$ ttyplay ~/nethack/rec/Tim-20070613233807.tty

And you will watch my game, from the beginning to the end. The time is recorded, too–you will know exactly for how long I contemplated each and every action. Using the ttytime utility, you can find out how many seconds long a recorded game is before playing it, e.g:

$ ttytime ~/nethack/rec/Tim-20070613233807.tty
  71946 ~/nethack/rec/Tim-20070613233807.tty

However, if you don’t want to wait for 71946 seconds while watching the replay, you can always press any button in order to forward to the next movement. If you hold a key, you will fast-forward through the entire game.

It is even possible to watch a live game as it is being played, using “-p”:

$ ttyplay -p ~/nethack/rec/Tim-20070613233807.tty

This will let you watch my NetHack game as it progresses in real-time, and thus lets you tell me as soon as I make a stupid mistake.

How to Find Deleted Torrents in Deluge

If you keep seeding torrents that have been removed from a tracker, some trackers will harass you with messages like the following, from OiNK:

You have at least one torrent in your client that’s been deleted from our system. Right now it keeps announcing uselessly to the tracker. Please stop or remove all OiNK torrents that don’t exist on our site any more. You will get this message once for every 100 announces you make for nonexistent torrents.

It is simple to identify which torrents are not left on the tracker in Deluge (0.5.1). Just look in the “Status” column. Probably, most torrents will say either “Seeding” or “Leeching”. Look for torrents that stay at “Connecting” for a long time–that’s probably what you’re looking for. Select them and press the “Remove Torrent” icon, making sure not to remove the data.

Problem solved!

How to Install Plugins in Deluge 0.5.1 BitTorrent Client

Getting tired of the simplicity of the original BitTorrent UI and the memory-hogging of Java-based Azureus, I recently turned to Deluge. According to Wiktionary, deluge is (paraphrased):

1. A great flood or rain.
2. An overwhelming amount of something
3. The Deluge: The Biblical flood during the time of Noah.
4. (transitive) To flood with water.
5. (transitive) To overwhelm.

It also happens to be a handy, well-featured and good-looking BitTorrent client for Linux. However, since it is still in beta, there is naturally faulty documentation and some other problems. It worked great from scratch, but then I decided that I wanted to divide my torrents into two categories–checked out and not checked out. The announced Categorize plugin seemed optimal for this purpose, and thus, I proceeded to download it from the Plugins page of the Deluge wiki:

$ cd ~
$ wget http://www.icebeach.de/categorize.tar.gz

I opened the tarball and found three files:

$ tar xzf categorize.tar.gz
$ ls
categorize.jpg
Categorize/cat.glade
Categorize/plugin.py

Unable to find instructions on how to install the plugin, I did an slocate to find the other plugins. That plugin.py file seemed pretty generic, so I searched for others.

$ sudo slocate -u
$ slocate plugin.py
/home/tim/Categorize/plugin.py
/usr/share/deluge/plugins/TorrentCreator/plugin.py
/usr/share/deluge/plugins/NetworkGraph/plugin.py
/usr/share/deluge/plugins/ExamplePlugin/plugin.py
/usr/share/deluge/plugins/NetworkHealth/plugin.py
/usr/share/deluge/plugins/TorrentSearch/plugin.py
/usr/lib/gimp/2.0/python/gimpplugin.py
/usr/lib/gimp/2.0/python/gimpplugin.pyc

Evidently, the files should be placed in /usr/share/deluge/plugins. So I did:

$ cd /usr/share/deluge/plugins
$ sudo tar xzf ~/Desktop/categorize.tar.gz

However, starting Deluge failed. I tried starting it from the command line in order to find the problem, and the verbose output helped a lot.

$ delugeno existing Deluge session
Starting new Deluge session...
deluge_core; using libtorrent 0.13.0.0. Compiled with NDEBUG value: 1
Applying preferences
Pickling state...
Scanning plugin dir /usr/share/deluge/plugins
Loading module HelloWorld
Initialising plugin HelloWorld
Loading module TorrentCreator
Initialising plugin TorrentCreator
Loading module NetworkGraph
Initialising plugin NetworkGraph
Loading module ExamplePlugin
Initialising plugin ExamplePlugin
Loading module NetworkHealth
Initialising plugin NetworkHealth
Loading module TorrentSearch
Initialising plugin TorrentSearch
Traceback (most recent call last):
  File "/usr/bin/deluge", line 83, in <module>
    start_deluge()
  File "/usr/bin/deluge", line 57, in start_deluge
    interface = deluge.interface.DelugeGTK()
  File "/usr/lib/python2.5/site-packages/deluge/interface.py", line 54, in __init__
    self.plugins.scan_for_plugins()
  File "/usr/lib/python2.5/site-packages/deluge/plugins.py", line 44, in scan_for_plugins
    if '__init__.py' in os.listdir(path):
OSError: [Errno 20] Not a directory: '/usr/share/deluge/plugins/categorize.jpg'

Deluge sees the image file categorize.jpg as a plugin folder. Remove it:

$ sudo rm /usr/share/deluge/plugins/categorize.jpg

and it should work. This was done on Ubuntu 7.04 Feisty Fawn with Deluge 0.5.1, installed through apt-get.

Help, I’m trapped in a NetHack release

Something is written here in the dust.
You read: "help i'm trapped in a nethack release".

                               ------
                               |.....#################
                              #-....|                #     -------------
                            ###|...{|      ----------###   |...........|
              ------------  #  |....|    ##...f.....|  #  #............|
              |..........|###  |....|    # |........|  ####|.....^.....|
              |..........|#    ----.-  ### |....@...|   ## |...........|
              |..........|#        #   #   |.........######............|
             #-..........|#        # ###   ----------      |...........|
           ###|..........-#       ## #                     -------------
           #  ------------     ---.--#
         ###                   |....|#
 -----.- #                     |....|#
 |......##                     |....-#
 |.....|                       ------
 |..<..|
 |.....|
 -------

Tim the Evoker            St:14 Dx:11 Co:12 In:16 Wi:12 Ch:12  Chaotic
Dlvl:4  $:320 HP:17(17) Pw:12(12) AC:8  Xp:2/26

Current record: 20516. My Barbarian was frozen by a floating eye. Far, far from winning.

Amarok 256-songs limit for iPod

I decided to give my iPod a try under Linux (Fedora Core 4, x86_64). For a long time now, I’ve been using Amarok for playing music. Conveniently, it already has a function for transfering files to an iPod under the “Media Device” tab. However, there was a problem.

I added one song, after which the iTunes database file (iTunesDB) was corrupted, showing only a few songs. Note that only the database file was corrupted; the files were still there, but they were orphan in that they could not be accessed by the iPod software. Although I was unable to restore the iTunesDB, a lot of testing showed that, no matter how many or which songs I tried to transfer or delete, it was always only the 256 first songs that were visible.

This 256 songs limit was very confusing. After some research, I found that it was caused by the old Amarok version that I was using (Amarok 1.3.9). Upgrading to the latest stable release (Amarok 1.4.5) should fix the problem, although I’m currently too lazy and will have to settle with 256 files.