halcy.de v4 » Version four-point-oh and counting.Two weeks!

So I had the idea to generate fake twitter pages from a users old tweets. I wrote a bunch of Perl scripts to do just that. It worked out well.

The process essentially consists of four steps:

1. Acquire a users tweets.
2. Train a fourth order markov model with those tweets.
3. Generate new tweets by having the markov model spew out new chains.
4. Generate a twitter page from those tweets.

Except for the last step, I’ve written scripts to do this. (Writing one for the last step wouldn’t be hard, but boring, and it’s easy enough to do by hand)

Step 1: Obtaining tweets.

Essentially, it’s a loop that gets the tweets via twitters api, page for page. Simple enough. The only real problem is twitters rate limiting, so grabbing more than one user per hour does not work.

#!/usr/bin/perl

my $max_page = 200;
my $start_page = 0;
my $user = "username";

for( my $page = $start_page; $page < $max_page; $page++ ) {
        my $cmd = "wget http://twitter.com/statuses/user_timeline/$user.json?page=$page -O - >> tweets.json";
        `$cmd`;
}

Step 2 and 3: Training and generating.

This script takes json as output by the script in step 1 as the input, and outputs generated, fake tweets, one per line.

Since I was too lazy to implement a markov chain myself, I used a library off CPAN to do the heavy lifting.

#!/usr/bin/perl

use JSON;
use Encode;
use Algorithm::MarkovChain;

# Parse JSON
my @tweetsJsonA = <>;
my $tweetsJson =decode_utf8( join( "", @tweetsJsonA ) );
$tweetsJson =~ s/\]\[/,/gi;
my $tweets = decode_json( $tweetsJson );

# Train
my $user = Algorithm::MarkovChain::->new();
foreach my $tweet (@{$tweets}) {
    my @symbs = ("START", split( " ", $tweet->{text}), "END" );
    $user->seed(
        symbols => \@symbs,
        longest => 4
    );
}

# Generate 20 tweets
binmode STDOUT, ":utf8";
for( my $i = 0; $i < 20; $i++ ) {
    my @generated = ("START");
    my $l = 1;
    while( $generated[-1] ne "END" ) {
        @generated = $user->spew(
            length => $l,
            complete => \@generated
        );
        $l++;
    }
    @generated = @generated[1..(@generated-2)];
    print join( " ", @generated ) . "\n";
}

Step 4: Generating a fake twitter page.

This consists of two parts, making the tweets into twittery html, and adding what comes before and after the tweets in a twitter HTML page. For the former, I wrote a small script, again, which mostly just concatenates text a lot, I put it here if you want it (Save as “mktwpage.pl”).

The second part, I’ve done by hand, thus far, assisted by my browsers “Save page” feature. Too lazy to automate.

And, there you have it: Autogenerated fake twitter pages. Halfway convincing, too. Go generate your own!

Don’t know if this is worth an entire post, but hey. Since the website pixiv.net, a Japanese art gallery site with many, many talented artists does not offer any RSS feeds, I wrote a small Perl script to scrape one together for the “Bookmarks” page.

Code on github / Screenshot of feed (Does not include thumb images, which the newest version will download for you and put as the description)

Two things, both graph-related:

NEWSDAQLOG

A pretty visualization for a stupid little trading game, plotting the graphs using Ruby and Gruff: NEWSDAQ graphs (Git repository)

Quota

A mashup of google trends and yahoo finance - compare stock values with search volumes: Quota (Git repository). This one uses SVG::TT::Graph for plotting, and is interesting insofar as that I’ve managed to make that module produce really pretty graphs via some stylesheet fumbling.

Also, one can just include those stock/trend graphs in places now by just constructing the right image URL. The parameters are explained on the Quota demo page.

It takes a file of a certain syntax which should be quite figure-outable from an example as an argument, then reads tapes from STDIN and executes on them.

• turing.pl - Really stupid turing machine simulation thing.
• machine_add1 - An example programm. Adds 1 to a binary number given on the tape.
• machine_compare - Compares two binary numbers on tape.

Releasing mootykins3 0.93 beta, with the following improvements:

• Colours in the config file: Syntax a’la %C01,01
• Bold text: %B
• Underlined: %U
• Reset style: %O
• Latex math rendering plugin: !latex. Read README.latex to find out how to actually use it.
• A few fixed bugs.

Get mootykins3 v0.93 beta, or check the mootykins3 page.

From time to time, I want to record the sound some application makes under Linux. Sadly, my sound card does not provide a “Stereo Mix” device which lets you record all the sound something makes, so I made myself this perl script which records ALSA output of an application and prints it to stdout as PCM.

Update: Added a second of sleep before making sound.

#!/usr/bin/perl

use strict;

my @cards = `cat /etc/asound.names |
    grep -C 1 -i loop |
    grep -C 1 -i PCM |
    grep name`;

my $recordcard = @cards[ 0 ];
my ($recordcard) = ($recordcard =~ /'([^']*)'/);
my ($card, $device) = ($recordcard =~ /:([0-9]*),([0-9]*)$/);

if( @ARGV < 1 ) {
    print STDERR 
        "USAGE: ./record_loopback.pl some_command_that_makes_sound\n";
    exit;
}

my $has_aloop = 0;
my @modules = `lsmod`;

foreach( @modules ) {

    if( $_ =~ /snd_aloop/ ) {
        $has_aloop = 1;
        last;
    }
}

if( $has_aloop == 0 ) {
    print STDERR 
        "FATAL: You need to have the module 'snd_aloop' loaded. ";
    print STDERR
        "Please do 'sudo modprobe snd_aloop' to load it.\n";
    exit;
}

print STDERR "Assuming PCM loop driver is card $card, recording device 1.\n";
my $command =   "env ALSA_PCM_CARD=$card env ALSA_PCM_DEVICE=1 " .
        join( " ", @ARGV ) . " 1> /dev/null";

if( fork() ) {
    sleep(1);
    system( $command );
}
else {
    system( "arecord -D hw:$card,1 -f cd" );
}

(Note that you won’t hear stuff while recording.)

“Eine Seminararbeit von Michael Hoffmann, Lorenz Diener, Johannes Jäger und Tobias Poot”

Basically, this was a project 3 friends and me did in our last year in school. We also gave a presentation on it, and created a smallish browser game (Now mostly defunct).

Abstractish thing rudimentarily translated into english:

Cyberspace, virtual worlds, alternate realities, nearly everyone knows these terms or has even used them a few times. But what do they actually mean? In this document, we don’t only want to explain what’s the deal with those words, but also how they affect our “Real world”.

• Project documentation, 63 Pages, 2.4mb PDF. (German)
• Presentation, 63 Slides, 5.7mb PDF. (German, but has lots of pretty pictures)
• The game, it mostly will not work, but you’re free to fumble around a little. (Also in German, but it looks really pretty!)

I’ve had time to clean up my folders a little, and I guess it’s time to publish some code I have not previously mentioned on here.

First up, we have rot13ajax.pl, which is the worst piece of AJAX abuse I could come up with. A little quote from the script:

Dynamically loading javascript which dynamically loads rot13’d javascript, which is decoded and then dynamically loads rot13’d html, decodes it and inserts it into the document FTW.

So there. You can see it in action and view the source here: Rot13AJAX.

Second, the unicodeificator, basically a web version of debians “uniencode” script. Ιt аⅼⅼоwѕ уօυ tо ‟υɳⅰсօԁеⅰfу‟ tеⅹt, ⅼіkе tҺіѕ ѕеɳtеɳсе. Source and usable script for downloading.

Third, an a bit hackish backup script, for backing up web sites. It’s here for sake of completeness, altough I would not really recommend using it much.

Number four, a script for getting a list of all the IP adresses of the proxies on samair.ru, which is useful for creating lists for mass-banning.

Finally, there is the Super Amazing Wikifier, which is a script to convert saved html pages, specifically those of from site called wikichan, to wiki syntax. It’s useful if your wiki site just accidentally got deleted and you are trying to recreate it from google cache and web.archive.org ;). The Super Amazing Wikifiers source is here, note that you might have to install some perl modules from CPAN to make it work.

Thanks to pihl, there is now a cat typefile for mootykins3s give_stuff plugin, which can be used something like this:

(config)

[plugin::give_stuff::cat]
type_file = plugins/cat_types.txt
prefix = a furry, a supercute, a supersoft

(channel)

[18:29] <halcyon> !cat
[18:29] * satfkins hands halcyon a furry, cute California Spangled Cat.
[18:29] <pihl> !cat halcyon
[18:29] <halcyon> Awesome!

Get it here!

mootykins3 - An extensible IRC bot written in Perl

The previous update was completley borked, so here’s a correction with even more changed documentation. Also, mootykins should now work on windows.

mootykins3 - An extensible IRC bot written in Perl

Download mootykins3 v0.92 beta

Mootykins now has a bash plugin, which can get random or specific quotes from bash.org, qdb.us, german-bash.org and GroupHug.us. Also, the documentation has been updated where neccesary.

mootykins3 - An extensible IRC bot written in Perl

Download mootykins3 v0.92 beta

mootykins3 now has a relay plugin, so he can relay messages from one channel to other channels on the same or on different servers. In addition to that, the config system should be more stable now.

mootykins3 - An extensible IRC bot written in Perl

Download mootykins3 v0.91 beta

Bugfix release: More case sensitivity issues. With this, it’s back to “No known bugs.”

mootykins3 - An extensible IRC bot written in Perl

Download mootykins3 v0.902 beta

Bugfix release: Channels were treated as case-sensitive by the plugin system. Now, channels are lowercased by default. Should be fixed.

Much thanks to Lamune and the peoples at #gensokyo, who noticed this.

mootykins3 - An extensible IRC bot written in Perl

Download mootykins3 v0.901 beta

This is the first public release of mootykins3, an extensible IRC bot written in Perl using POE::Component::IRC and licensed under the GNU GPL. It features:

• Flexible configuration options. The bot can be made to do a lot of things by just changing the config a bit.
• Multi-Server multi-channel support.
• Ease of use, the bot makes a lot of good guesses about what you will probably need to do and works accordingly.
• Extensive documentation. Everything is explained, so you’ll have no trouble getting everything to work.
• An easy-to-use plugin interface so you can easily extend the bot. (If you know a bit of perl, that is.)
• Some default-plugins out of the box to get you started:
  • triggers - Can react to several user-definable triggers.
  • infobot - A basic infobot, with fuzzy matching on factoids.
  • give_stuff - A plugin which can give people types of stuff, like !beer.
  • kirby_dance - The most sophisticated kirby dance creator, ever.
  • help - Help system, to give help for all the other plugins.
  • goauld - Speaks some Goa’uld.
  • none - A placeholder plugin that does nothing.

Get more info and download it on my mootykins3 page.