More Segment HMMs

It’s weird how you can muck about with structures for so long and still forget what the hell you’re doing all the time. This post is supposed to try and clarify segment HMMs a bit for me, so that I can approach the learning problem with a clear mind. It’s all very specific; and it’s all below the fold.

Python and Numpy integers

I keep getting bitten on the bum with Python and ‘proper’ programming things. Today it’s this:

In [253]: 8**14
Out[253]: 4398046511104L

In [254]: 8**(numpy.int32(14))
Out[254]: 0

I would feel guilty about this if I had to specify the type of numbers I was using, but this is a dynamic programming language! How the hell am I supposed to know that raising an int by a numpy.int32 will force the result to also be an numpy.int32 (and therefore (silently) 0 because int32 can’t hold my number) whereas raising an int by an int will result in a long? Why didn’t the int32 become a long also? Or an int64 or something that could hold my number?

Gah!

Nasty Python Things

So I seem to keep writing commands that look like this:

delta[t][q] = max(
    [delta[tau][j] +
        pylab.log(
            pylab.array([
                output_dist(Q=q,L=(t-tau),Y=Y[tau+1:t]),
                duration_dist(Q=q,L=(t-tau)),
                transition_dist[q,j]]).prod())
    for j in self.state_range])

Is this bad? The above is the max of a list. The list is made up using a list comprehension, where each element is the log of a product of a 1D array plus a bit. Each element of each array is a call to a function associated with my model. The trouble is, if I break it down into some for loops, then I start having to invent temporary names for my variables, which seems clunky.

Any opinions?

M

MICROCOSMOGRAPHIA ACADEMICA

I came across the Microcosmographia Academia in one of Niranjan’s slides (which are an interesting view into the world of academic politics and bureaucracy) and then found it in full at the webpage of Dr Utting of Kent University. It’s a lovely satire on academic politics from 1908 in the form of a short pamphlet. Just short enough, in fact, that I’ve wasted a good hour typesetting it [pdf,tex]. It starts with this small ‘advertisement’:

If you are young, do not read this book; it is not fit for you;
If you are old, throw it away; you have nothing to learn from it;
If you are unambitious, light the fire with it; you do not need its guidance.

But, if you are neither less than twenty-five years old, nor more than thirty;
And if you are ambitious withal, and your spirit hankers after academic politics;
Read, and may your soul (if you have a soul) find mercy!

Pebl

Look! A python module for learning bayesian network structures has surfaced on the JMLR software track – bits of the author’s blog are quite interesting too… This is article number 5 for the software track in JMLR, and it’s been going over a year now! I suppose 5 examples is quite enough to figure out what’s required for submission here, though, so maybe uptake will improve…

The Pirate Bay Trial

So Stina wrote a piece for the independent about the Pirate Bay Trials which reminded me: I’m really upset about the implications of this attack one of the main hubs of the new social order! Just in case  you don’t know, The Pirate Bay is a place for downloading torrent files, which are little files that help you download bigger files from people all over the world. Generally, it’s used for downloading things you’ve not bought, and so helps us in the grand copyright infringement which, apart from making us feel kind of smug and priatey, proves the inefficiency in the market for such digital things and ultimately, hopefully, will change the state of play. See Steal This Film for more info.

Here, I just wanted to link to some coverage of the trial, which is illuminating. There’s Spectrical which is the sort of `official’ coverage, then there’s also TorrentFreak’s coverage which has an especially moving piece about one of the wife of one of the witnesses for the defence being sent a load of flowers. Then there’s also the mainstream coverage in the Guardian, in Cory Doctorow’s column Digital rights, digital wrongs. It all goes to show the lengths people go in order to restrict change. I wonder if I’ll be as motivated as the record companies when something I don’t want to change comes along? Hopefully, at least, I’ll be better informed than these dinosaurs…

Latex, Beamer, Python, Beauty

This post quickly shows how to create pretty latex beamer slides, that don’t look anything much like maths seminar slides, and  that can include well formatted (Python) code, as well as the normal stuff that Beamer is so good at (maths, videos, etc). I’m assuming you’ve got a `standard’ install of latex (by standard I mean: like mine) that includes beamer and the listings package. I’m also assuming that you already know how to use beamer, so I’ve not included all the documentclass junk and so on. Here’s a taster of what it looks like, how to do it is after the break:

Forward Equations for the Segment HMM

This is a derivation of the forward equations for the Segment HMM to follow on from my last post. Again I should say that this mostly based on Kevin Murphy’s Segment HMM tutorial which you should probably be reading instead of these posts.

Segment HMMs

I’ve been finding out about segment HMMs. They’re a generalisation of explicit state duration HMMs and a specific case of the hierarchical HMM. I’ll describe them a bit in this post, and then describe the forward algorithm in the next post. Both posts will be culled mostly from Kevin Murphy’s Segment HMM tutorial. All of the errors in this post will be mine, as I’m trying to interpret Kevin’s stuff without using the Dynamic Bayesian Networks, which I’m starting to think is a horrible mistake on my part.

Instead of this post you should probably read a tech report from Cambridge called The Theory of Segmental Hidden Markov Models, which is from the early 90s, and also stuff about hierarchical HMMs, which I think generalise segment HMMs, starting with The Hierarchical Hidden Markov Model: Analysis and Applications by Fine, Singer and Tishby in 1998 and maybe Kevin’s 2001 NIPS paper Linear Time Inference in Hierarchical HMMs.

Python Scope Example

Wow I just learnt a new thing about Python’s scope! Below are two examples.

1:

x = 1
def foo():
    print x
foo()

2:

x = 1
def foo():
    print x
    x = 2
foo()

Code snippet number 1 works. It looks up the variable x in its global namespace and prints it. Easy!

Code snippet number 2 breaks on line 3, because there’s an x defined in the function’s local namespace (even though it comes later) and so it knows that x is local and hence barfs when you try to print it before assigning it a value!