Log 03-29-2013

What I did today:

1. Finished semantic parsing write-up

2. Split Pi-system write-up into two parts

3. Thought about how to learn trees that do a good job of approximating marginal distributions; only limited progress so far, it seems like the actual cost function we care about is infeasible to optimize but there may be a good surrogate.

4. Met with Andreas

Log 03-28-2013

What I did today:

1. Finished Pi-system write-up

2. Wrote most of the semantic parsing write-up (still need to include pseudocode for a few more functions). This took much longer than I expected (I budgeted 90 minutes but it’s taken 4-5 hours so far and will probably take another hour or so today to finish).

3. Met with someone over lunch to who was interested in getting involved with Vannevar

Log 03-27-2013

What I did today:

1. Worked out details of to-down message passing

2. Worked out span-independent abstract logical forms

3. Wrote relatively detailed pseudocode for semantic parsing with abstract logical forms (there are still a few details to fill in but it’s mostly there)

4. Wrote outline for the Pi-system and top-down message passing writeup (this was originally going to be two writeups but I decided it made more sense to merge into one; I will possibly regret this later)

5. Started to fill in the outline, got through 3 sections and halfway through a fourth, but hit a snag on the 4th (weighted relative entropy, which I thought would work out the same as weighted KL, but appears not to, and also is possibly non-convex, which would be annoying). I’ll have to think about this tonight / tomorrow.

6. Met with Dario for an hour about Vannevar stuff

Plan for tomorrow is to finish filling in the outline, write up some stuff about semantic parsing (with pseudocode), and send it to Percy; then I’ll start reading some of the papers that are likely to be relevant to the two NIPS projects.

Log 03-26-2013

The main goal of today was to get a write-up of the approach we’re going to take for the project with Noah. This failed, due to various issues with my approach.

I started by trying to work out my idea more concretely on the “asia” example, which seemed to work reasonably. Then I worked out the criterion for when to further expand a node. I then took a break to eat lunch / meet with Dario about Vannevar. When I came back, I started to write up the approach, but after about 90 minutes of writing (plus a 1-hour nap, I think) I realized that we would run into issues because trying to well-approximate the posterior marginal yields a very flat cost landscape, and so greedily expanding nodes works poorly.

So, at this point, I stopped trying to work on the write-up and tried to find a way to fix this. The idea I came up with was to, instead of optimizing the posterior marginal for one set of evidence in an online fashion, optimize the posterior marginal given several different sets of evidence in an offline fashion; since we’re doing it offline, we can afford to do non-myopic reasoning, and hence don’t have as many issues with the flatness of the cost landscape. However, there’s still a problem that any fixed offline structure will probably not respond well to new data; it seemed that boosting would probably help with this, and googling quickly led me to this very nice paper by Vincent Tan, whose PhD thesis also looks really cool and is on similar topics.

After talking with Andreas, it seems that the upfront offline learning stage is not viable in the probabilistic programming setting, so we spent a while trying to get rid of it. However, I’m becoming increasingly convinced that something like it is necessary, although I have not yet been able to give any formal reasons why. We agreed that Andreas would write some code to do some preliminary exploration on the data set we created.

In the meantime, I started reading about sequential Monte Carlo and particle filters, which are relevant to my project with Percy.

Log 03-25-2013

What I did today (plus some of yesterday):

1. Tried to get SMILE to compute joint posterior probabilities instead of posterior marginals, without success.

2. Found datasets that SMILE solvers had some trouble on (~30 seconds per test case).

3. Turned on some optimizations on SMILE that got the time per test case to about 2 seconds, which is smaller than I had hoped, but still leaves room for improvement.

4. Got test datasets in good working order, plus wrote an API and testing script so that implementing and comparing different algorithms should be easy

5. Cleaned out my inbox

6. Wrote questions for the SPARC app / dealt with other SPARC administration

7. Wrote up interview notes for Vannevar

8. Worked out details of the relevance-directed inference algorithm in Bayes nets; the plan is to write it up tomorrow and send to Andreas / Noah.

Log 03-23-2013

I basically spent most of today trying to find good software and datasets for Bayes nets. (Other highlights include the Hertz lunch and running 7.5 miles with plambda group.)

There’s a lot of good datasets at http://www.bnlearn.com/bnrepository/, but they are in weird formats, and I don’t want to use bnlearn because I don’t want to have to use R.

I read through some papers to try to find possible alternate datasets, but the ones on the bnlearn website seem to be by far the most nice and standardized. So I tried to find software that could parse these (and hopefully have nice Bayes net inference software, as well). I took a look at JavaBayes (not updated since 2002!) and Samiam (reasonable GUI but not great in terms of command-line / API support) before finally setting on SMILE (http://genie.sis.pitt.edu/), which I am super-happy with (I had a few upfront compilation difficulties, but they are mainly due to me not knowing how to use g++).

SMILE is a C++ library for doing Bayes net inference that can handle a pretty extensive array of file formats, including the Hugins .net format that the bnlearn datasets are saved in. I installed SMILE, downloaded the bnlearn datasets, and wrote a simple program that uses SMILE to do inference on one of the smaller bnlearn datasets. I’m looking forward tomorrow to trying to stress SMILE out a bit more and see exactly what it can do.

Here’s some sample SMILE code in case you’re wondering whether to use it yourself:

#include <stdio.h>
#include “smile/smile.h”

double getProbability(DSL_nodeValue* value, int stateIndex){
DSL_sysCoordinates coord(*value);
coord[0] = stateIndex;
return coord.UncheckedValue();

double getProbability(DSL_node* node, const char* stateName){
DSL_nodeValue* value = node->Value();
int stateIndex = node->Definition()->GetOutcomesNames()->FindPosition(stateName);
return getProbability(value, stateIndex);

int main(){
DSL_network net;
net.ReadFile(“asia.net”, DSL_HUGIN_FORMAT);

int asia = net.FindNode(“asia”);
int tub = net.FindNode(“tub”);
int lung = net.FindNode(“lung”);
int xray = net.FindNode(“xray”);

printf(“Computing initial beliefs…\n”);
printf(“P(lung = yes) = %f\n”, getProbability(net.GetNode(lung), “yes”));
printf(“P(tub = yes) = %f\n”, getProbability(net.GetNode(tub), “yes”));
printf(“P(asia = yes) = %f\n”, getProbability(net.GetNode(asia), “yes”));

printf(“Adding and computing probability of evidence (xray = yes)…\n”);
int yes = 0, no = 1;
double p_evidence;
printf(“P(xray = yes) = %f\n”, p_evidence);

printf(“Updating beliefs given xray = yes…\n”);
printf(“P(lung = yes | xray = yes) = %f\n”, getProbability(net.GetNode(lung), “yes”));
printf(“P(tub = yes | xray = yes) = %f\n”, getProbability(net.GetNode(tub), “yes”));
printf(“P(asia = yes | xray = yes) = %f\n”, getProbability(net.GetNode(asia), “yes”));

return 0;

Log 03-22-2013

What I did today:

1. Installed Linux on my new machine
2. Percy’s group meeting (Arun and Andreas gave a nice tutorial on SMT solvers)

3. Meet with Percy to discuss possible applications of the theory I’ve been developing over the past 3 months, as well as write-ups and pseudocode I should produce over Spring Break. We also want to write a NIPS paper on probabilistic abstractions for high-dimensional inference (with particle filters and string mapping as two target applications).

4. Met with Andreas to discuss a plan for our NIPS paper on relevance-directed inference.

5. Hertz board dinner. I talked to a lot of interesting people, and Ed Boyden gave a really interesting talk on the work in his lab (which is most famous for optogenetics but also has a lot of other really interesting neural measurement tools).