# Log 04-29-2013 to 06-08-2013

Lots of logs! I fell behind posting them online but they still existed on paper:

04-29-2013:
[0:30] 1. Wrote script to extract graphs from the output of my experiments
[???] 2. Thought about the right way to represent a distribution with a $\pi$-system, when the $\pi$-system is not a tree; I failed to time this but I probably spent a couple hours on this part. The approach was to try to replace various probability masses with default values when we don’t have time to compute the mass exactly.

04-30-2013:[3:00] 1. Continued to think about non-tree $\pi$-systems. Came up with a sort of sum-product factorization.
[1:00] 2. Met with Percy.
[1:30] 3. Met with Andreas.
[1:00] 4. Thought about a kernel interpretation of neural networks that Roy showed me.

[3:00] 2. Worked out $\pi$-system details (not sure exactly what I meant by this, I think I was still thinking about the sum-product framework).
[1:00] 3. Modified code to average results across multiple trials.

05-02-2013:[1:15] 1. Think more about sum-product framework
[1:30] 2. NLP lunch
[1:00] 3. Meet with Percy. We decided that the sum-product factorization was probably too complicated for the NIPS paper, and shelved it for now. A simpler thing is to just do inclusion-exclusion out to a depth of two, which is guaranteed to give an upper bound on all the probabilities we care about.
[2:00] 4. Vannevar prep.
[2:00] 5. Vannevar meeting
[3:00] 6. Prepare slides for talk at group meeting (I forgot to time this, 3 hours is an estimate).

05-03-2013:
??? Not sure what I did today, other than speak at group meeting. I was probably fairly sleep-deprived from the day before.

05-06-2013:
[1:00] Work out how to combine $\pi$-system inference with dynamic programming.[0:40] Figure out how to present the approach.
[3:30] Write up new version of framework.
[1:30] Jerry Kaplan talk.

05-07-2013:
[0:30] Worked out how to send backwards messages for $\pi$-system DP.[0:40] Went to the bank to deposit checks.
[1:50] Thought about how to prove a theoretical characterization for how well $\pi$-system inference works in HMMs. The criterion I came up with is to attempt to show that the representation gets monotonically better as the size of the $\pi$-system increases.

05-08-2013:[2:30] Thought about how to ensure that the model gets monotonically better with more elements in the $\pi$-system. It seems to not be possible to write down a purely deterministic condition.
[1:30] NLP lunch
[1:00] Paper outline
[0:30] CVX reading (CVX = Boyd’s convex optimization lecture notes)

05-09-2013:[3:00] Continued working on HMM condition; there are two settings — edge cutting in Bayes nets, and nested discriminative training.
[1:15] Statistics lunch
[1:00] Climbing
[1:30] Meet with Dario
[1:20] USACO flights + forms
[1:00] Meet with Percy. We decided that the dynamic programming stuff was also too complicated for NIPS, and just to focus on the vanilla framework.

05-10-2013:??? I’m apparently pretty bad at logging Fridays.

05-13-2013:
I put a bunch of stuff down for today but failed to remember what stuff I did. In theory, it was 4 hours of writing, a 1-hour meeting with Percy, 1 hour of exercise, and 1.5 hours of drawing figures. I also came up with a proof technique over the weekend that should get the monotonicity result I need.

<several days missing>

05-17-2013:
Now I am at the write-up stage for the paper:
[0:45] Factorial HMM figure
[0:30] Re-wrote motivating example section
[0:15] Shrink figures to fit more nicely
[0:15] Replace “semilattice” with “$\pi$-system” in the exposition
[0:15] Shorten pseudocode
[1:30] Re-implement $\pi$-system code based on KL divergence instead of max-ent

05-18 and 05-19: got experiments working with the factorial HMM, no timings because this was over the weekend.

05-20 and 05-21: no log, spent my time trying to get experiments working for the Czech NER (named  entity recognition) corpus, but failed to do so. I spent far too long on this before realizing that it didn’t have the phenomenon we care about.

05-22 through 05-31: USACO camp

06-01 through 06-06: visit Boston

06-07 through 06-08: visit family

And now I’m back!