Lots of logs! I fell behind posting them online but they still existed on paper:
[0:30] 1. Wrote script to extract graphs from the output of my experiments
[???] 2. Thought about the right way to represent a distribution with a -system, when the -system is not a tree; I failed to time this but I probably spent a couple hours on this part. The approach was to try to replace various probability masses with default values when we don’t have time to compute the mass exactly.
04-30-2013:[3:00] 1. Continued to think about non-tree -systems. Came up with a sort of sum-product factorization.
[1:00] 2. Met with Percy.
[1:30] 3. Met with Andreas.
[1:00] 4. Thought about a kernel interpretation of neural networks that Roy showed me.
05-01-2013:[1:30] 1. NLP reading group
[3:00] 2. Worked out -system details (not sure exactly what I meant by this, I think I was still thinking about the sum-product framework).
[1:00] 3. Modified code to average results across multiple trials.
05-02-2013:[1:15] 1. Think more about sum-product framework
[1:30] 2. NLP lunch
[1:00] 3. Meet with Percy. We decided that the sum-product factorization was probably too complicated for the NIPS paper, and shelved it for now. A simpler thing is to just do inclusion-exclusion out to a depth of two, which is guaranteed to give an upper bound on all the probabilities we care about.
[2:00] 4. Vannevar prep.
[2:00] 5. Vannevar meeting
[3:00] 6. Prepare slides for talk at group meeting (I forgot to time this, 3 hours is an estimate).
??? Not sure what I did today, other than speak at group meeting. I was probably fairly sleep-deprived from the day before.
[1:00] Work out how to combine -system inference with dynamic programming.[0:40] Figure out how to present the approach.
[3:30] Write up new version of framework.
[1:30] Jerry Kaplan talk.
[0:30] Worked out how to send backwards messages for -system DP.[0:40] Went to the bank to deposit checks.
[1:50] Thought about how to prove a theoretical characterization for how well -system inference works in HMMs. The criterion I came up with is to attempt to show that the representation gets monotonically better as the size of the -system increases.
05-08-2013:[2:30] Thought about how to ensure that the model gets monotonically better with more elements in the -system. It seems to not be possible to write down a purely deterministic condition.
[1:30] NLP lunch
[1:00] Paper outline
[0:30] CVX reading (CVX = Boyd’s convex optimization lecture notes)
05-09-2013:[3:00] Continued working on HMM condition; there are two settings — edge cutting in Bayes nets, and nested discriminative training.
[1:15] Statistics lunch
[1:30] Meet with Dario
[1:20] USACO flights + forms
[0:30] CVX reading
[1:00] Meet with Percy. We decided that the dynamic programming stuff was also too complicated for NIPS, and just to focus on the vanilla framework.
05-10-2013:??? I’m apparently pretty bad at logging Fridays.
I put a bunch of stuff down for today but failed to remember what stuff I did. In theory, it was 4 hours of writing, a 1-hour meeting with Percy, 1 hour of exercise, and 1.5 hours of drawing figures. I also came up with a proof technique over the weekend that should get the monotonicity result I need.
<several days missing>
Now I am at the write-up stage for the paper:
[0:45] Factorial HMM figure
[0:30] Re-wrote motivating example section
[0:15] Shrink figures to fit more nicely
[0:15] Replace “semilattice” with “-system” in the exposition
[0:15] Shorten pseudocode
[1:30] Re-implement -system code based on KL divergence instead of max-ent
05-18 and 05-19: got experiments working with the factorial HMM, no timings because this was over the weekend.
05-20 and 05-21: no log, spent my time trying to get experiments working for the Czech NER (named entity recognition) corpus, but failed to do so. I spent far too long on this before realizing that it didn’t have the phenomenon we care about.
05-22 through 05-31: USACO camp
06-01 through 06-06: visit Boston
06-07 through 06-08: visit family
And now I’m back!