What I did today:
1. Reviewed AISTATS papers (referee reports are due tonight)
2. Debugged code output from yesterday, concluded that issues came from:
-inability to handle superlatives well
-lingering non-convexities in the cost function
-examples where we found an incorrect logical form that nevertheless has a correct denotation end up pushing hard against the gradient
3. To solve the superlative issue, I hard-baked some morphology into the model (changing “largest” to “larg-” and “-est”).
4. Every so many iterations, we drop all the examples that we are having trouble with and recompute the parameters, then add those examples back in and use beam search with the new parameters
5. Alter the cost function to be convex (this also required a bit of math to figure out something suitable).
I also finally got around to updating my academic web page; it’s now at http://cs.stanford.edu/~jsteinhardt/