Update README.md

Updated as suggested in #117

.gitignore

@@ -36,3 +36,11 @@ java.hprof.txt
 /x
 scr
 rdf
+
+# interactive outputs
+int-output*
+int-backup
+interactive/.ipynb_checkpoints
+# Community server logs
+community-server/data
+community-server/data-backup

DOCUMENTATION.md

@@ -146,7 +146,7 @@ The possible `Value`s, along with an example are as follows:
 
     In general:
 
-        (table (<header-string-1> ... <header-string-n>) (<row-1-value-1> ... <value-1-value-n>) ...)
+        (table (<header-string-1> ... <header-string-n>) (<row-1-value-1> ... <row-1-value-n>) ...)
 
 There are some more arcane `Value`s (see `Values.java` for a list), but they
 are not that important from the point of view of specifying a logical form.
@@ -945,7 +945,7 @@ which are used in our first sematic parsing applications, but they probably
 should be avoided unless you're specifically doing Freebase QA.  Even in that
 case, the main thing you should think about is:
 
-    (rule $Entity ($PHRASE) (LexiconFn fbsearch)
+    (rule $Entity ($PHRASE) (LexiconFn fbsearch))
 
 which uses the Freebase Search API to look up entities.  Be aware here that the
 API will generously return many candidate entities for any string you give it,

README.md

@@ -1,4 +1,4 @@
-# SEMPRE 2.1: Semantic Parsing with Execution
+# SEMPRE 2.4: Semantic Parsing with Execution
 
 ## What is semantic parsing?
 
@@ -62,6 +62,8 @@ SEMPRE has been used in the following papers:
   offshoot, and does not use many of the core learning and parsing utiltiies in
   SEMPRE.  To reproduce those results, check out SEMPRE 1.0.
 
+Please refer to the [project page](https://nlp.stanford.edu/software/sempre/) for a more complete list.
+
 ## Where do I go next?
 
 - If you're new to semantic parsing, you can learn more from the [background
@@ -82,6 +84,8 @@ You must have the following already installed on your system.
 - Ant 1.8.2
 - Ruby 1.8.7 or 1.9
 - wget
+- make (for compiling fig and Virtuoso)
+- zip (for unzip downloaded dependencies)
 
 Other dependencies will be downloaded as you need them.  SEMPRE has been tested
 on Ubuntu Linux 12.04 and MacOS X.  Your mileage will vary depending on how
@@ -95,18 +99,21 @@ similar your system is.
 
 2. Download the minimal core dependencies (all dependencies will be placed in `lib`):
 
-        ./pull-dependencies core
+        ruby ./pull-dependencies core
 
 3. Compile the source code (this produces `libsempre/sempre-core.jar`):
 
         ant core
+        
+   If there is any issue during compilation, try deleting the directories `lib` and `fig`
+   and rerunning `./pull-dependencies core`
 
 4. Run an interactive shell:
 
-        ./run @mode=simple
+        ruby ./run @mode=simple
 
     You should be able to type the following into the shell and get the answer `(number 7)`:
-   
+
         (execute (call + (number 3) (number 4)))
 
 To go further, check out the [tutorial](TUTORIAL.md) and then the [full
@@ -118,24 +125,29 @@ If you will be using natural language to query databases (e.g., Freebase), then
 you will also need to setup your own Virtuoso database (unless someone already
 has done this for you):
 
-    # For Ubuntu, make sure these dependencies are installed
+For Ubuntu, follow this:
+
     sudo apt-get install -y automake gawk gperf libtool bison flex libssl-dev
 
     # Clone the repository
-    git clone https://github.com/openlink/virtuoso-opensource
+    ./pull-dependencies virtuoso
+
+    # Make and install
     cd virtuoso-opensource
-    git checkout tags/v7.0.0
-
-    # Configure
     ./autogen.sh
-    mv INSTALL INSTALL.txt  # Avoid conflict on case-insensitive file systems
     ./configure --prefix=$PWD/install
-
-    # Make (this takes a while)
     make
     make install
     cd ..
 
+on OS/X you can install virtuoso using homebrew by following the instructions
+[here](http://carsten.io/virtuoso-os-on-mac-os/)
+
+To have SEMPRE interact with Virtuoso, the required modules need to be compiled as follow:
+
+    ./pull-dependencies core corenlp freebase
+    ant freebase
+
 # Contribute
 
 To contribute code or resource to SEMPRE:
@@ -161,3 +173,19 @@ Changes from SEMPRE 2.0 to SEMPRE 2.1:
 
 - Added the `tables` package for the paper *Compositional semantic parsing on semi-structured tables* (ACL 2015).
 - Add and `overnight` package for the paper *Building a semantic parser overnight* (ACL 2015).
+
+Changes from SEMPRE 2.1 to SEMPRE 2.2:
+
+- Added code for the paper *Inferring Logical Forms From Denotations* (ACL 2016).
+
+Changes from SEMPRE 2.2 to SEMPRE 2.3:
+
+- Added the `interactive` package for the paper *Naturalizing a programming language through interaction* (ACL 2017).
+
+Changes from SEMPRE 2.3 to SEMPRE 2.3.1:
+
+- Modified the `tables` module to resemble SEMPRE 2.1, effectively making it work again.
+
+Changes from SEMPRE 2.3.1 to SEMPRE 2.4:
+
+- Added the `cprune` package for the paper *Macro Grammars and Holistic Triggering for Efficient Semantic Parsing* (EMNLP 2017).

TUTORIAL.md

@@ -17,7 +17,8 @@ Recall that in semantic parsing, *natural language utterances* are mapped into
 *logical forms* (think programs), which are executed to produce some
 *denotation* (think return value).
 
-We have assumed you have already downloaded SEMPRE and can open up a shell:
+We have assumed you have already [installed](README.md#installation)
+SEMPRE and can open up a shell:
 
     ./run @mode=simple
 
@@ -32,7 +33,7 @@ shell command and executes it.  To see which command is run, do:
 
 This should print out:
 
-    rlwrap java -cp libsempre/*:lib/* -ea edu.stanford.nlp.sempre.Main -interactive
+    java -cp libsempre/*:lib/* -ea edu.stanford.nlp.sempre.Main -Main.interactive
 
 You can pass in additional options:
 
@@ -552,7 +553,7 @@ where $s$ and $o$ are entities and $p$ is a property.  For example:
 is one triple.  If we think of the entities as nodes in a directed graph, the
 each triple is a directed edge between two nodes labeled with the property.
 
-See `data/tutorial.ttl` for an example of a tiny subset of the Freebase graph
+See `freebase/data/tutorial.ttl` for an example of a tiny subset of the Freebase graph
 pertaining to geography about California.
 
 First, pull the dependencies needed for Freebase:
@@ -586,7 +587,7 @@ To stop the server:
 ### Setting up a copy of Freebase
 
 The best case is someone already installed Freebase for you and handed you a
-host:port.  Otherwise, to run your own copy of the Freebase graph (a
+host:port.  Otherwise, to run your own copy of the entire Freebase graph (a
 2013 snapshot), read on.
 
 Download it (this is really big and takes a LONG time):
@@ -611,12 +612,9 @@ We assume you have started the Virtuoso database:
 
 Then start up a prompt:
 
-    ./run @mode=simple-freebase @sparqlserver=localhost:3001
-
-The simplest logical formula in lambda DCS is a single entity:
-
-    fb:en.california
+    ./run @mode=simple-freebase-nocache @sparqlserver=localhost:3001
 
+The simplest logical formula in lambda DCS is a single entity such as `fb:en.california`.
 To execute this query, simply type the following into the interactive prompt:
 
     (execute fb:en.california)
@@ -753,7 +751,7 @@ is the basis for the overloading.
 
 Now start the interactive prompt:
 
-    ./run @mode=simple-freebase @sparqlserver=localhost:3001 -Grammar.inPaths freebase/data/tutorial-freebase.grammar -SimpleLexicon.inPaths freebase/data/tutorial-freebase.lexicon
+    ./run @mode=simple-freebase-nocache @sparqlserver=localhost:3001 -Grammar.inPaths freebase/data/tutorial-freebase.grammar -SimpleLexicon.inPaths freebase/data/tutorial-freebase.lexicon
 
 We should be able to parse the following utterances:
 

build.xml

@@ -25,7 +25,7 @@
     <antcall target="compile.released"/>
   </target>
 
-  <target name="compile.released" depends="init,core,cache,corenlp,freebase,tables,overnight"/>
+  <target name="compile.released" depends="init,core,cache,corenlp,freebase,tables,cprune,overnight"/>
 
   <!-- Compile core -->
   <target name="core" depends="init">
@@ -69,7 +69,7 @@
   </target>
 
   <!-- Compile tables -->
-  <target name="tables" depends="init,core,corenlp">
+  <target name="tables" depends="init,core,corenlp,cprune">
     <echo message="Compiling ${ant.project.name}: tables"/>
     <mkdir dir="${classes}/tables"/>
     <javac srcdir="${src}" destdir="${classes}/tables" classpathref="lib.path" debug="true" includeantruntime="false" source="${source}" target="${target}">
@@ -78,6 +78,16 @@
     <jar destfile="${libsempre}/sempre-tables.jar" basedir="${classes}/tables"/>
   </target>
 
+  <!-- Compile cprune -->
+  <target name="cprune" depends="init,core">
+    <echo message="Compiling ${ant.project.name}: cprune"/>
+    <mkdir dir="${classes}/cprune"/>
+    <javac srcdir="${src}" destdir="${classes}/cprune" classpathref="lib.path" debug="true" includeantruntime="false" source="${source}" target="${target}">
+      <include name="edu/stanford/nlp/sempre/cprune/"/>
+    </javac>
+    <jar destfile="${libsempre}/sempre-cprune.jar" basedir="${classes}/cprune"/>
+  </target>
+
   <!-- Compile overnight -->
   <target name="overnight" depends="init,core">
     <echo message="Compiling ${ant.project.name}: overnight"/>
@@ -88,6 +98,25 @@
     <jar destfile="${libsempre}/sempre-overnight.jar" basedir="${classes}/overnight"/>
   </target>
 
+  <!-- Compile interactive -->
+  <target name="interactive" depends="init,core">
+    <echo message="Compiling ${ant.project.name}: interactive"/>
+    <mkdir dir="${classes}/interactive"/>
+    <javac srcdir="${src}" destdir="${classes}/interactive" classpathref="lib.path" debug="true" includeantruntime="false" source="${source}" target="${target}">
+      <include name="edu/stanford/nlp/sempre/interactive/"/>
+    </javac>
+    <jar destfile="${libsempre}/sempre-interactive.jar" basedir="${classes}/interactive"/>
+  </target>
+
+  <!-- Compile geo880 -->
+  <target name="geo880" depends="init,core,corenlp,tables">
+    <echo message="Compiling ${ant.project.name}: geo880"/>
+    <mkdir dir="${classes}/geo880"/>
+    <javac srcdir="${src}" destdir="${classes}/geo880" classpathref="lib.path" debug="true" includeantruntime="false" source="${source}" target="${target}">
+      <include name="edu/stanford/nlp/sempre/geo880/"/>
+    </javac>
+    <jar destfile="${libsempre}/sempre-geo880.jar" basedir="${classes}/geo880"/>
+  </target>
 
   <!-- Clean up -->
   <target name="clean">

freebase/scripts/extract-freebase-schema.rb

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 require 'open-uri'
 

freebase/scripts/fbshell.rb

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 require 'open-uri'
 require 'json'

freebase/scripts/virtuoso

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 # This script provides a convenient wrapper for the Virtuoso SPARQL server.
 
@@ -151,7 +151,7 @@ def add
     # Virtuoso can't handle prefixes in the second argument, so need to expand.
     tmp = "virtuoso-#{port}-#{i}.ttl"
     log "==== Step A: Converting #{ttl} to #{tmp}..."
-    run "time #{chunkCmd} -indices #{i} | sed -r -e 's/\\tfb:([^\\t]*).$/\\t\\<http:\\/\\/rdf.freebase.com\\/ns\\/\\1\\>./' > #{tmp}"
+    run "time #{chunkCmd} -indices #{i} | sed -r -e 's/\\tfb:([^\\t ]*)[\\t ]*\\.$/\\t\\<http:\\/\\/rdf.freebase.com\\/ns\\/\\1\\>./' > #{tmp}"
 
     # Index the ttl file
     log "==== Step B: Indexing #{tmp}..."

interactive/README.md

@@ -0,0 +1,115 @@
+# README
+
+This `interactive` package is the code for our paper
+*Naturalizing a programming language through interaction* (ACL 2017).
+A live demo is at [www.voxelurn.com](http://www.voxelurn.com).
+
+voxelurn is a language interface to a voxel world.
+This server handles commands used to learn from definitions, and other interactive queries.
+In this setting, the system begin with the dependency-based action language (`dal.grammar`), and gradually expand the language through interacting with it users.
+
+## Overview of the components
+
+### sempre.interactive
+
+The `edu.stanford.nlp.sempre.interactive` package live in this repo contains code for
+* running interactive commands (such as query, accept, reject, definition)
+* executor for the dependency-based action (DAL) language
+* voxelurn specific code in `edu.stanford.nlp.sempre.interactive.voxelurn` for actually   generating the voxel and manipulating them
+Utilties and resources such as the grammar and run script are in this directory, and the code in in the regular `sempre/src` directory.
+
+### voxelurn client
+
+It queries the server, and renders the voxels to a browser. A live version is at [www.voxelurn.com](http://www.voxelurn.com), which queries our server. You can also find a client for localhost at [http://local.voxelurn.com](http://local.voxelurn.com), which is the same client, but with queries going to `http://localhost:8410` instead.
+Code for the client at `https://github.com/sidaw/shrdlurn`. See its [README.md](https://github.com/sidaw/shrdlurn/blob/master/README.md) if you want to work with and build the client yourself.
+
+
+### voxelurn community server
+Located at `interactive/community-server`, the community server
+handles other functionalities such as logging client actions, leaderboard, submiting structures, authentication etc. and generally functions not related to parsing. This server is needed for running interactive experiments, but is not required just for trying out voxelurn.
+
+
+## Running the SEMPRE server for Voxelurn
+
+0. Setup SEMPRE dependencies and compile
+
+         ./pull-dependencies core
+         ant interactive
+
+1. Start the server
+
+        ./interactive/run @mode=voxelurn -server -interactive
+
+  things in the core language such as `add red left`, `repeat 3 [select left]` should work.
+
+2. Feed the server existing definitions, which should take less than 2 minutes.
+
+        ./interactive/run @mode=simulator @server=local @sandbox=none @task=freebuilddef -maxQueries 2496
+
+  try `add dancer`  now.
+
+### Interacting with the server
+
+After you run the above, there are 3 ways to interact and try your own commands.
+
+* The visual way is to use the client: [http://local.voxelurn.com](http://local.voxelurn.com).
+  Code for the client is at `https://github.com/sidaw/shrdlurn` (see its [README.md](https://github.com/sidaw/shrdlurn/blob/master/README.md)).
+  Try `[add dancer; front 5] 3 times`.
+
+* Hit `Ctrl-D` on the terminal running the server, and type `add red top`, or `add green monster`
+
+* On a browser, type `http://localhost:8410/sempre?q=(:q add green monster)`
+
+
+## Experiments in ACL2017
+
+1. Start the server
+
+       ./interactive/run @mode=voxelurn -server -interactive
+
+2. Feed the server all the query logs
+
+       ./interactive/run @mode=simulator @server=local @sandbox=none @task=freebuild -maxQueries 103874
+
+  This currently takes just under 30 minutes. Decrease maxQuery for a quicker experiment. This generate `plotInfo.json` in `./state/execs/${lastExec}.exec/` where `lastExec` is `cat ./state/lastExec`.
+
+3. Taking `../state/execs/${lastExec}.exec/plotInfo.json` as input, we can analyze the data and produce some plots using the following ipython notebook
+
+       ipython notebook interactive/analyze_data.ipynb
+
+  which prints out basic statistics and generates the plots used in our paper. The plots are saved at `../state/execs/${lastExec}.exec/`
+
+
+## Misc.
+
+There are some unit tests
+
+    ./interactive/run @mode=test
+
+To specify a specific test class and verbosity
+
+    ./interactive/run @mode=test @class=DALExecutorTest -verbose 5
+
+Clean up or backup data
+
+    ./interactive/run @mode=backup # save previous data logs
+    ./interactive/run @mode=trash # deletes previous data logs
+
+Data, in .gz can be found in queries.
+
+* `./interactive/queries/freebuild.def.json.gz`
+has 2495 definitions combining just over 10k utterances.
+* `./interactive/queries/freebuild.json.gz` has 103873 queries made during the main experiment.
+
+## Voxelurn community server (optional and in development)
+
+This server helps with client side logging, leaderboard, authentication etc. basically anything that is not directly related to parsing.
+This component is only required if you want to run the interactive experiment yourself. It is fairly coupled with the [voxelurn  client](http://github.com/sidaw/shrdlurn), which sends the request to this server.
+
+    cd interactive/community-server
+    python install-deps.py
+    python server.py --port 8403
+
+    # required keys for authentication
+    export SEMPRE_JWT_SECRET=ANY_RANDOM_SEQEUNCE
+    export SLACK_OAUTH_SECRET=OAUTH_KEY_FROM_SLACK

interactive/analyze_data.ipynb

@@ -0,0 +1,424 @@
+{
+ "cells": [
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "A line looks like this\n",
+    "  \"time\": \"2017-01-21T05:31:57.474\",\n",
+    "  \"id\": \"AMT_A1HKYY6XI2OHO1\",\n",
+    "  \"log\": \"(:q \\\"repeat 10 [ Ebony Wall; select front]\\\")\",\n",
+    "  \"stats.type\": \"q\",\n",
+    "  \"stats.size\": 1,\n",
+    "  \"stats.status\": \"Induced\",\n",
+    "  \"queryCount\": 2707"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import csv\n",
+    "import numpy as np\n",
+    "import pandas as pd  \n",
+    "import matplotlib\n",
+    "import matplotlib.pyplot as plt\n",
+    "import json\n",
+    "import os\n",
+    "import subprocess\n",
+    "from collections import OrderedDict\n",
+    "%matplotlib inline  \n",
+    "\n",
+    "with open('../state/lastExec', 'rb') as lastExec:\n",
+    "    lastExecInd = lastExec.readline().strip()\n",
+    "print lastExecInd\n",
+    "     \n",
+    "rows = []; \n",
+    "execInd = lastExecInd;\n",
+    "execPath = '../state/execs/%s.exec/' % execInd\n",
+    "#print 'analyzing: ' + execPath\n",
+    "def printOptions():\n",
+    "    with open(os.path.join(execPath,'options.map')) as optionsfile:\n",
+    "        opts = filter(lambda l: 'file' in l or 'logFiles' in l  or 'reqParams' in l, optionsfile.readlines());\n",
+    "        for opt in opts: print opt.strip()\n",
+    "    # egrep 'file|Simulator'\n",
+    "printOptions()\n",
+    "\n",
+    "with open('../state/execs/%s.exec/plotInfo.json' % execInd, 'rb') as jsonfile:\n",
+    "    json_lines = jsonfile.readlines()\n",
+    "\n",
+    "rawrows = [json.loads(l) for l in json_lines]\n",
+    "rows = [r for r in rawrows if r.has_key('stats.type')]\n",
+    "\n",
+    "print '%d queries in plotInfo' % len(json_lines)\n",
+    "\n",
+    "def print_defstats():\n",
+    "    filtered_rows = [r for r in rows if r['stats.type'] == 'def' and r['stats.num_rules'] >= 1]\n",
+    "    total_failed = np.sum([r['stats.num_failed'] for r in filtered_rows])\n",
+    "    total_body = np.sum([r['stats.num_body'] for r in filtered_rows])\n",
+    "\n",
+    "    stats =  {\\\n",
+    "           'total_def_queries': len(filtered_rows), \\\n",
+    "           'total_rules': np.sum([r['stats.num_rules'] for r in filtered_rows]), \\\n",
+    "           'total_failed': total_failed, \\\n",
+    "           'total_body': total_body\n",
+    "           }\n",
+    "    print ''\n",
+    "    print stats\n",
+    "    print 'failpercent: %.4f' % (float(total_failed)/float(total_body))\n",
+    "print_defstats()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "scrolled": false
+   },
+   "outputs": [],
+   "source": [
+    "\n",
+    "\n",
+    "def print_stats():\n",
+    "    filtered_rows = [r for r in rows if r['stats.type'] == 'accept']\n",
+    "    induced_rows = [r for r in rows if r['stats.type'] == 'accept' and r['stats.status']=='Induced']\n",
+    "    core_rows = [r for r in rows if r['stats.type'] == 'accept' and r['stats.status']=='Core']\n",
+    "    none_rows = [r for r in rows if r['stats.type'] == 'accept' and r['stats.status']=='Nothing']\n",
+    "\n",
+    "    stats =  {\\\n",
+    "           'accepted': len(filtered_rows), \\\n",
+    "           'induced': len(induced_rows), \\\n",
+    "           'inducedp':len(induced_rows)/float(len(filtered_rows)),\\\n",
+    "           'core': len(core_rows), \\\n",
+    "           'corep':len(core_rows)/float(len(filtered_rows)),\\\n",
+    "           'none': len(none_rows), \\\n",
+    "           'nonep':len(none_rows)/float(len(filtered_rows))\\\n",
+    "           }\n",
+    "    print '\\nCount of accepted query / percentage'\n",
+    "    print ' total:{accepted}\\n induced:{induced}({inducedp:.4f})\\n core:{core}({corep:.4f})\\n none:{none}({nonep:.4f}),'.format(**stats)\n",
+    "    #print 'check %f' % (stats['inducedp']+stats['corep']+stats['nonep'])\n",
+    "    \n",
+    "    statscorrect = {\n",
+    "        'accepted0': np.mean([1 if r['stats.rank']==0 and r['stats.status']!='Nothing' else 0 for r in filtered_rows]),\\\n",
+    "        'accepted1': np.mean([1 if r['stats.rank']>=0 and r['stats.status']!='Nothing'  else 0 for r in filtered_rows]),\\\n",
+    "        'induced0': np.mean([1 if r['stats.rank']==0 and r['stats.status']=='Induced' else 0 for r in filtered_rows]),\\\n",
+    "        'induced1': np.mean([1 if r['stats.rank']>=0 and r['stats.status']=='Induced' else 0 for r in filtered_rows]),\\\n",
+    "        'core0': np.mean([1 if r['stats.rank']==0 and r['stats.status']=='Core' else 0 for r in filtered_rows]),\\\n",
+    "        'core1': np.mean([1 if r['stats.rank']>=0 and r['stats.status']=='Core' else 0 for r in filtered_rows])\\\n",
+    "    }\n",
+    "    print '\\nranked 1 accepted / found accepted'\n",
+    "    print ' total:{accepted0:.4f}/{accepted1:.4f}\\n induced:{induced0:.4f}/{induced1:.4f}\\n core:{core0:.4f}/{core1:.4f}'.format(**statscorrect)\n",
+    "\n",
+    "    token_types = set();\n",
+    "    rows_types = [r for r in rows if r['stats.type'] == 'q'];\n",
+    "    for r in rows_types:\n",
+    "        token_types |= set(r['q'].split(' '))\n",
+    "        \n",
+    "    print '\\nThere are %d token types of %d query types' % (len(token_types), len(rows_types))\n",
+    "print_stats()\n",
+    "\n",
+    "def percent_core():\n",
+    "    allq = [r for r in rows if r['stats.type'] == 'accept'];\n",
+    "    is_status = [100 if r['stats.status'] == 'Induced' else 0 for r in allq]\n",
+    "    print 'percent_induced %f' % np.mean(is_status)\n",
+    "    print 'percent_induced_last10k %f' % np.mean(is_status[len(is_status)-10000:])\n",
+    "percent_core()\n",
+    "\n",
+    "# print filtered_rows[0]\n",
+    "\n",
+    "\n",
+    "\n",
+    "def percent_error():\n",
+    "    allerror = [r for r in rawrows if r.has_key('stats.error')];\n",
+    "    alluerror = [r for r in rawrows if r.has_key('stats.uncaught_error')];\n",
+    "    print 'errors %d (%d uncaught)' % (len(allerror), len(alluerror))\n",
+    "percent_error()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "\n",
+    "def plot_reset():\n",
+    "    global p\n",
+    "    p = {'color': 'r', 'linewidth': 2, 'alpha':0.5}\n",
+    "                       #, 'marker':'*', 'markersize':0.3}\n",
+    "plot_reset()\n",
+    "def savefig(filename = 'fig.pdf'):\n",
+    "    plt.savefig(os.path.join(execPath,filename) ,  bbox_inches=\"tight\")\n",
+    "\n",
+    "def plot_cumavg(x, y, xlabel='query#', ylabel='recall', title=None):\n",
+    "    y_cum = np.cumsum(y).tolist()\n",
+    "    #print accepts_np[:,1]\n",
+    "    y_cumavg = [cum / float(count+1) for count,cum in enumerate(y_cum)]\n",
+    "    #N = 500;\n",
+    "    #y_cumavg = np.convolve(np.array(y), np.ones((N,))/N, mode='same').tolist()\n",
+    "    #print accept_rate \n",
+    "    #plt.scatter(means_baseline[0:], means[0:], s=colors, alpha=0.8, c='r')\n",
+    "    plt.plot(x, y_cumavg, **p)\n",
+    "    \n",
+    "    plt.xlabel(xlabel, fontsize=12)\n",
+    "    plt.ylabel(ylabel, fontsize=12)\n",
+    "    # plt.xlim(0, 0.65)\n",
+    "    plt.ylim(0, max(y_cumavg)*1.02)\n",
+    "    plt.xlim(0, len(rows)*1.02)\n",
+    "    xp = np.linspace(0, 0.65, 300)\n",
+    "    \n",
+    "    #plt.gca().set_aspect('equal', adjustable='box')\n",
+    "    plottitle = title if title is not None else '%s_vs_%s.pdf' % (xlabel, ylabel)\n",
+    "    # plt.savefig(os.path.join(execPath,plottitle) ,  bbox_inches=\"tight\")\n",
+    "    \n",
+    "def print_avg(x, name = 'unnamed'):\n",
+    "    print 'avg(%s): %f' % (name, reduce(lambda a,b: a+b, x) / float(len(x)));\n",
+    "def average_stat(stat = 'stats.size', type = 'accept'):\n",
+    "    filtered_rows = [r for r in rows if r['stats.type'] == type]\n",
+    "    query_counts = [r['queryCount'] for r in filtered_rows]\n",
+    "    stats = [r[stat] for r in filtered_rows]\n",
+    "    plot_cumavg(query_counts, stats, xlabel='query#', ylabel=stat.replace('stats.','').replace('size','# parses'));\n",
+    "    print_avg(stats, stat)\n",
+    "plt.figure()\n",
+    "p['color'] = 'b'\n",
+    "average_stat(stat = 'stats.size')\n",
+    "#savefig('ambiguity.pdf')\n",
+    "\n",
+    "\n",
+    "def precent_status(filtered_rows, status = 'Core'):\n",
+    "    query_counts = [r['queryCount'] for r in filtered_rows]\n",
+    "    is_status = [100 if r['stats.status'] == status else 0 for r in filtered_rows]\n",
+    "    print_avg(is_status, 'percent of status ' + status)\n",
+    "    plot_cumavg(query_counts, is_status, xlabel='query #', ylabel='percent');\n",
+    "\n",
+    "\n",
+    "def plotCoreInducedNone(filtered_rows):\n",
+    "    p['color'] = 'g'; p['label'] = 'induced';\n",
+    "    precent_status(filtered_rows,status = 'Induced');\n",
+    "    p['color'] = 'b'; p['label'] = 'core';\n",
+    "    precent_status(filtered_rows, status = 'Core');\n",
+    "    plt.legend(frameon=False)\n",
+    "    \n",
+    "plt.figure()\n",
+    "plotCoreInducedNone([r for r in rows if r['stats.type'] == 'accept'])\n",
+    "# savefig('parse_status_accepted.pdf')\n",
+    "\n",
+    "def plotCoreInducedNone(filtered_rows):\n",
+    "    p['color'] = 'r'; p['label'] = 'none';\n",
+    "    precent_status(filtered_rows, status = 'Nothing');\n",
+    "    p['color'] = 'g'; p['label'] = 'induced';\n",
+    "    precent_status(filtered_rows,status = 'Induced');\n",
+    "    p['color'] = 'b'; p['label'] = 'core';\n",
+    "    precent_status(filtered_rows, status = 'Core');\n",
+    "    plt.legend(frameon=False)\n",
+    "plt.figure()\n",
+    "plotCoreInducedNone([r for r in rows if r['stats.type'] == 'q' and r.has_key('stats.status')])\n",
+    "plt.ylim(0, 70)\n",
+    "savefig('parse_status_q.pdf')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def top_users(counts, line):\n",
+    "    id = 'sessionId'\n",
+    "    if line[id] in counts:\n",
+    "        counts[line[id]] = counts[line[id]] + 1\n",
+    "    else:\n",
+    "        counts[line[id]] = 1\n",
+    "    return counts\n",
+    "\n",
+    "accept_all = [r for r in rows if r['stats.type']=='accept']\n",
+    "accept_nothing = [r for r in rows if r['stats.type']=='accept' and r['stats.status']=='Nothing']\n",
+    "\n",
+    "for r in accept_nothing[:5]:\n",
+    "    print '{q}'.format(**r)\n",
+    "    \n",
+    "sorted(reduce(top_users, rows, {}).items(), key=lambda x: -x[1]);"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def precent_status_user(filtered_rows):\n",
+    "    query_counts = [r['queryCount'] for r in filtered_rows]\n",
+    "    is_status = [100 if r['stats.status'] == 'Induced' else 0 for r in filtered_rows]\n",
+    "    plot_cumavg(query_counts, is_status, xlabel='query #', ylabel='precent induced');\n",
+    "\n",
+    "rows_to_count = [r for r in rows if r['stats.type'] == 'accept']\n",
+    "ranked_users = sorted(reduce(top_users, rows_to_count[5000:], {}).items(), key=lambda x: -x[1])\n",
+    "\n",
+    "plt.figure()\n",
+    "topnum = 5;\n",
+    "plot_reset()\n",
+    "p['alpha'] = 1;\n",
+    "p['linewidth'] = 5;\n",
+    "p['label'] = 'all'; \n",
+    "p['color'] = 'k';\n",
+    "precent_status_user([r for r in rows if r['stats.type'] == 'accept']);    \n",
+    "\n",
+    "colors = ['c', 'r', 'm', 'y', 'b', 'g']\n",
+    "plot_reset()\n",
+    "p['alpha'] = 1;\n",
+    "p['linewidth'] = 5;\n",
+    "p['alpha'] = 0.5;\n",
+    "p['marker'] = 'o';\n",
+    "p['markersize'] = 1;\n",
+    "\n",
+    "for g in enumerate(ranked_users[0:5]):\n",
+    "    # (0, (u'AMT_A1HKYY6XI2OHO1', 2830))\n",
+    "    p['label'] = '#%d' % (g[0]+1);\n",
+    "    #plotsetting['alpha'] = 1-float(g[0])/topnum;\n",
+    "    p['color'] = colors[g[0]];\n",
+    "    # print plotsetting['color']\n",
+    "    precent_status_user([r for r in rows if r['stats.type'] == 'accept' and r['sessionId'] == g[1][0]])\n",
+    "    print g\n",
+    "\n",
+    "plt.ylim(-0.1, 100)\n",
+    "plt.legend(frameon=False, loc='lower right')\n",
+    "savefig('parse_status_topuser.pdf');"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "def expressivity(status = 'Core'):\n",
+    "    filtered_rows = [r for r in rows if r['stats.type'] == 'accept' and r['stats.status'] == status]\n",
+    "    query_counts = [r['queryCount'] for r in filtered_rows]\n",
+    "    len_formula = [r['stats.len_formula'] for r in filtered_rows]\n",
+    "    len_utterance = [r['stats.len_utterance'] for r in filtered_rows]\n",
+    "    form_per_q = [float(ls[0])/ls[1] for ls in zip(len_formula, len_utterance)]\n",
+    "    plot_cumavg(query_counts, form_per_q, xlabel='query#', ylabel='\"expressiveness\"');\n",
+    "\n",
+    "    #plot_cumavg(query_counts, len_utterance, xlabel='query#', ylabel='length');\n",
+    "    print_avg(form_per_q, 'formula length')\n",
+    "plt.figure()\n",
+    "plot_reset()\n",
+    "p['color'] = 'b'; p['label'] = 'core'; expressivity('Core')\n",
+    "p['color'] = 'g'; p['label'] = 'induced'; expressivity('Induced')\n",
+    "plt.legend(frameon=False, loc='upper left')\n",
+    "# savefig('expressiveness.pdf')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def expressivity_by_users(filtered_rows):\n",
+    "    query_counts = [r['queryCount'] for r in filtered_rows]\n",
+    "    len_formula = [r['stats.len_formula'] for r in filtered_rows]\n",
+    "    len_utterance = [r['stats.len_utterance'] for r in filtered_rows]\n",
+    "    form_per_q = [float(ls[0])/ls[1] for ls in zip(len_formula, len_utterance)]\n",
+    "    plot_cumavg(query_counts, form_per_q, xlabel='query#', ylabel='len(z) / len(x)');\n",
+    "\n",
+    "plot_reset()\n",
+    "plt.figure()\n",
+    "p['alpha'] = 1;\n",
+    "p['linewidth'] = 5;\n",
+    "p['label'] = 'all'; \n",
+    "p['color'] = 'k';\n",
+    "expressivity_by_users([r for r in rows if r['stats.type'] == 'accept' ])\n",
+    "    \n",
+    "colors = ['c', 'r', 'm', 'y', 'b']\n",
+    "plot_reset()\n",
+    "p['alpha'] = 1;\n",
+    "p['linewidth'] = 5;\n",
+    "p['alpha'] = 0.5;\n",
+    "p['marker'] = 'o';\n",
+    "p['markersize'] = 1;\n",
+    "rows_to_count = [r for r in rows if r['stats.type'] == 'accept']\n",
+    "ranked_users = sorted(reduce(top_users, rows_to_count, {}).items(), key=lambda x: -x[1])\n",
+    "for g in enumerate(ranked_users[0:5]):\n",
+    "    print g\n",
+    "    p['label'] = '#%d' % (g[0]+1);\n",
+    "    p['color'] = colors[g[0]];\n",
+    "    expressivity_by_users([r for r in rows if r['stats.type'] == 'accept' \\\n",
+    "                     and r['sessionId'] == g[1][0]])\n",
+    "\n",
+    "plt.ylim([0,30])\n",
+    "plt.legend(frameon=False, loc='lower right')\n",
+    "savefig('expressiveness_by_user.pdf')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def query_reformulation_by_user():\n",
+    "    q_all = [r for r in rows if r['stats.type']=='q']\n",
+    "    q_nothing = [r for r in rows if r['stats.type']=='q' and r.has_key('stats.status') and r['stats.status']=='Nothing']\n",
+    "    print 'q_nothing: %d / %d totallines: %d' % (len(q_nothing), len(q_all), len(json_lines))\n",
+    "    ranked_users = sorted(reduce(top_users, q_all, {}).items(), key=lambda x: -x[1])\n",
+    "\n",
+    "    num_user = 3;\n",
+    "    num_query = 100;\n",
+    "    for g in (ranked_users[0:num_user]):\n",
+    "        print g\n",
+    "        print '********************'\n",
+    "        rows_userg = [r for r in rows if r['stats.type']=='q' and r['sessionId']==g[0] and r.has_key('stats.status') ]\n",
+    "        print_count = 0\n",
+    "        prev_nothing = False\n",
+    "        for r in rows_userg[-num_query:]:\n",
+    "            if prev_nothing or r['stats.status']=='Nothing':\n",
+    "                print_count = print_count + 1\n",
+    "                if print_count>100: break\n",
+    "                print r['stats.status'] + ':\\t'  + r['q'].replace('(:q \"','').replace('\")','')\n",
+    "            prev_nothing = True if r['stats.status']=='Nothing' else False\n",
+    "\n",
+    "query_reformulation_by_user()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

interactive/community-server/install-deps.py

@@ -0,0 +1,13 @@
+#!/usr/bin/env python
+
+"""
+Installs the needed dependencies for the community-server.
+
+Ideally, you would use virtualenv.
+"""
+
+import pip
+
+print("Installing community-server dependencies...")
+
+pip.main(['install', '-r', 'requirements.txt'])

interactive/community-server/requirements.txt

@@ -0,0 +1,15 @@
+eventlet==0.20.1
+Flask==0.12
+Flask-Cors==3.0.2
+Flask-SocketIO==2.8.2
+greenlet==0.4.11
+httplib2==0.18.0
+itsdangerous==0.24
+Jinja2==2.9.4
+jsonschema==2.5.1
+pep8==1.7.0
+pexpect==4.2.0
+PyJWT==1.4.2
+python-socketio==1.6.2
+requests==2.10.0
+Werkzeug==0.11.15

interactive/community-server/server.py

@@ -0,0 +1,573 @@
+#!/usr/bin/env python
+
+"""
+SHRDLURN - Community & Logging Server
+
+# Instructions
+
+You can run the server by running ```./server.py --port <PORT_NUMBER>```
+"""
+
+import json
+import sys
+import time
+import os
+import random
+import eventlet
+import glob
+from optparse import OptionParser
+from flask import Flask, request, session
+from flask_cors import CORS
+from flask_socketio import SocketIO, emit, join_room, leave_room
+import jwt
+import requests
+
+# Setup flask
+app = Flask(__name__)
+app.config['SECRET_KEY'] = os.urandom(24)
+
+# We need to enable CORS support to handle CORS flights from the frontend
+CORS(app)
+
+# The community server runs through websockets to enable real-time updates
+socketio = SocketIO(app)
+
+# Hardcoded folders for the data (mirrored in data_rotate.py)
+DATA_FOLDER = "community-server/data/"
+LOG_FOLDER = os.path.join(DATA_FOLDER, "log/")
+STRUCTS_FOLDER = os.path.join(DATA_FOLDER, "structs/")
+
+CITATION_FOLDER = "../int-output/citation"
+
+# Scoring function parameters
+GRAVITY = 1.1  # higher the gravity, the faster old structs lose score
+TIME_INTERVAL = 7200.0  # break off by every 30 minutes
+
+# Default port for the server
+DEFAULT_PORT = 8406
+
+JWT_SECRET = os.environ['SEMPRE_JWT_SECRET']
+SLACK_SECRET = os.environ['SLACK_OAUTH_SECRET']
+
+
+@app.route("/")
+def index():
+    return "Hello World! ~ SHRDLURN Community Server"
+
+
+def is_safe_path(basedir, path, follow_symlinks=True):
+    """https://security.openstack.org/guidelines/dg_using-file-paths.html"""
+    # resolves symbolic links
+    if follow_symlinks:
+        return os.path.realpath(path).startswith(os.path.realpath(basedir))
+
+    return os.path.abspath(path).startswith(os.path.abspath(basedir))
+
+
+def score_struct(timestamp, upvotesN):
+    """We use the HN formula to score structures for ranking.
+
+    Formula is: (P + 1) / ((T + 2)^GRAVITY)
+    where: - P: the number of unique upvotes for the structure
+           - T: the amount of TIME_INTERVALs that have elapsed since the
+             structure was submitted
+           - GRAVITY: a constant to determine the weight of T v. P
+    """
+    time_ago = (current_unix_time() / TIME_INTERVAL) - \
+        (int(timestamp) / TIME_INTERVAL)
+    return (upvotesN + 1) / ((time_ago + 2) ** GRAVITY)
+
+
+def current_unix_time():
+    """Returns the number of seconds since the epoch."""
+    return int(time.time())
+
+
+def emit_structs():
+    """Walk through the STRUCTS_FOLDER directory and read each struct and emit
+    it to the user one by one."""
+
+    structs = []
+    for uid in [name for name in os.listdir(STRUCTS_FOLDER) if os.path.isdir(os.path.join(STRUCTS_FOLDER, name))]:
+        uid_folder = os.path.join(STRUCTS_FOLDER, uid)
+
+        count = 0
+        for name in os.listdir(uid_folder):
+            if count > 100:
+                break
+
+            path = os.path.join(uid_folder, name)
+            if not os.path.isfile(path):
+                continue
+
+            fname = name[:-5]
+
+            try:
+                with open(path, 'r') as f:
+                    lines = f.readlines()
+
+                    upvotes = json.loads(lines[0].strip())
+                    timestamp = json.loads(lines[1].strip())
+                    struct = json.loads(lines[2].strip())
+                    image = lines[3].strip()
+
+                    score = score_struct(timestamp, len(upvotes))
+
+                    message = {"uid": uid, "id": fname, "score": score, "upvotes": [
+                        up for up in upvotes], "struct": struct, "image": image}
+
+                    structs.append(message)
+                    count += 1
+            except:
+                pass
+
+    emit("structs", structs)
+
+
+def emit_user_structs_count(uid):
+    """"Emits a count of the total number of user structs in the folder."""
+    path = os.path.join(STRUCTS_FOLDER, uid)
+    if not is_safe_path(STRUCTS_FOLDER, path):
+        return
+        structs = [name[:-5] for name in os.listdir(path) if os.path.isfile(
+            os.path.join(path, name)) and os.path.join(path, name).endswith(".json")]
+    if os.path.isdir(path):
+        structs = [name[:-5] for name in os.listdir(path) if os.path.isfile(
+            os.path.join(path, name)) and os.path.join(path, name).endswith(".json")]
+        emit("user_structs", {"structs": structs})
+
+
+def emit_utterances():
+    """Emit a list of the last 11 utterances for the 5 most recent turkers."""
+    latest_5 = []
+    for dirname, subdirs, files in os.walk(LOG_FOLDER):
+        for fname in files:
+            path = os.path.join(dirname, fname)
+
+            mtime = os.stat(path).st_mtime
+            file_info = (mtime, fname[:-5], path)
+
+            if len(latest_5) < 3:
+                latest_5.append(file_info)
+            else:
+                earliest_time = latest_5[0][0]
+                earliest_idx = 0
+                for idx, l in enumerate(latest_5):
+                    if l[0] < earliest_time:
+                        earliest_time = l[0]
+                        earliest_idx = idx
+
+                if mtime > earliest_time:
+                    latest_5[earliest_idx] = file_info
+
+    for (time, uid, path) in sorted(latest_5, key=lambda s: int(s[0]), reverse=True):
+        uid = uid
+        utts = []
+        count = 0
+        for line in reverse_readline(path):
+            data = json.loads(line)
+            if (data["type"] == "accept" or data["type"] == "define"):
+                utts.append(line)
+                count += 1
+
+            if count > 10:
+                break
+
+        message = {"uid": uid, "utterances": utts}
+        emit("utterances", message)
+
+
+def h_index(citations):
+    """https://github.com/kamyu104/LeetCode/blob/master/Python/h-index.py"""
+    n = len(citations)
+    count = [0] * (n + 1)
+    for x in citations:
+        # Put all x >= n in the same bucket.
+        if x >= n:
+            count[n] += 1
+        else:
+            count[x] += 1
+
+    h = 0
+    for i in reversed(xrange(0, n + 1)):
+        h += count[i]
+        if h >= i:
+            return i
+    return h
+
+
+def compute_citations(dir):
+    citations = []
+    for fname in os.listdir(dir):
+        if not fname.endswith(".json"):
+            continue
+
+        path = os.path.join(dir, fname)
+
+        with open(path, 'r') as f:
+            data = json.load(f)
+            citations.append(data)
+
+    citation_numbers = [citation["cite"] + citation["self"]
+                        for citation in citations]
+    citation_score = h_index(citation_numbers)
+
+    return (citations, citation_score)
+
+
+def emit_top_builders():
+    top_5_builders = []
+    for uid in os.listdir(CITATION_FOLDER):
+        subdir = os.path.join(CITATION_FOLDER, uid)
+        if not os.path.isdir(subdir):
+            continue
+
+        top_5_builders = sorted(
+            top_5_builders, key=lambda b: b[1], reverse=True)
+
+        (citations, citation_score) = compute_citations(subdir)
+
+        top_5_builders = sorted(
+            top_5_builders, key=lambda b: b[1], reverse=True)
+        if len(top_5_builders) < 10 or citation_score > top_5_builders[9][1]:
+            # If there are more than 5 citations with cites, only return those
+            citations = sorted(
+                citations, key=lambda c: c["cite"] + c["self"], reverse=True)[:10]
+            # if len(citations_with_cites) >= 6:
+            #     citations = citations_with_cites
+
+            # Sort them by score and return the top 7.
+            citations = sorted(
+                citations, key=lambda c: c["cite"] + c["self"], reverse=True)[:10]
+
+            struct = (uid, citation_score, citations)
+            if len(top_5_builders) < 10:
+                top_5_builders.append(struct)
+            else:
+                top_5_builders[9] = struct
+
+    emit("top_builders", {"top_builders": top_5_builders},
+         broadcast=True, room="community")
+
+
+def log(message):
+    """Logs the given message by writing it in the uid's JSON log file."""
+    uid = message["uid"] if 'uid' in message else "NULL_session"
+
+    user = current_user(message['token'])
+    if user:
+        uid = user['id']
+
+    path = os.path.join(LOG_FOLDER, uid + ".json")
+
+    if not is_safe_path(LOG_FOLDER, path):
+        print("NOT SAFE!", path)
+        return
+
+    # Add a timestamp to the log
+    message["timestamp"] = current_unix_time()
+
+    # Append the log to the end of the file
+    with open(path, 'a') as f:
+        json.dump(message, f)
+        f.write('\n')
+
+
+@socketio.on('getscore')
+def get_score(data):
+    user = current_user(data['token'])
+    if not user:
+        return
+
+    uid = user['id']
+    subdir = os.path.join(CITATION_FOLDER, uid)
+    if (os.path.isdir(subdir) and is_safe_path(CITATION_FOLDER, subdir)):
+        (citations, score) = compute_citations(subdir)
+        emit("score", {"score": score})
+
+
+@socketio.on('delete_struct')
+def delete_struct(data):
+    user = current_user(data['token'])
+
+    if not user:
+        return
+
+    uid = user['id']
+
+    struct_id = data["id"]
+    struct_path = struct_id + ".json"
+    subdir = os.path.join(STRUCTS_FOLDER, uid)
+    if not (is_safe_path(STRUCTS_FOLDER, subdir)):
+        return
+    path = os.path.join(subdir, struct_path)
+    if (is_safe_path(subdir, path) and os.path.isfile(path)):
+        delete_dir = os.path.join(STRUCTS_FOLDER, uid, "deleted")
+        make_dir_if_necessary(delete_dir)
+        os.rename(path, os.path.join(delete_dir, struct_path))
+
+
+@socketio.on('join')
+def on_join(data):
+    """When a user joins the "community" room, emit to them the list of
+    the top 5 most recent users' most recent 11 utterances and all of the
+    submitted structs."""
+
+    room = data['room']
+    join_room(room)
+
+    if (room == "community"):
+        # And then we emit the most recent 5 users' utterances per file
+        emit_utterances()
+
+        # and also emit the top builders when first joining
+        emit_top_builders()
+
+        # We iterate through all the shared structs and emit them one by one
+        emit_structs()
+
+
+@socketio.on('leave')
+def on_leave(data):
+    """A user can leave a room"""
+    # username = data['sessionId']
+    room = data['room']
+    leave_room(room)
+
+
+@socketio.on('share')
+def handle_share(data):
+    """Users can share structs. We save this struct in STRUCTS_FOLDER/UID/SCORE_ID.json
+
+    where UID is the uid of the user who submitted the struct, SCORE is the
+    current score of the struct and ID is the unique index (auto-incremented) of
+    this particular struct.."""
+
+    user = current_user(data['token'])
+    if not user:
+        return
+
+    uid = user['id']
+
+    user_structs_folder = os.path.join(STRUCTS_FOLDER, uid)
+    if not is_safe_path(STRUCTS_FOLDER, user_structs_folder):
+        return
+    make_dir_if_necessary(user_structs_folder)
+    new_struct_path = os.path.join(user_structs_folder, data["id"] + ".json")
+    if not is_safe_path(user_structs_folder, new_struct_path):
+        return
+
+    submission_time = current_unix_time()
+    score = score_struct(submission_time, 0)
+
+    with open(new_struct_path, 'w') as f:
+        f.write("[]\n")  # it starts with no upvoters
+        f.write(str(submission_time) + "\n")  # timestamp of submission
+        f.write(json.dumps(data["struct"]) + "\n")  # the actual struct
+        f.write(data["image"])  # the png of the struct
+
+    # Broadcast addition to the "community" room
+    message = {"uid": uid, "id": data["id"], "score": score, "upvotes": [
+    ], "struct": data["struct"], "image": data["image"]}
+    emit("struct", message, broadcast=True, room="community")
+
+
+@socketio.on('upvote')
+def upvote(data):
+    """Users can upvote other users' structures."""
+
+    user = current_user(data['token'])
+
+    # if no authenticated user, do nothing
+    if not user:
+        return
+
+    subdir = os.path.join(STRUCTS_FOLDER, data["struct_uid"])
+    if not is_safe_path(STRUCTS_FOLDER, subdir):
+        return
+    struct_path = os.path.join(subdir, str(data["id"]) + ".json")
+
+    # if the struct does not exist, do nothing
+    if not (is_safe_path(subdir, struct_path) or os.path.isfile(struct_path)):
+        print("not", struct_path)
+        return
+
+    # Read the first line of the file to get the number of upvotes
+    upvotes = []
+    score = 0
+    with open(struct_path, 'r+') as f:
+        upvotes = json.loads(f.readline().strip())
+
+        # If the user has not already upvoted this, add them
+        if user['id'] not in upvotes:
+            upvotes.append(user['id'])
+
+            timestamp = f.readline()
+            struct = f.readline()
+            image = f.readline()
+
+            # reset file to top
+            f.seek(0)
+
+            # write file back with updated upvotes
+            f.write(json.dumps(upvotes) + "\n")
+            f.write(timestamp)  # rewrite the timestamp
+            f.write(struct)  # rewrite the actual struct
+            f.write(image)  # rewrite the image
+
+            message = {"uid": data["struct_uid"],
+                       "id": data["id"], "up": user['id'], "score": score}
+
+            f.truncate()  # truncate to ensure flush appropriate
+
+            # calculate score
+            score = score_struct(timestamp, len(upvotes))
+
+            # and then broadcast the new upvote to the room:
+            message = {"uid": data["struct_uid"],
+                       "id": data["id"], "up": user['id'], "score": score}
+            emit("upvote", message, broadcast=True, room="community")
+
+
+@socketio.on('log')
+def handle_log(data):
+    """Receive a log message in the form of {"type": LOG_TYPE, "msg": LOG_OBJECT}
+
+    If the log type is an accept of utterance, then broadcast that to all
+    community-connected clients."""
+
+    if "type" not in data:
+        # If the log object is improper, don't do anything.
+        return
+
+    log(data)
+
+    # If the message is an accept or define type, broadcast it to all
+    # community-connected clients so they can update their display.
+    if data["type"] == "accept":
+        emit("new_accept", {"uid": data['uid'], "query": data["msg"]["query"], "timestamp": current_unix_time()},
+             broadcast=True, room="community")
+    elif data["type"] == "define":
+        emit("new_define", {"uid": data['uid'], "defined": data["msg"]["defineAs"], "timestamp": current_unix_time()},
+             broadcast=True, room="community")
+
+
+@socketio.on('session')
+def get_session(data):
+    """On every new connection, the client should transmit the sessionId to tell
+    the server that a new session has started. This sessionId is then used for
+    all future authentication by storing it as uid in the session global
+    context variable."""
+    session.uid = data['uid']
+    log({"type": "connect", "token": ""})
+
+
+@socketio.on('getstructcount')
+def getstructcount(data):
+    emit_user_structs_count(data['uid'])
+
+
+@socketio.on('connect')
+def connect():
+    """Return an ok if connection worked"""
+    emit('ok', {'data': 'Connected'})
+
+
+@socketio.on('disconnect')
+def disconnect():
+    """Log the fact that a user disconnected."""
+    if 'uid' in session:
+        log({"uid": session.uid, "type": "disconnect"})
+
+
+@socketio.on('sign_in')
+def sign_in(data):
+    r = requests.get("https://slack.com/api/oauth.access", params={
+                     'code': data['code'], 'client_id': '130265636855.151294060356', 'client_secret': SLACK_SECRET})
+    data = r.json()
+    session['access_token'] = data['access_token']
+    user = data['user']
+    session['user'] = user
+
+    encoded = jwt.encode(
+        {'name': user['name'], 'email': user['email'], 'id': user['id']}, JWT_SECRET, algorithm='HS256').decode('utf-8')
+
+    emit('sign_in', {
+         "name": user['name'], "email": user['email'], 'id': user['id'], 'token': encoded})
+
+
+@socketio.on('get_user')
+def get_user(data):
+    user = current_user(data['token'])
+    if user:
+        emit('sign_in', {
+            "name": user['name'], "email": user['email'], 'id': user['id'], 'token': data['token']
+        })
+    else:
+        emit('sign_in_failed')
+
+
+def current_user(token):
+    try:
+        return jwt.decode(token, JWT_SECRET, algorithms=['HS256'])
+    except:
+        return False
+
+
+# http://stackoverflow.com/questions/2301789/read-a-file-in-reverse-order-using-python
+def reverse_readline(filename, buf_size=8192):
+    """a generator that returns the lines of a file in reverse order"""
+    with open(filename) as fh:
+        segment = None
+        offset = 0
+        fh.seek(0, os.SEEK_END)
+        file_size = remaining_size = fh.tell()
+        while remaining_size > 0:
+            offset = min(file_size, offset + buf_size)
+            fh.seek(file_size - offset)
+            buffer = fh.read(min(remaining_size, buf_size))
+            remaining_size -= buf_size
+            lines = buffer.split('\n')
+            # the first line of the buffer is probably not a complete line so
+            # we'll save it and append it to the last line of the next buffer
+            # we read
+            if segment is not None:
+                # if the previous chunk starts right from the beginning of line
+                # do not concact the segment to the last line of new chunk
+                # instead, yield the segment first
+                if buffer[-1] is not '\n':
+                    lines[-1] += segment
+                else:
+                    yield segment
+            segment = lines[0]
+            for index in range(len(lines) - 1, 0, -1):
+                if len(lines[index]):
+                    yield lines[index]
+        # Don't yield None if the file was empty
+        if segment is not None:
+            yield segment
+
+
+def make_dir_if_necessary(dir_name):
+    """Creates the directory if not already created"""
+    if not os.path.exists(dir_name):
+        os.makedirs(dir_name)
+
+
+if __name__ == "__main__":
+    # Create any missing directories
+    make_dir_if_necessary(DATA_FOLDER)
+    make_dir_if_necessary(LOG_FOLDER)
+    make_dir_if_necessary(STRUCTS_FOLDER)
+
+    # Parse arguments
+    parser = OptionParser()
+    parser.add_option("-p", "--port", dest="port",
+                      help="port number to run the server", default=DEFAULT_PORT)
+    (options, args) = parser.parse_args()
+
+    # Run the server
+    # NB: socketio.run uses eventlet to run a production webserver
+    # so, make sure that "eventlet" is installed, or else it will default to
+    # the werkzeug development server which is unsafe and slow.
+    socketio.run(app, host='0.0.0.0', port=int(options.port))

interactive/csscolors.lexicon

@@ -0,0 +1,148 @@
+{"lexeme": "AliceBlue", "formula": "aliceblue", "type": "color"}
+{"lexeme": "AntiqueWhite", "formula": "antiquewhite", "type": "color"}
+{"lexeme": "Aqua", "formula": "aqua", "type": "color"}
+{"lexeme": "Aquamarine", "formula": "aquamarine", "type": "color"}
+{"lexeme": "Azure", "formula": "azure", "type": "color"}
+{"lexeme": "Beige", "formula": "beige", "type": "color"}
+{"lexeme": "Bisque", "formula": "bisque", "type": "color"}
+{"lexeme": "Black", "formula": "black", "type": "color"}
+{"lexeme": "BlanchedAlmond", "formula": "blanchedalmond", "type": "color"}
+{"lexeme": "Blue", "formula": "blue", "type": "color"}
+{"lexeme": "BlueViolet", "formula": "blueviolet", "type": "color"}
+{"lexeme": "Brown", "formula": "brown", "type": "color"}
+{"lexeme": "BurlyWood", "formula": "burlywood", "type": "color"}
+{"lexeme": "CadetBlue", "formula": "cadetblue", "type": "color"}
+{"lexeme": "Chartreuse", "formula": "chartreuse", "type": "color"}
+{"lexeme": "Chocolate", "formula": "chocolate", "type": "color"}
+{"lexeme": "Coral", "formula": "coral", "type": "color"}
+{"lexeme": "CornflowerBlue", "formula": "cornflowerblue", "type": "color"}
+{"lexeme": "Cornsilk", "formula": "cornsilk", "type": "color"}
+{"lexeme": "Crimson", "formula": "crimson", "type": "color"}
+{"lexeme": "Cyan", "formula": "cyan", "type": "color"}
+{"lexeme": "DarkBlue", "formula": "darkblue", "type": "color"}
+{"lexeme": "DarkCyan", "formula": "darkcyan", "type": "color"}
+{"lexeme": "DarkGoldenRod", "formula": "darkgoldenrod", "type": "color"}
+{"lexeme": "DarkGray", "formula": "darkgray", "type": "color"}
+{"lexeme": "DarkGrey", "formula": "darkgrey", "type": "color"}
+{"lexeme": "DarkGreen", "formula": "darkgreen", "type": "color"}
+{"lexeme": "DarkKhaki", "formula": "darkkhaki", "type": "color"}
+{"lexeme": "DarkMagenta", "formula": "darkmagenta", "type": "color"}
+{"lexeme": "DarkOliveGreen", "formula": "darkolivegreen", "type": "color"}
+{"lexeme": "DarkOrange", "formula": "darkorange", "type": "color"}
+{"lexeme": "DarkOrchid", "formula": "darkorchid", "type": "color"}
+{"lexeme": "DarkRed", "formula": "darkred", "type": "color"}
+{"lexeme": "DarkSalmon", "formula": "darksalmon", "type": "color"}
+{"lexeme": "DarkSeaGreen", "formula": "darkseagreen", "type": "color"}
+{"lexeme": "DarkSlateBlue", "formula": "darkslateblue", "type": "color"}
+{"lexeme": "DarkSlateGray", "formula": "darkslategray", "type": "color"}
+{"lexeme": "DarkSlateGrey", "formula": "darkslategrey", "type": "color"}
+{"lexeme": "DarkTurquoise", "formula": "darkturquoise", "type": "color"}
+{"lexeme": "DarkViolet", "formula": "darkviolet", "type": "color"}
+{"lexeme": "DeepPink", "formula": "deeppink", "type": "color"}
+{"lexeme": "DeepSkyBlue", "formula": "deepskyblue", "type": "color"}
+{"lexeme": "DimGray", "formula": "dimgray", "type": "color"}
+{"lexeme": "DimGrey", "formula": "dimgrey", "type": "color"}
+{"lexeme": "DodgerBlue", "formula": "dodgerblue", "type": "color"}
+{"lexeme": "FireBrick", "formula": "firebrick", "type": "color"}
+{"lexeme": "FloralWhite", "formula": "floralwhite", "type": "color"}
+{"lexeme": "ForestGreen", "formula": "forestgreen", "type": "color"}
+{"lexeme": "Fuchsia", "formula": "fuchsia", "type": "color"}
+{"lexeme": "Gainsboro", "formula": "gainsboro", "type": "color"}
+{"lexeme": "GhostWhite", "formula": "ghostwhite", "type": "color"}
+{"lexeme": "Gold", "formula": "gold", "type": "color"}
+{"lexeme": "GoldenRod", "formula": "goldenrod", "type": "color"}
+{"lexeme": "Gray", "formula": "gray", "type": "color"}
+{"lexeme": "Grey", "formula": "grey", "type": "color"}
+{"lexeme": "Green", "formula": "green", "type": "color"}
+{"lexeme": "GreenYellow", "formula": "greenyellow", "type": "color"}
+{"lexeme": "HoneyDew", "formula": "honeydew", "type": "color"}
+{"lexeme": "HotPink", "formula": "hotpink", "type": "color"}
+{"lexeme": "IndianRed", "formula": "indianred", "type": "color"}
+{"lexeme": "Indigo", "formula": "indigo", "type": "color"}
+{"lexeme": "Ivory", "formula": "ivory", "type": "color"}
+{"lexeme": "Khaki", "formula": "khaki", "type": "color"}
+{"lexeme": "Lavender", "formula": "lavender", "type": "color"}
+{"lexeme": "LavenderBlush", "formula": "lavenderblush", "type": "color"}
+{"lexeme": "LawnGreen", "formula": "lawngreen", "type": "color"}
+{"lexeme": "LemonChiffon", "formula": "lemonchiffon", "type": "color"}
+{"lexeme": "LightBlue", "formula": "lightblue", "type": "color"}
+{"lexeme": "LightCoral", "formula": "lightcoral", "type": "color"}
+{"lexeme": "LightCyan", "formula": "lightcyan", "type": "color"}
+{"lexeme": "LightGoldenRodYellow", "formula": "lightgoldenrodyellow", "type": "color"}
+{"lexeme": "LightGray", "formula": "lightgray", "type": "color"}
+{"lexeme": "LightGrey", "formula": "lightgrey", "type": "color"}
+{"lexeme": "LightGreen", "formula": "lightgreen", "type": "color"}
+{"lexeme": "LightPink", "formula": "lightpink", "type": "color"}
+{"lexeme": "LightSalmon", "formula": "lightsalmon", "type": "color"}
+{"lexeme": "LightSeaGreen", "formula": "lightseagreen", "type": "color"}
+{"lexeme": "LightSkyBlue", "formula": "lightskyblue", "type": "color"}
+{"lexeme": "LightSlateGray", "formula": "lightslategray", "type": "color"}
+{"lexeme": "LightSlateGrey", "formula": "lightslategrey", "type": "color"}
+{"lexeme": "LightSteelBlue", "formula": "lightsteelblue", "type": "color"}
+{"lexeme": "LightYellow", "formula": "lightyellow", "type": "color"}
+{"lexeme": "Lime", "formula": "lime", "type": "color"}
+{"lexeme": "LimeGreen", "formula": "limegreen", "type": "color"}
+{"lexeme": "Linen", "formula": "linen", "type": "color"}
+{"lexeme": "Magenta", "formula": "magenta", "type": "color"}
+{"lexeme": "Maroon", "formula": "maroon", "type": "color"}
+{"lexeme": "MediumAquaMarine", "formula": "mediumaquamarine", "type": "color"}
+{"lexeme": "MediumBlue", "formula": "mediumblue", "type": "color"}
+{"lexeme": "MediumOrchid", "formula": "mediumorchid", "type": "color"}
+{"lexeme": "MediumPurple", "formula": "mediumpurple", "type": "color"}
+{"lexeme": "MediumSeaGreen", "formula": "mediumseagreen", "type": "color"}
+{"lexeme": "MediumSlateBlue", "formula": "mediumslateblue", "type": "color"}
+{"lexeme": "MediumSpringGreen", "formula": "mediumspringgreen", "type": "color"}
+{"lexeme": "MediumTurquoise", "formula": "mediumturquoise", "type": "color"}
+{"lexeme": "MediumVioletRed", "formula": "mediumvioletred", "type": "color"}
+{"lexeme": "MidnightBlue", "formula": "midnightblue", "type": "color"}
+{"lexeme": "MintCream", "formula": "mintcream", "type": "color"}
+{"lexeme": "MistyRose", "formula": "mistyrose", "type": "color"}
+{"lexeme": "Moccasin", "formula": "moccasin", "type": "color"}
+{"lexeme": "NavajoWhite", "formula": "navajowhite", "type": "color"}
+{"lexeme": "Navy", "formula": "navy", "type": "color"}
+{"lexeme": "OldLace", "formula": "oldlace", "type": "color"}
+{"lexeme": "Olive", "formula": "olive", "type": "color"}
+{"lexeme": "OliveDrab", "formula": "olivedrab", "type": "color"}
+{"lexeme": "Orange", "formula": "orange", "type": "color"}
+{"lexeme": "OrangeRed", "formula": "orangered", "type": "color"}
+{"lexeme": "Orchid", "formula": "orchid", "type": "color"}
+{"lexeme": "PaleGoldenRod", "formula": "palegoldenrod", "type": "color"}
+{"lexeme": "PaleGreen", "formula": "palegreen", "type": "color"}
+{"lexeme": "PaleTurquoise", "formula": "paleturquoise", "type": "color"}
+{"lexeme": "PaleVioletRed", "formula": "palevioletred", "type": "color"}
+{"lexeme": "PapayaWhip", "formula": "papayawhip", "type": "color"}
+{"lexeme": "PeachPuff", "formula": "peachpuff", "type": "color"}
+{"lexeme": "Peru", "formula": "peru", "type": "color"}
+{"lexeme": "Pink", "formula": "pink", "type": "color"}
+{"lexeme": "Plum", "formula": "plum", "type": "color"}
+{"lexeme": "PowderBlue", "formula": "powderblue", "type": "color"}
+{"lexeme": "Purple", "formula": "purple", "type": "color"}
+{"lexeme": "RebeccaPurple", "formula": "rebeccapurple", "type": "color"}
+{"lexeme": "Red", "formula": "red", "type": "color"}
+{"lexeme": "RosyBrown", "formula": "rosybrown", "type": "color"}
+{"lexeme": "RoyalBlue", "formula": "royalblue", "type": "color"}
+{"lexeme": "SaddleBrown", "formula": "saddlebrown", "type": "color"}
+{"lexeme": "Salmon", "formula": "salmon", "type": "color"}
+{"lexeme": "SandyBrown", "formula": "sandybrown", "type": "color"}
+{"lexeme": "SeaGreen", "formula": "seagreen", "type": "color"}
+{"lexeme": "SeaShell", "formula": "seashell", "type": "color"}
+{"lexeme": "Sienna", "formula": "sienna", "type": "color"}
+{"lexeme": "Silver", "formula": "silver", "type": "color"}
+{"lexeme": "SkyBlue", "formula": "skyblue", "type": "color"}
+{"lexeme": "SlateBlue", "formula": "slateblue", "type": "color"}
+{"lexeme": "SlateGray", "formula": "slategray", "type": "color"}
+{"lexeme": "SlateGrey", "formula": "slategrey", "type": "color"}
+{"lexeme": "Snow", "formula": "snow", "type": "color"}
+{"lexeme": "SpringGreen", "formula": "springgreen", "type": "color"}
+{"lexeme": "SteelBlue", "formula": "steelblue", "type": "color"}
+{"lexeme": "Tan", "formula": "tan", "type": "color"}
+{"lexeme": "Teal", "formula": "teal", "type": "color"}
+{"lexeme": "Thistle", "formula": "thistle", "type": "color"}
+{"lexeme": "Tomato", "formula": "tomato", "type": "color"}
+{"lexeme": "Turquoise", "formula": "turquoise", "type": "color"}
+{"lexeme": "Violet", "formula": "violet", "type": "color"}
+{"lexeme": "Wheat", "formula": "wheat", "type": "color"}
+{"lexeme": "White", "formula": "white", "type": "color"}
+{"lexeme": "WhiteSmoke", "formula": "whitesmoke", "type": "color"}
+{"lexeme": "Yellow", "formula": "yellow", "type": "color"}
+{"lexeme": "YellowGreen", "formula": "yellowgreen", "type": "color"}

interactive/dal.grammar

@@ -0,0 +1,104 @@
+# Grammar for Dependency-based Action Language (DAL)
+# which is based on lambda DCS, and designed to interpolate
+# well with NL
+
+(def @select select)
+(def @update update)
+(def @remove remove)
+(def @R reverse)
+
+(rule $ROOT ($Actions) (IdentityFn) (anchored 1))
+(rule $ROOT ($Actions) (interactive.BlockFn isolate) (anchored 1))
+(rule $ROOT ($Actions) (interactive.BlockFn block) (anchored 1))
+
+(rule $Actions ($Actions ; $Action) (lambda a1 (lambda a2 (:s (var a1) (var a2)))) (anchored 1))
+(rule $Actions ($Action) (IdentityFn) (anchored 1))
+(rule $Action ({ $Actions }) (lambda a (:blk (var a))) (anchored 1))
+(rule $Action (return { $Actions }) (lambda a (:blkr (var a))) (anchored 1))
+(rule $Action ([ $Actions ]) (IdentityFn) (anchored 1))
+
+(rule $Action (repeat $Numbers $Action) (lambda n (lambda a (:loop (var n) (var a)))) (anchored 1))
+(rule $Action (if $ValueSets $Action) (lambda b (lambda a (:if (var b) (var a)))) (anchored 1))
+(rule $Action (if $Sets $Action) (lambda b (lambda a (:if (var b) (var a)))) (anchored 1))
+
+# basic actions, local scope "for has color red [add red left; add red top]"
+# (rule $Action (for $Sets $Action ) (lambda s (lambda a (:for (var s) (var a)))) (anchored 1))
+(rule $Action (foreach $Sets $Action ) (lambda s (lambda a (:foreach (var s) (var a)))) (anchored 1))
+(rule $Action (foreach $Action ) (lambda a (:foreach this (var a))) (anchored 1))
+(rule $Action (isolate $Action ) (lambda a (:isolate (var a))) (anchored 1))
+(rule $Action (while $Sets $Action ) (lambda s (lambda a (:while (var s) (var a)))) (anchored 1))
+
+# global actions
+(rule $Action (select $Sets) (lambda s (: @select (var s))) (anchored 1))
+(rule $Action (select) (ConstantFn (: @select)) (anchored 1))
+(rule $Action (update $Rel $ValueSets) (lambda r (lambda v (: @update (var r) (var v)))) (anchored 1))
+(rule $Action (remove) (ConstantFn (: @remove)) (anchored 1))
+(rule $Action (remove $Sets) (lambda s (: @remove (var s))) (anchored 1))
+
+############### unary sets
+(rule $Set (selected) this (anchored 1))
+(rule $Set (previous) prev (anchored 1))
+(rule $Set (this) this (anchored 1))
+(rule $Set (all) * (anchored 1))
+(rule $Set (nothing) nothing (anchored 1))
+(rule $Set (none) nothing (anchored 1))
+
+############### basic set operations
+# just use or and filter, which does and
+(for @op (or and)
+(rule $Sets ($Sets @op $Set) (lambda s1 (lambda s2
+					(@op (var s1) (var s2)))) (anchored 1))
+(rule $ValueSets ($ValueSets @op $ValueSet) (lambda s1 (lambda s2
+					(@op (var s1) (var s2)))) (anchored 1))
+)
+
+# this would require universe of each type
+# (rule $ValueSet (not $ValueSet) (lambda s (not (var s))) (anchored 1))
+
+(rule $Set (not $Sets) (lambda s (not (var s))) (anchored 1))
+
+(rule $ValueSets ($ValueSet) (IdentityFn) (anchored 1))
+(rule $ValueSet ([ $ValueSets ]) (IdentityFn) (anchored 1))
+
+(rule $Sets ($Set) (IdentityFn) (anchored 1))
+(rule $Set ([ $Sets ]) (IdentityFn) (anchored 1))
+
+############### Joins
+(rule $ValueSet ($Rel of $Set) (lambda r (lambda s
+			   ((@R (var r)) (var s)))) (anchored 1))
+
+(rule $Set (has $Rel $ValueSet) (lambda r (lambda s
+			   ((var r) (var s)))) (anchored 1))
+
+############### Arithmetic
+(for @op (+ -)
+(rule $Numbers ($Numbers @op $Number) (lambda n1  (lambda n2
+(@op (var n1) (var n2)))) (anchored 1))
+)
+
+(rule $Number ([ $Numbers ]) (IdentityFn) (anchored 1))
+(rule $Numbers ( $Number ) (IdentityFn) (anchored 1))
+
+#############################
+# Floating rules
+#############################
+
+# (rule $Action ($Numbers $Actions) (lambda n (lambda a (:loop (var n) (var a)))) (floating 1)) # do add red top 3 times
+# (rule $Action ($Sets $Actions) (lambda s (lambda a (:foreach (var s) (var a)))) (floating 1))
+# (rule $Action ($Actions $Actions) (lambda a1 (lambda a2 (:s (var a1) (var a2)))) (floating 1)) # add red top then remove them
+
+# let floating rules continue to propagate, and generate a lot of candidates
+# (rule $Actions ($Action) (IdentityFn) (floating 1))
+
+# (rule $ValueSets ($Rel $Sets) (lambda r (lambda s
+#			   ((reverse (var r)) (var s)))) (floating 1))
+
+# (rule $Action ($Sets) (lambda s (: @select (var s))) (floating 1))
+
+# (rule $Set ($Rel $ValueSets) (lambda r (lambda s
+# 			   ((var r) (var s)))) (floating 1))
+# (rule $Action ($Rel $ValueSets) (lambda r (lambda v (: @update (var r) (var v)))) (floating 1))
+# (rule $Action (nothing) (ConstantFn (: @remove)) (floating 1))
+
+# (rule $FROOT ($Action) (IdentityFn) (floating 1))
+# (rule $FROOT ($Actions) (IdentityFn) (floating 1))

interactive/queries/README.md

@@ -0,0 +1,23 @@
+# README
+
+## Processed
+
+* freebuild.def.json.gz contains just the 2495 definition queries in freebuild.json.gz
+
+        gzcat freebuild.json.gz | grep '(:def'
+
+* freebuild.json.gz is the main data file. Obtained by:
+
+      awk '{print "{\"qid\":" NR "," substr($0,2)}' freebuildbig-0206 > freebuild.id.json
+      jq -c '{"qid":.qid, "q":.q, "sessionId": .sessionId[:10], "time":.time}' freebuild.id.json > freebuild.json
+
+
+## Raw queries
+
+* freebuildbig-0206.def is the raw query log, without context
+
+* qualifier3-0201: 30 turkers, 1 rejection
+
+* qualifiers: turkers had to build a fixed target
+
+* freebuild[12]: qualified turkers can build whatever they want

interactive/queries/sidaw.json

@@ -0,0 +1,185 @@
+{"time":"2017-01-22T00:14:52.709","sessionId":"sidaw","q":"(:q \"add red top 3 times\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.739","sessionId":"sidaw","q":"(:q \"add red top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.786","sessionId":"sidaw","q":"(:accept \"add red top\" \"(: add red top)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.797","sessionId":"sidaw","q":"(:q \"add red top 3 times\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.805","sessionId":"sidaw","q":"(:q \"repeat 3 [add red top]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.812","sessionId":"sidaw","q":"(:accept \"repeat 3 [add red top]\" \"(:loop (number 3) (: add red top))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.820","sessionId":"sidaw","q":"(:def_ret \"add red top 3 times\" \"[[\\\"repeat 3 [add red top]\\\",\\\"(:loop (number 3) (: add red top))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.852","sessionId":"sidaw","q":"(:q \"add cardinal\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.858","sessionId":"sidaw","q":"(:q \"add red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.863","sessionId":"sidaw","q":"(:accept \"add red\" \"(: add red here)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.868","sessionId":"sidaw","q":"(:def_ret \"add cardinal\" \"[[\\\"add red\\\",\\\"(: add red here)\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.884","sessionId":"sidaw","q":"(:q \"yellow stick size 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.889","sessionId":"sidaw","q":"(:q \"repeat 5 [ add yellow; select top]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.898","sessionId":"sidaw","q":"(:accept \"repeat 5 [ add yellow; select top]\" \"(:loop (number 5) (:s (: add yellow here) (: select (call adj top))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.905","sessionId":"sidaw","q":"(:def_ret \"yellow stick size 5\" \"[[\\\"repeat 5 [ add yellow; select top]\\\",\\\"(:loop (number 5) (:s (: add yellow here) (: select (call adj top))))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.922","sessionId":"sidaw","q":"(:q \"yellow beam left 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.929","sessionId":"sidaw","q":"(:q \"{repeat  5 [ add yellow; select left]}\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.939","sessionId":"sidaw","q":"(:accept \"{repeat  5 [ add yellow; select left]}\" \"(:blk (:loop (number 5) (:s (: add yellow here) (: select (call adj left)))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.947","sessionId":"sidaw","q":"(:def_ret \"yellow beam left 5\" \"[[\\\"{repeat  5 [ add yellow; select left]}\\\",\\\"(:blk (:loop (number 5) (:s (: add yellow here) (: select (call adj left)))))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.962","sessionId":"sidaw","q":"(:q \"move down\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.968","sessionId":"sidaw","q":"(:q \"move bot\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.978","sessionId":"sidaw","q":"(:q \"select bot\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.984","sessionId":"sidaw","q":"(:accept \"select bot\" \"(: select (call adj bot))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.990","sessionId":"sidaw","q":"(:def_ret \"move down\" \"[[\\\"select bot\\\",\\\"(: select (call adj bot))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:52.999","sessionId":"sidaw","q":"(:q \"move down\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.006","sessionId":"sidaw","q":"(:q \"move bot\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.012","sessionId":"sidaw","q":"(:q \"move right\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.015","sessionId":"sidaw","q":"(:accept \"move right\" \"(: move right)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.021","sessionId":"sidaw","q":"(:q \"move up\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.028","sessionId":"sidaw","q":"(:q \"move top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.032","sessionId":"sidaw","q":"(:accept \"move top\" \"(: move top)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.036","sessionId":"sidaw","q":"(:def_ret \"move up\" \"[[\\\"move top\\\",\\\"(: move top)\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.046","sessionId":"sidaw","q":"(:q \"move up\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.051","sessionId":"sidaw","q":"(:accept \"move up\" \"(: move top)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.057","sessionId":"sidaw","q":"(:accept \"move up\" \"(not *)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.063","sessionId":"sidaw","q":"(:q \"select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.066","sessionId":"sidaw","q":"(:accept \"select top\" \"(: select (call adj top))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.070","sessionId":"sidaw","q":"(:def_ret \"move up\" \"[[\\\"select top\\\",\\\"(: select (call adj top))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.081","sessionId":"sidaw","q":"(:q \"move up\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.090","sessionId":"sidaw","q":"(:q \"select yellow\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.096","sessionId":"sidaw","q":"(:q \"select has color yellow\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.102","sessionId":"sidaw","q":"(:accept \"select has color yellow\" \"(: select (color yellow))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.106","sessionId":"sidaw","q":"(:def_ret \"select yellow\" \"[[\\\"select has color yellow\\\",\\\"(: select (color yellow))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.115","sessionId":"sidaw","q":"(:q \"update color red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.119","sessionId":"sidaw","q":"(:accept \"update color red\" \"(: update color red)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.124","sessionId":"sidaw","q":"(:q \"select origin\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.128","sessionId":"sidaw","q":"(:accept \"select origin\" \"(: select (call origin))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.135","sessionId":"sidaw","q":"(:q \"select red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.139","sessionId":"sidaw","q":"(:accept \"select red\" \"(: select (color red))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.148","sessionId":"sidaw","q":"(:q \"foreach red [move right]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.155","sessionId":"sidaw","q":"(:q \"foreach has color red [move right]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.162","sessionId":"sidaw","q":"(:accept \"foreach has color red [move right]\" \"(:foreach (color red) (: move right))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.169","sessionId":"sidaw","q":"(:def_ret \"foreach red [move right]\" \"[[\\\"foreach has color red [move right]\\\",\\\"(:foreach (color red) (: move right))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.182","sessionId":"sidaw","q":"(:q \"remove red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.187","sessionId":"sidaw","q":"(:q \"remove has color red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.191","sessionId":"sidaw","q":"(:accept \"remove has color red\" \"(: remove (color red))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.195","sessionId":"sidaw","q":"(:def_ret \"remove red\" \"[[\\\"remove has color red\\\",\\\"(: remove (color red))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.204","sessionId":"sidaw","q":"(:q \"select none\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.207","sessionId":"sidaw","q":"(:accept \"select none\" \"(: select nothing)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.213","sessionId":"sidaw","q":"(:q \"select origin\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.216","sessionId":"sidaw","q":"(:accept \"select origin\" \"(: select (call origin))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.220","sessionId":"sidaw","q":"(:q \"yellow sq size 3\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.225","sessionId":"sidaw","q":"(:q \"yellow beam back 3\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.230","sessionId":"sidaw","q":"(:q \"repeat 3 [yellow beam back 3; select left]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.238","sessionId":"sidaw","q":"(:accept \"repeat 3 [yellow beam back 3; select left]\" \"(:loop (number 3) (:s (:blk (:loop (number 3) (:s (: add yellow here) (: select (call adj back))))) (: select (call adj left))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.250","sessionId":"sidaw","q":"(:q \"select front\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.255","sessionId":"sidaw","q":"(:accept \"select front\" \"(: select (call adj front))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.259","sessionId":"sidaw","q":"(:def \"yellow sq size 3\" \"[[\\\"repeat 3 [yellow beam back 3; select left]\\\",\\\"(:loop (number 3) (:s (:blk (:loop (number 3) (:s (: add yellow here) (: select (call adj back))))) (: select (call adj left))))\\\"],[\\\"select front\\\",\\\"(: select (call adj front))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.276","sessionId":"sidaw","q":"(:q \"yellow sq size 3\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.281","sessionId":"sidaw","q":"(:accept \"yellow sq size 3\" \"(:blk (:loop (number 3) (:s (:blk (:loop (number 3) (:s (: add yellow here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.291","sessionId":"sidaw","q":"(:q \"select origin\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.295","sessionId":"sidaw","q":"(:accept \"select origin\" \"(: select (call origin))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.303","sessionId":"sidaw","q":"(:q \"red sq size 4\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.311","sessionId":"sidaw","q":"(:accept \"red sq size 4\" \"(:blk (:loop (number 4) (:s (:blk (:loop (number 4) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.327","sessionId":"sidaw","q":"(:q \"red cube size 4\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.331","sessionId":"sidaw","q":"(:q \"red sq size 4\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.337","sessionId":"sidaw","q":"(:accept \"red sq size 4\" \"(:blk (:loop (number 4) (:s (:blk (:loop (number 4) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.345","sessionId":"sidaw","q":"(:q \"repeat 4-1 [select top; red sq size 4]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.358","sessionId":"sidaw","q":"(:accept \"repeat 4-1 [select top; red sq size 4]\" \"(:loop (- (number 4) (number 1)) (:s (: select (call adj top)) (:blk (:loop (number 4) (:s (:blk (:loop (number 4) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front)))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.377","sessionId":"sidaw","q":"(:def \"red cube size 4\" \"[[\\\"red sq size 4\\\",\\\"(:blk (:loop (number 4) (:s (:blk (:loop (number 4) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front)))\\\"],[\\\"repeat 4-1 [select top; red sq size 4]\\\",\\\"(:loop (- (number 4) (number 1)) (:s (: select (call adj top)) (:blk (:loop (number 4) (:s (:blk (:loop (number 4) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front)))))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.412","sessionId":"sidaw","q":"(:q \"brown cube size 4\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.436","sessionId":"sidaw","q":"(:q \"brown cube size 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.462","sessionId":"sidaw","q":"(:q \"move back\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.471","sessionId":"sidaw","q":"(:accept \"move back\" \"(: move back)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.490","sessionId":"sidaw","q":"(:q \"select very back of very top of red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.498","sessionId":"sidaw","q":"(:q \"select very back of very top of has color red\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.508","sessionId":"sidaw","q":"(:accept \"select very back of very top of has color red\" \"(: select (call veryx back (call veryx top (color red))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.522","sessionId":"sidaw","q":"(:q \"add blue top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.528","sessionId":"sidaw","q":"(:accept \"add blue top\" \"(: add blue top)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.542","sessionId":"sidaw","q":"(:q \"select blue\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.549","sessionId":"sidaw","q":"(:accept \"select blue\" \"(: select (color blue))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.561","sessionId":"sidaw","q":"(:q \"select this and not very left and not very right\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.573","sessionId":"sidaw","q":"(:accept \"select this and not very left and not very right\" \"(: select (and (and this (not (call veryx left))) (not (call veryx right))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.593","sessionId":"sidaw","q":"(:q \"add yellow top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.599","sessionId":"sidaw","q":"(:accept \"add yellow top\" \"(: add yellow top)\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.605","sessionId":"sidaw","q":"(:q \"brown beam top 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.609","sessionId":"sidaw","q":"(:q \"brown beam top 10\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.613","sessionId":"sidaw","q":"(:q \"brown beam top 12\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.616","sessionId":"sidaw","q":"(:q \"brown beam top 9\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.621","sessionId":"sidaw","q":"(:q \"brown beam top 10\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.624","sessionId":"sidaw","q":"(:accept \"brown beam top 10\" \"(:blk (:loop (number 10) (:s (: add brown here) (: select (call adj top)))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.630","sessionId":"sidaw","q":"(:q \"green branch 3\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.637","sessionId":"sidaw","q":"(:q \"repeat 3 [add green left; add green right; add green front; add green back]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.644","sessionId":"sidaw","q":"(:accept \"repeat 3 [add green left; add green right; add green front; add green back]\" \"(:loop (number 3) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.651","sessionId":"sidaw","q":"(:def_ret \"green branch 3\" \"[[\\\"repeat 3 [add green left; add green right; add green front; add green back]\\\",\\\"(:loop (number 3) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.664","sessionId":"sidaw","q":"(:q \"select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.666","sessionId":"sidaw","q":"(:accept \"select top\" \"(: select (call adj top))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.672","sessionId":"sidaw","q":"(:q \"green branch 3\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.678","sessionId":"sidaw","q":"(:q \"green branch 4\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.682","sessionId":"sidaw","q":"(:accept \"green branch 4\" \"(:loop (number 4) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.695","sessionId":"sidaw","q":"(:q \"select top of top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.699","sessionId":"sidaw","q":"(:accept \"select top of top\" \"(: select (call adj top (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.704","sessionId":"sidaw","q":"(:q \"yellow branch 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.712","sessionId":"sidaw","q":"(:q \"green branch 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.717","sessionId":"sidaw","q":"(:accept \"green branch 5\" \"(:loop (number 5) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.728","sessionId":"sidaw","q":"(:q \"select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.734","sessionId":"sidaw","q":"(:accept \"select top\" \"(: select (call adj top))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.745","sessionId":"sidaw","q":"(:q \"green branch 3\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.751","sessionId":"sidaw","q":"(:accept \"green branch 3\" \"(:loop (number 3) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.763","sessionId":"sidaw","q":"(:q \"select top of top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.769","sessionId":"sidaw","q":"(:accept \"select top of top\" \"(: select (call adj top (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.780","sessionId":"sidaw","q":"(:q \"green branch 2\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.785","sessionId":"sidaw","q":"(:accept \"green branch 2\" \"(:loop (number 2) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.801","sessionId":"sidaw","q":"(:q \"select top of top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.808","sessionId":"sidaw","q":"(:accept \"select top of top\" \"(: select (call adj top (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.818","sessionId":"sidaw","q":"(:q \"green branch\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.826","sessionId":"sidaw","q":"(:q \"green branch 1\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.831","sessionId":"sidaw","q":"(:accept \"green branch 1\" \"(:loop (number 1) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.839","sessionId":"sidaw","q":"(:q \"brown beam top 10\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.844","sessionId":"sidaw","q":"(:accept \"brown beam top 10\" \"(:blk (:loop (number 10) (:s (: add brown here) (: select (call adj top)))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.850","sessionId":"sidaw","q":"(:q \"green branch 1; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.854","sessionId":"sidaw","q":"(:accept \"green branch 1; select top\" \"(:s (:loop (number 1) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.862","sessionId":"sidaw","q":"(:q \"green branch 2; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.866","sessionId":"sidaw","q":"(:accept \"green branch 2; select top\" \"(:s (:loop (number 2) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.872","sessionId":"sidaw","q":"(:q \"green branch 5; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.876","sessionId":"sidaw","q":"(:accept \"green branch 5; select top\" \"(:s (:loop (number 5) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.885","sessionId":"sidaw","q":"(:q \"green branch 4; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.891","sessionId":"sidaw","q":"(:accept \"green branch 4; select top\" \"(:s (:loop (number 4) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.903","sessionId":"sidaw","q":"(:q \"green branch 3; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.909","sessionId":"sidaw","q":"(:accept \"green branch 3; select top\" \"(:s (:loop (number 3) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.922","sessionId":"sidaw","q":"(:q \"green branch 2; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.927","sessionId":"sidaw","q":"(:accept \"green branch 2; select top\" \"(:s (:loop (number 2) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.941","sessionId":"sidaw","q":"(:q \"green branch 1; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.946","sessionId":"sidaw","q":"(:accept \"green branch 1; select top\" \"(:s (:loop (number 1) (:s (:s (:s (: add green left) (: add green right)) (: add green front)) (: add green back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.960","sessionId":"sidaw","q":"(:q \"red branch 1; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.966","sessionId":"sidaw","q":"(:accept \"red branch 1; select top\" \"(:s (:loop (number 1) (:s (:s (:s (: add red left) (: add red right)) (: add red front)) (: add red back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.980","sessionId":"sidaw","q":"(:q \"red branch 2; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:53.988","sessionId":"sidaw","q":"(:accept \"red branch 2; select top\" \"(:s (:loop (number 2) (:s (:s (:s (: add red left) (: add red right)) (: add red front)) (: add red back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:54.004","sessionId":"sidaw","q":"(:q \"red branch 3; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:54.012","sessionId":"sidaw","q":"(:accept \"red branch 3; select top\" \"(:s (:loop (number 3) (:s (:s (:s (: add red left) (: add red right)) (: add red front)) (: add red back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:54.030","sessionId":"sidaw","q":"(:q \"red branch 4; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:54.041","sessionId":"sidaw","q":"(:accept \"red branch 4; select top\" \"(:s (:loop (number 4) (:s (:s (:s (: add red left) (: add red right)) (: add red front)) (: add red back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:54.060","sessionId":"sidaw","q":"(:q \"red branch 5; select top\")","remote":"localhost"}
+{"time":"2017-01-22T00:14:54.073","sessionId":"sidaw","q":"(:accept \"red branch 5; select top\" \"(:s (:loop (number 5) (:s (:s (:s (: add red left) (: add red right)) (: add red front)) (: add red back))) (: select (call adj top)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:16:42.444","sessionId":"w3nFZG1uYN","q":"(:q \"while [has color red] [select up; add yellow]\")","remote":"localhost"}
+{"time":"2017-01-22T00:16:45.576","sessionId":"w3nFZG1uYN","q":"(:accept \"while [has color red] [select up; add yellow]\" \"(:while (color red) (:s (: select (call adj top)) (: add yellow here)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:17:16.687","sessionId":"w3nFZG1uYN","q":"(:q \"while [this and has color red] [select up; add yellow]\")","remote":"localhost"}
+{"time":"2017-01-22T00:17:25.866","sessionId":"w3nFZG1uYN","q":"(:q \"red cube size 5\")","remote":"localhost"}
+{"time":"2017-01-22T00:17:26.172","sessionId":"w3nFZG1uYN","q":"(:accept \"red cube size 5\" \"(:blk (:blk (:loop (number 5) (:s (:blk (:loop (number 5) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front))) (:loop (- (number 5) (number 1)) (:s (: select (call adj top)) (:blk (:loop (number 5) (:s (:blk (:loop (number 5) (:s (: add red here) (: select (call adj back))))) (: select (call adj left)))) (: select (call adj front))))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:17:27.580","sessionId":"w3nFZG1uYN","q":"(:q \"while [this and has color red] [select up; add yellow]\")","remote":"localhost"}
+{"time":"2017-01-22T00:17:42.205","sessionId":"w3nFZG1uYN","q":"(:q \"while [this and has color red] [add yellow; select up]\")","remote":"localhost"}
+{"time":"2017-01-22T00:17:43.220","sessionId":"w3nFZG1uYN","q":"(:accept \"while [this and has color red] [add yellow; select up]\" \"(:while (and this (color red)) (:s (: add yellow here) (: select (call adj top))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:24.781","sessionId":"w3nFZG1uYN","q":"(:q \"replace the red with yellow\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:33.117","sessionId":"w3nFZG1uYN","q":"(:def_ret \"replace the red with yellow\" \"[[\\\"while [this and has color red] [add yellow; select up]\\\",\\\"(:while (and this (color red)) (:s (: add yellow here) (: select (call adj top))))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:37.579","sessionId":"w3nFZG1uYN","q":"(:q \"select origin\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:38.099","sessionId":"w3nFZG1uYN","q":"(:accept \"select origin\" \"(: select (call origin))\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:41.179","sessionId":"w3nFZG1uYN","q":"(:q \"select right of right\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:45.019","sessionId":"w3nFZG1uYN","q":"(:q \"select left of left\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:45.439","sessionId":"w3nFZG1uYN","q":"(:accept \"select left of left\" \"(: select (call adj left (call adj left)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:52.202","sessionId":"w3nFZG1uYN","q":"(:q \"replace the red with orange\")","remote":"localhost"}
+{"time":"2017-01-22T00:19:52.897","sessionId":"w3nFZG1uYN","q":"(:accept \"replace the red with orange\" \"(:while (and this (color red)) (:s (: add orange here) (: select (call adj top))))\")","remote":"localhost"}
+{"time":"2017-01-22T00:20:50.449","sessionId":"w3nFZG1uYN","q":"(:q \"add red\")","remote":"localhost"}
+{"time":"2017-01-22T00:20:50.777","sessionId":"w3nFZG1uYN","q":"(:accept \"add red\" \"(: add red here)\")","remote":"localhost"}
+{"time":"2017-01-22T00:20:55.194","sessionId":"w3nFZG1uYN","q":"(:q \"select left 6 times\")","remote":"localhost"}
+{"time":"2017-01-22T00:20:55.638","sessionId":"w3nFZG1uYN","q":"(:accept \"select left 6 times\" \"(:loop (number 6) (: select (call adj left)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:21:07.717","sessionId":"w3nFZG1uYN","q":"(:q \"move right to the red block\")","remote":"localhost"}
+{"time":"2017-01-22T00:21:38.542","sessionId":"w3nFZG1uYN","q":"(:q \"while [has color red and not this] [move right]\")","remote":"localhost"}
+{"time":"2017-01-22T00:21:45.078","sessionId":"w3nFZG1uYN","q":"(:q \"while [has color red and not this] [select right]\")","remote":"localhost"}
+{"time":"2017-01-22T00:21:46.845","sessionId":"w3nFZG1uYN","q":"(:accept \"while [has color red and not this] [select right]\" \"(:while (and (color red) (not this)) (: select (call adj right)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:21:53.183","sessionId":"w3nFZG1uYN","q":"(:def_ret \"move right to the red block\" \"[[\\\"while [has color red and not this] [select right]\\\",\\\"(:while (and (color red) (not this)) (: select (call adj right)))\\\"]]\")","remote":"localhost"}
+{"time":"2017-01-22T00:21:58.891","sessionId":"w3nFZG1uYN","q":"(:q \"move left 3 times\")","remote":"localhost"}
+{"time":"2017-01-22T00:22:03.707","sessionId":"w3nFZG1uYN","q":"(:q \"select left 3 times\")","remote":"localhost"}
+{"time":"2017-01-22T00:22:04.077","sessionId":"w3nFZG1uYN","q":"(:accept \"select left 3 times\" \"(:loop (number 3) (: select (call adj left)))\")","remote":"localhost"}
+{"time":"2017-01-22T00:22:07.389","sessionId":"w3nFZG1uYN","q":"(:q \"move right to the red block\")","remote":"localhost"}
+{"time":"2017-01-22T00:22:07.797","sessionId":"w3nFZG1uYN","q":"(:accept \"move right to the red block\" \"(:while (and (color red) (not this)) (: select (call adj right)))\")","remote":"localhost"}

interactive/run

@@ -0,0 +1,233 @@
+#!/usr/bin/env ruby
+
+# This is the main entry point for running SHRDLURN.  See
+# fig/lib/execrunner.rb for more documentation for how commands are generated.
+# There are a bunch of modes that this script can be invoked with, which
+# loosely correspond to the modules.
+
+$: << 'fig/lib'
+require 'execrunner'
+$optPrefix = '-'
+$path = 'interactive'
+$output = 'interactive/output'
+$modes = []
+def addMode(name, description, func)
+  $modes << [name, description, func]
+end
+
+def codalab(dependencies=nil)
+  # Set @cl=1 to run job on CodaLab
+  dependencies ||= l(':fig', ':lib', ':module-classes.txt', ':libsempre')
+  l(
+    letDefault(:cl, 0),
+    sel(:cl,
+      l(),
+      l('cl', 'run', dependencies, '---', 'LC_ALL=C.UTF-8'),
+    nil),
+  nil)
+end
+
+def header(modules='core', codalabDependencies=nil)
+  l(
+    codalab(codalabDependencies),
+    # Queuing system
+    letDefault(:q, 0), sel(:q, l(), l('fig/bin/q', '-shareWorkingPath', o('mem', '5g'), o('memGrace', 10), '-add', '---')),
+    # Create execution directory
+    letDefault(:pooldir, 1),
+    sel(:pooldir, l(), 'fig/bin/qcreate'),
+    # Run the Java command...
+    'java',
+    '-ea',
+    '-Dmodules='+modules,
+    # Memory size
+    letDefault(:memsize, 'default'),
+    sel(:memsize, {
+      'tiny' => l('-Xms2G', '-Xmx4G'),
+      'low' => l('-Xms5G', '-Xmx7G'),
+      'default' => l('-Xms8G', '-Xmx10G'),
+      'medium' => l('-Xms12G', '-Xmx14G'),
+      'high' => l('-Xms20G', '-Xmx24G'),
+      'higher' => l('-Xms40G', '-Xmx50G'),
+      'impressive' => l('-Xms75G', '-Xmx90G'),
+    }),
+    # Classpath
+    '-cp', 'libsempre/*:lib/*',
+    # Profiling
+    letDefault(:prof, 0), sel(:prof, l(), '-Xrunhprof:cpu=samples,depth=100,file=_OUTPATH_/java.hprof.txt'),
+    # Debugging
+    letDefault(:debug, 0), sel(:debug, l(), l('-Xdebug', '-Xrunjdwp:server=y,transport=dt_socket,suspend=y,address=8898')),
+  nil)
+end
+
+def figOpts; l(selo(:pooldir, 'execDir', 'exec', '_OUTPATH_'), o('overwriteExecDir'), o('addToView', 0)) end
+
+############################################################
+# Unit tests
+
+addMode('backup', 'small commands like run community server, backup, or simulator', lambda { |e| l(
+  lambda { |e| system 'echo "backing up with mv"'},
+  lambda { |e| system 'mkdir -p ./int-backup/'},
+  letDefault(:msg, 'backing up, no message'),
+  lambda { |e| l('echo ', :msg, '> ./int-backup/message')},
+  lambda { |e| l('echo ', '`date +%Y-%m-%d.%H:%M:%S`', '>> ./int-backup/message')},
+  lambda { |e| system 'mv int-output int-backup/`date +%Y-%m-%d.%H:%M:%S`'},
+  lambda { |e| system 'mkdir -p ./int-output'},
+nil)})
+
+addMode('backup-data', 'put community-server into trash with time stamp', lambda { |e| l(
+  lambda { |e| system 'echo "backing up data with mv"'},
+  lambda { |e| system 'mkdir -p ./community-server/data-backup'},
+  lambda { |e| system 'mv ./community-server/data ./community-server/data-backup/`date +%Y-%m-%d.%H:%M:%S`'},
+nil)})
+
+addMode('trash', 'put int-output into trash with time stamp', lambda { |e| l(
+  lambda { |e| system 'echo "trashing int-output with time stamp"'},
+  lambda { |e| system 'mv int-output int-output-trash-`date +%Y-%m-%d.%H:%M:%S`'},
+  lambda { |e| system 'rm -rf int-output-trash-*'},
+  lambda { |e| system 'mkdir -p ./int-output'},
+nil)})
+
+addMode('test', 'Run unit tests for interactive stuff', lambda { |e|
+  l(
+    'java', '-ea', '-Xmx12g', '-cp', 'libsempre/*:lib/*',
+    letDefault(:debug, 0), sel(:debug, l(), l('-Xdebug', '-Xrunjdwp:server=y,transport=dt_socket,suspend=y,address=8898')),
+    'org.testng.TestNG',
+    lambda { |e|
+      if e[:class]
+        l('-testclass', 'edu.stanford.nlp.sempre.interactive.test.' + e[:class])
+      else
+        "./#{$path}/testng.xml"
+      end
+    },
+  nil)
+})
+
+addMode('simulator', 'run the simulator', lambda { |e| l(
+  # rlwrap,
+  header('core,interactive'),
+  'edu.stanford.nlp.sempre.interactive.Simulator',
+  figOpts,
+  letDefault(:server, 'local'),
+  sel(:server, {
+    'local' => o('serverURL', 'http://localhost:8410'),
+    'remote' => o('serverURL', 'http://jonsson.stanford.edu:8410')
+  }),
+  # set to 0 to enable logging
+  o('numThreads', 1),
+  letDefault(:sandbox, 'full'),
+  sel(:sandbox, {
+    'all' => o('reqParams', 'grammar=0\&cite=0\&learn=0\&logging=0'),
+    'nolog' => o('reqParams', 'grammar=0\&cite=0\&learn=1\&logging=0'),
+    'nolearn' => o('reqParams', 'grammar=1\&cite=1\&learn=0\&logging=0'),
+    'none' => o('reqParams', 'grammar=1\&cite=1\&learn=1\&logging=0'),
+    'nocite' => o('reqParams', 'grammar=1\&cite=0\&learn=1\&logging=0'),
+  }),
+  letDefault(:task, 'sidaw'),
+  sel(:task, {
+    'freebuild' => o('logFiles', "./#{$path}/queries/freebuild.json.gz"),
+    'freebuilddef' => o('logFiles', "./#{$path}/queries/freebuild.def.json.gz"),
+
+    'qual1' => o('logFiles', "./#{$path}/queries/rawqueries/qualifier1-0118.json.gz"),
+    'qual2' => o('logFiles', "./#{$path}/queries/rawqueries/qualifier2-0129.json.gz"),
+    'qual3' => o('logFiles', "./#{$path}/queries/rawqueries/qualifier3-0201.json.gz"), #  both 2 and 3
+    'free1' => o('logFiles', "./#{$path}/queries/rawqueries/freebuild1-0121.json.gz"),
+    'free2' => o('logFiles', "./#{$path}/queries/rawqueries/freebuild2-0127.json.gz"),
+    }),
+nil)})
+
+############################################################
+# {2016-07-02} [sidaw]: interactive semantic parsing
+addMode('voxelurn', 'interactive semantic parsing in a VoxelWorld', lambda { |e| l(
+  #rlwrap,
+  header('core,interactive'),
+  'edu.stanford.nlp.sempre.Main',
+  #figOpts,
+  o('server'),
+  o('masterType', 'edu.stanford.nlp.sempre.interactive.InteractiveMaster'),
+  o('Executor', 'interactive.DALExecutor'),
+  o('LanguageAnalyzer', 'interactive.DALAnalyzer'),
+  o('DALExecutor.convertNumberValues', true),
+  o('DALExecutor.printStackTrace', true),
+  o('VoxelWorld.maxBlocks', 100000),
+  selo(0, 'DALExecutor.worldType', 'VoxelWorld', 'CalendarWorld', 'Otherworld'),
+  selo(0, 'Grammar.inPaths', "./#{$path}/voxelurn.grammar"),
+
+  o('Params.initWeightsRandomly', false),
+  o('Grammar.binarizeRules', false),
+  o('Grammar.useApplyFn', 'interactive.ApplyFn'),
+
+  o('LanguageAnalyzer.lowerCaseTokens', true),
+
+  o('Parser.pruneErrorValues', true),
+  o('Parser', 'interactive.InteractiveBeamParser'),
+  o('Parser.callSetEvaluation', false),
+  o('Parser.coarsePrune', true),
+
+  o('Parser.beamSize', 50),
+  o('InteractiveBeamParser.maxNewTreesPerSpan', 5001),
+  o('ParserState.customExpectedCounts', 'None'),
+
+  selo(0, 'InteractiveBeamParser.floatStrategy', 'Never', 'NoParse', 'Always'),
+  o('InteractiveBeamParser.trackedCats', 'Number', 'Numbers', 'Color',  'Direction', 'Set', 'Sets', 'Action', 'Actions'),
+
+  o('Derivation.derivComparator', 'AnchorPriorityScoreComparator'),
+  o('Params.l1Reg', 'nonlazy'),
+  o('Params.l1RegCoeff', 0.0001),
+
+  o('Params.initStepSize', 0.1),
+  o('Params.adaptiveStepSize', true),
+  #o('Params.stepSizeReduction', 0.25),
+
+  o('FeatureExtractor.featureComputers', 'interactive.DALFeatureComputer'),
+  o('FeatureExtractor.featureDomains', ':rule', ':span', ':stats', ':scope', ':social', ':window'),
+  # o('FeatureExtractor.featureDomains', ':rule'),
+
+  o('InteractiveMaster.intOutputPath', './int-output/'),
+  o('InteractiveMaster.onlyInteractive', true),
+  o('InteractiveUtils.citationPath', './int-output/citation/'),
+
+  o('InteractiveMaster.useAligner', false),
+  o('InteractiveMaster.maxSequence', 20),
+  o('InteractiveMaster.maxChars', 200),
+
+  o('DefinitionAligner.strategies', 'ExactExclusion'),
+
+  o('InteractiveServer.numThreads', 16),
+  o('InteractiveServer.maxCandidates', 50),
+  o('InteractiveServer.queryLogPath', './int-output/query.log'),
+  o('InteractiveServer.responseLogPath', './int-output/response.log'),
+  o('InteractiveServer.port', 8410),
+
+  o('GrammarInducer.useBestPacking', true),
+  o('GrammarInducer.useSimplePacking', true),
+  o('GrammarInducer.maxNonterminals', 3),
+
+  o('Derivation.showTypes', false),
+  o('Derivation.showValues', false),
+  o('Derivation.showRules', false),
+  o('Derivation.anchoredBonus', 1.0),
+
+  o('NumberFn.allowedRange', 0, 100),
+  o('SimpleLexicon.inPaths', "./#{$path}/csscolors.lexicon"),
+
+  lambda { |e| system 'mkdir -p ./int-output/'; nil},
+  lambda { |e| system 'mkdir -p ./int-output/log/'; nil},
+  lambda { |e| system 'mkdir -p ./int-output/citation/'; nil},
+nil) })
+############################################################
+
+if ARGV.size == 0
+  puts "#{$0} @mode=<mode> [options]"
+  puts
+  puts 'This is the main entry point for all interactive related modes.'
+  puts "Modes:"
+  $modes.each { |name,description,func|
+    puts "  #{name}: #{description}"
+  }
+end
+
+modesMap = {}
+$modes.each { |name,description,func|
+  modesMap[name] = func
+}
+run!(sel(:mode, modesMap))

interactive/testng.xml

@@ -0,0 +1,16 @@
+<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
+
+<suite name="SempreInteractiveTestSuite" verbose="10">
+  <test name="InteractiveTests">
+    <classes>
+      <class name="edu.stanford.nlp.sempre.interactive.test.GrammarInducerTest">
+      </class>
+			<class name="edu.stanford.nlp.sempre.interactive.test.DALExecutorTest">
+      </class>
+			<!--
+			<class name="edu.stanford.nlp.sempre.interactive.test.FloatingParsingTest">
+			</class>
+			-->
+    </classes>
+  </test>
+</suite>

interactive/voxelurn.grammar

@@ -0,0 +1,74 @@
+# domain specific grammar for voxelurn
+
+(include dal.grammar)
+
+# domain specific actions:
+(def @put put)
+(def @base base)
+(def @add add)
+(def @move move)
+
+# syntactic sugar
+(def @veryx veryx)
+
+# typed lambdas
+# Action a, ItemSet i, ValueSet v, Rel r, Direction d, Color c, Number n
+# Rule.defaultIsFloating should be false
+
+# utilities
+(rule $Action (base $Number $Number) (lambda x (lambda y (: @base (var x) (var y)))) (anchored 1))
+(rule $Action (noop) (ConstantFn (: noop)) (anchored 1))
+
+(rule $Set (origin) (ConstantFn (call origin)) (anchored 1))
+
+# primitive actions
+(rule $Action (add $Color) (lambda c (: @add (var c) here)) (anchored 1))
+(rule $Action (add $Color $Direction) (lambda c (lambda d (: @add (var c) (var d)))) (anchored 1))
+(rule $Action (move $Direction) (lambda d (: @move (var d))) (anchored 1))
+
+# primitive actions with arguments
+
+# cube to cube relations
+(rule $Set ($Direction of $Set) (lambda d (lambda s
+			   (call adj (var d) (var s)))) (anchored 1))
+(rule $Set ($Extreme of $Set) (lambda e (lambda s
+			   (call @veryx (var e) (var s)))) (anchored 1))
+(rule $Set ($Direction) (lambda d
+			   (call adj (var d))) (anchored 1))
+(rule $Set ($Extreme) (lambda e
+			   (call @veryx (var e))) (anchored 1))
+
+(for @dir (top bot left right front back)
+  (rule $Direction (@dir) @dir (anchored 1))
+)
+(rule $Extreme (very $Direction) (IdentityFn) (anchored 1))
+
+(for @rel (col row height color name age)
+  (rule $Rel (@rel) (ConstantFn @rel) (anchored 1))
+)
+
+(rule $ValueSet ($Color) (IdentityFn) (anchored 1))
+(rule $ValueSet ($Number) (IdentityFn) (anchored 1))
+
+# set the range via NumberFn options
+
+(rule $Number ($TOKEN) (NumberFn) (anchored 1))
+(rule $Color ($TOKEN) (SimpleLexiconFn (type color)) (anchored 1))
+
+####### domain specific floating rules
+# red means has color red
+# (rule $Sets ($Color) (lambda c (color (var c))))
+
+# various actions can be performed
+# (rule $Action ($Color) (lambda c (: @add (var c) here)))
+# (rule $Action ($Color $Direction) (lambda c (lambda d (: @add (var c) (var d)))))
+# (rule $Action ($Direction) (lambda d (: @move (var d))))
+
+# (rule $Set ($Direction $Sets) (lambda d (lambda s
+#			   (call adj (var d) (var s)))))
+# (rule $Set ($Extreme $Sets) (lambda e (lambda s
+#			   (call @veryx (var e) (var s)))))
+# (rule $Set ($Direction) (lambda d
+#			   (call adj (var d))))
+# (rule $Set ($Extreme) (lambda e
+#			   (call @veryx (var e))))

overnight/README.md

@@ -14,7 +14,7 @@ To generate each individual domain:
 
 ## Training
 
-After generating the paraphrases via AMT and setting up the approprate example
+After generating the paraphrases via AMT and setting up the appropriate example
 files, we train a model.
 
 Run the following to train with all the features:
@@ -38,3 +38,7 @@ No PPDB features:
 Full system:
 
     ./run @mode=overnight @domain=<domain> -OvernightFeatureComputer.featureDomains match ppdb skip-bigram root alignment lexical root_lexical lf simpleworld
+
+# Generating alignment files for a new domain
+
+[See the instructions here](https://docs.google.com/document/d/1cgqThUAyArbgjQi0P7l--kCLH10copd1EXFnk_bHkY0/edit?usp=sharing) (work in progress).

pull-dependencies

@@ -43,7 +43,11 @@ def pull(sourcePath, dir=nil, opts={})
 
   if not $isLocal and not $isRelease
     # Download url => localPath
-    url = 'http://nlp.stanford.edu/software/sempre/dependencies-' + $version + sourcePath + ext
+    if sourcePath.start_with?('http://') || sourcePath.start_with?('https://')
+      url = sourcePath
+    else
+      url = 'http://nlp.stanford.edu/software/sempre/dependencies-' + $version + sourcePath + ext
+    end
     localPath = destDir + '/' + name + ext
     system "mkdir -p #{File.dirname(localPath)}" or exit 1
     system "wget -c '#{url}' -O #{localPath}" or exit 1
@@ -133,6 +137,9 @@ addModule('core', 'Core utilities (need to compile)', lambda {
   pull('/u/nlp/data/semparse/resources/jackson-core-2.2.0.jar')
   pull('/u/nlp/data/semparse/resources/jackson-annotations-2.2.0.jar')
   pull('/u/nlp/data/semparse/resources/jackson-databind-2.2.0.jar')
+
+  # jLine from maven central
+  pull('https://repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar')
 })
 
 addModule('corenlp', 'Stanford CoreNLP 3.6.0', lambda {
@@ -150,7 +157,11 @@ addModule('corenlp', 'Stanford CoreNLP 3.6.0', lambda {
    'stanford-corenlp-3.6.0-models.jar' => 'stanford-corenlp-models.jar',
    'stanford-corenlp-caseless-2015-04-20-models.jar' => 'stanford-corenlp-caseless-models.jar',
    'joda-time.jar' => 'joda-time.jar',
-   'jollyday.jar' => 'jollyday.jar'}.each { |key, value|
+   'jollyday.jar' => 'jollyday.jar',
+   'ejml-0.23.jar' => 'ejml.jar',
+   'slf4j-api.jar' => 'slf4j-api.jar',
+   'slf4j-simple.jar' => 'slf4j-simple.jar',
+  }.each { |key, value|
     system "ln -sfv stanford-corenlp-full-2015-12-09/#{key} lib/#{value}" or exit 1
   }
 })
@@ -170,7 +181,8 @@ addModule('corenlp-3.2.0', 'Stanford CoreNLP 3.2.0 (for backward reproducibility
    'stanford-corenlp-3.2.0-models.jar' => 'stanford-corenlp-models.jar',
    'stanford-corenlp-caseless-2013-06-07-models.jar' => 'stanford-corenlp-caseless-models.jar',
    'joda-time.jar' => 'joda-time.jar',
-   'jollyday.jar' => 'jollyday.jar'}.each { |key, value|
+   'jollyday.jar' => 'jollyday.jar'
+  }.each { |key, value|
     system "ln -sfv stanford-corenlp-full-2013-06-20/#{key} lib/#{value}" or exit 1
   }
 })
@@ -224,11 +236,30 @@ addModule('tables', 'Semantic parsing with execution on tables', lambda {
   pull('/u/nlp/data/semparse/resources/opencsv-3.0.jar')
 })
 
-addModule('tables-data', 'Data for semantic parsing with execution on tables', lambda {
-  pull('/u/nlp/data/semparse/wikitable/data', 'data/tables/', {:symlink => true})
-  pull('/u/nlp/data/semparse/wikitable/mturk-trivia-data/csv', 'data/tables/', {:symlink => true})
+addModule('tables-data', 'WikiTableQuestions dataset v1.0.2', lambda {
+  # Compact version of the dataset
+  pull('https://github.com/ppasupat/WikiTableQuestions/releases/download/v1.0.2/WikiTableQuestions-1.0.2-compact.zip', 'data')
+  # Remove old file (for backward compatibility)
+  if File.directory?('lib/data/WikiTableQuestions')
+    system 'rm -rv lib/data/WikiTableQuestions' or exit 1
+  end
+  system "cd lib/data && unzip WikiTableQuestions-1.0.2-compact.zip" or exit 1
 })
 
+addModule('tables-data-0.5', 'WikiTableQuestions dataset v0.5 (for backward reproducibility)', lambda {
+  # Compact version of the dataset
+  pull('https://github.com/ppasupat/WikiTableQuestions/releases/download/v0.5/WikiTableQuestions-0.5-compact.zip', 'data')
+  # Remove old file (for backward compatibility)
+  if File.directory?('lib/data/WikiTableQuestions')
+    system 'rm -rv lib/data/WikiTableQuestions' or exit 1
+  end
+  system "cd lib/data && unzip WikiTableQuestions-0.5-compact.zip" or exit 1
+})
+
+addModule('tables-cprune', 'Neighbor information for applying macro grammar on tables', lambda {
+  pull('/u/nlp/data/semparse/cprune/nn_0.zip', 'data/nn_0', {:symlink => true})
+  system "cd lib/data/nn_0 && unzip nn_0.zip" or exit 1
+})
 
 addModule('overnight', 'Creating a parser for multiple domains', lambda {
   # Geo evaluation
@@ -265,7 +296,19 @@ addModule('overnight', 'Creating a parser for multiple domains', lambda {
   pull('/u/nlp/data/semparse/overnight/calendar/eval/calendar.test.turk.examples', 'data/overnight/', {:symlink => true})
 })
 
+addModule('esslli_2016', 'Data for ESSLLI 2016 semantic parsing class', lambda {
+  pull('/u/nlp/data/semparse/esslli_2016', 'data/esslli_2016/', {:symlink => true})
+})
 
+addModule('geo880', 'Data, lexicon, grammars and KB for geo880', lambda {
+  pull('/u/nlp/data/semparse/geo880/geo880-test.examples', 'data/geo880', {:symlink => true})
+  pull('/u/nlp/data/semparse/geo880/geo880-test.preprocessed.examples', 'data/geo880', {:symlink => true})
+  pull('/u/nlp/data/semparse/geo880/geo880-train.preprocessed.examples', 'data/geo880', {:symlink => true})
+  pull('/u/nlp/data/semparse/geo880/geo880.grammar', 'data/geo880', {:symlink => true})
+  pull('/u/nlp/data/semparse/geo880/geo880.lexicon', 'data/geo880', {:symlink => true})
+  pull('/u/nlp/data/semparse/geo880/geo880.kg', 'data/geo880', {:symlink => true})
+  pull('/u/nlp/data/semparse/geo880/geo880.type_hierarchy', 'data/geo880', {:symlink => true})
+})
 ############################################################
 
 if ARGV.size == 0
@@ -275,6 +318,10 @@ if ARGV.size == 0
   $modules.each { |name,description,func|
     puts "  #{name}: #{description}"
   }
+  puts
+  puts "Internal use (Stanford NLP only):"
+  puts "  #{$0} -l <module-1> ...: Get the files from the local Stanford NLP server instead"
+  puts "  #{$0} -l -r <module-1> ...: Release to the public www directory on the server"
 end
 
 $modules.each { |name,description,func|

run

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 # This is the main entry point for running all SEMPRE programs.  See
 # fig/lib/execrunner.rb for more documentation for how commands are generated.
@@ -7,6 +7,7 @@
 
 $: << 'fig/lib'
 require 'execrunner'
+$optPrefix = '-'
 
 $modes = []
 def addMode(name, description, func)
@@ -55,8 +56,6 @@ def header(modules='core', codalabDependencies=nil)
   nil)
 end
 
-def rlwrap; system('which rlwrap') ? 'rlwrap' : nil end
-
 def unbalancedTrainDevSplit
   l(o('Dataset.trainFrac', 0.8), o('Dataset.devFrac', 0.2))
 end
@@ -297,7 +296,7 @@ addMode('freebase', 'Freebase (for EMNLP 2013, ACL 2014, TACL 2014)', lambda { |
     l('nlpsub', '-d/scr/yonatan/sandbox/blackhole', '-nyonatan', '-qjag', '-c3'),
     l('nlpsub', '-d/scr/yonatan/sandbox/blackhole', '-nyonatan', '-qjohn', '-c3'),
   nil),
-  sel(:interact, l(), rlwrap),
+  sel(:interact, l()),
   freebaseHeader,
   'edu.stanford.nlp.sempre.Main',
   freebaseOpts,
@@ -446,19 +445,27 @@ nil) })
 
 # Just start a simple interactive shell to try out SEMPRE commands
 addMode('simple', 'Simple shell', lambda { |e| l(
-  codalab, rlwrap, 'java', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
+  codalab, 'java', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
   o('Main.interactive'),
 nil) })
 
 addMode('simple-sparql', 'Simple shell for querying SPARQL', lambda { |e| l(
-  codalab, rlwrap, 'java', '-Dmodules=core,freebase', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
+  codalab, 'java', '-Dmodules=core,freebase', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
   o('executor', 'freebase.SparqlExecutor'),
   sparqlOpts,
   o('Main.interactive'),
 nil) })
 
+addMode('simple-lambdadcs', 'Simple shell for querying with the LambdaDCSExecutor', lambda { |e| l(
+  codalab, 'java', '-Dmodules=core,tables,corenlp', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
+  o('executor', 'tables.lambdadcs.LambdaDCSExecutor'),
+  o('FeatureExtractor.featureDomains', 'denotation lexAlign joinPos skipPos'.split),
+  o('LanguageAnalyzer.languageAnalyzer', 'corenlp.CoreNLPAnalyzer'),
+  o('Main.interactive'),
+nil) })
+
 addMode('simple-freebase', 'Simple shell for using Freebase', lambda { |e| l(
-  rlwrap, 'java', '-Dmodules=core,freebase', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
+  'java', '-Dmodules=core,freebase', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
   o('executor', 'freebase.SparqlExecutor'),
   letDefault(:sparqlserver, 'freebase.cloudapp.net:3093'),
   letDefault(:cacheserver, 'freebase.cloudapp.net:4000'),
@@ -483,6 +490,22 @@ addMode('simple-freebase', 'Simple shell for using Freebase', lambda { |e| l(
   o('Main.interactive'),
 nil) })
 
+addMode('simple-freebase-nocache', 'Simple shell for using Freebase (without a cache server)', lambda { |e| l(
+  'java', '-Dmodules=core,freebase', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.Main',
+  o('executor', 'freebase.SparqlExecutor'),
+  letDefault(:sparqlserver, 'freebase.cloudapp.net:3093'),
+  sparqlOpts,
+  o('FeatureExtractor.featureDomains', 'rule'),
+  o('Parser.coarsePrune'),
+  o('JoinFn.typeInference'),
+  o('UnaryLexicon.unaryLexiconFilePath', '/dev/null'),
+  o('BinaryLexicon.binaryLexiconFilesPath', '/dev/null'),
+  #o('JoinFn.showTypeCheckFailures'),  # Use this to debug
+  o('Grammar.inPaths', 'freebase/data/demo1.grammar'),  # Override with your own custom grammar
+  #o('SparqlExecutor.includeSupportingInfo'),  # Show full information
+  o('Main.interactive'),
+nil) })
+
 
 ############################################################
 # {2014-12-27} [Percy]: Overnight semantic parsing
@@ -501,7 +524,6 @@ def overnightFeatureDomains
 end
 
 addMode('overnight', 'Overnight semantic parsing', l(
-  rlwrap,
   header('core,freebase,overnight'),
   'edu.stanford.nlp.sempre.Main',
   figOpts,
@@ -530,7 +552,6 @@ addMode('overnight', 'Overnight semantic parsing', l(
   #o('initialization', 'paraphrase :: match,1', 'paraphrase :: size,-0.1', 'paraphrase :: ppdb,0.3',
   #  'paraphrase :: skip-bigram,0.8', 'paraphrase :: skip-ppdb,0.2','denotation :: error,-1000'),
   o('coarsePrune'),
-  o('OvernightDerivationPruningComputer.applyHardConstraints'),
   sel(2,
     l(), # no reg
     l(o('Params.l1Reg','lazy'),o('Params.l1RegCoeff',0)),
@@ -552,6 +573,7 @@ addMode('overnight', 'Overnight semantic parsing', l(
   o('phraseAlignmentPath', lambda { |e| 'lib/data/overnight/' + e[:domain] + '.phrase_alignments' }),
   o('PPDBModel.ppdbModelPath', lambda { |e| 'lib/data/overnight/' + e[:domain] + '-ppdb.txt' }),
   o('DerivationPruner.pruningComputers', ['overnight.OvernightDerivationPruningComputer']),
+  o('DerivationPruner.pruningStrategies', ['violateHardConstraints']),
   o('Dataset.inPaths', 
     lambda { |e| 'train:lib/data/overnight/' + e[:domain] + '.paraphrases.train.examples' },
     lambda { |e| 'test:lib/data/overnight/' + e[:domain] + '.paraphrases.test.examples' }),
@@ -572,8 +594,6 @@ addMode('overnight', 'Overnight semantic parsing', l(
       o('FloatingParser.maxDepth', 11),
       o('Grammar.tags','generate','general', 'geo880'),
       o('SimpleLexicon.inPaths', 'lib/data/overnight/geo880.lexicon'),
-      o('OvernightDerivationPruningComputer.usePredicateDict'),
-      o('OvernightDerivationPruningComputer.predicateDictPath','lib/data/overnight/geo880.predicate.dict'),
     nil),
     'calendar' => l(
       o('Grammar.tags','generate','general'),
@@ -613,24 +633,27 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
   # Add @cldir=1 to use CodaLab's directory paths
   letDefault(:cldir, 0),
   # Usual header
-  rlwrap, header('core,tables,corenlp'),
+  header('core,tables,corenlp,cprune'),
   # Select class
   letDefault(:class, 'main'),
   sel(:class, {
     'main' => 'edu.stanford.nlp.sempre.Main',
     'check' => 'edu.stanford.nlp.sempre.tables.test.DPDParserChecker',
-    'align' => l('edu.stanford.nlp.sempre.tables.alignment.AlignerMain', o('maxTrainIters', 11)),
     'dump' => 'edu.stanford.nlp.sempre.tables.serialize.SerializedDumper',
     'load' => l('edu.stanford.nlp.sempre.tables.serialize.SerializedLoader', let(:parser, 'serialized')),
     'stats' => 'edu.stanford.nlp.sempre.tables.test.TableStatsComputer',
-    'ann-data' => 'edu.stanford.nlp.sempre.tables.serialize.AnnotatedDatasetGenerator',
-    'ann-table' => 'edu.stanford.nlp.sempre.tables.serialize.AnnotatedTableGenerator',
+    'tag-data' => 'edu.stanford.nlp.sempre.tables.serialize.TaggedDatasetGenerator',
+    'tag-table' => 'edu.stanford.nlp.sempre.tables.serialize.TaggedTableGenerator',
+    'tag-fuzzy' => 'edu.stanford.nlp.sempre.tables.serialize.TaggedFuzzyGenerator',
     'alter' => l('edu.stanford.nlp.sempre.tables.alter.BatchTableAlterer', let(:parser, 'serialized')),
     'alter-ex' => l('edu.stanford.nlp.sempre.tables.alter.AlteredTablesExecutor', let(:parser, 'serialized')),
+    'filter' => 'edu.stanford.nlp.sempre.tables.serialize.DumpFilterer',
+    'column' => 'edu.stanford.nlp.sempre.tables.test.TableColumnAnalyzer',
+    'execute' => 'edu.stanford.nlp.sempre.tables.test.BatchTableExecutor',
   }),
   # Fig parameters
-  selo(:cldir, 'execDir', '_OUTPATH_', 'output'),
-  o('overwriteExecDir'), o('addToView', 11), o('jarFiles', 'libsempre/*'),
+  selo(:cldir, 'execDir', '_OUTPATH_', '.'),
+  o('overwriteExecDir'), o('addToView', 15), o('jarFiles', 'libsempre/*'),
   sel(:cldir, l(), '>/dev/null'),
   # Set environment for table execution
   o('executor', 'tables.lambdadcs.LambdaDCSExecutor'),
@@ -638,6 +661,7 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
   o('NumberFn.unitless'), o('NumberFn.alsoTestByConversion'),
   o('TypeInference.typeLookup', 'tables.TableTypeLookup'),
   o('JoinFn.specializedTypeCheck', false), o('JoinFn.typeInference', true),
+  o('Learner.outputPredValues'),
   # Value Evaluator
   letDefault(:eval, 'value'),
   sel(:eval, {
@@ -650,57 +674,67 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
   }),
   # Parser
   letDefault(:parser, 'floatsize'),
+  o('beamSize', 50),
+  o('useSizeInsteadOfDepth'),
   sel(:parser, {
-    'old-floatsize' => l(
+    'floatsize' => l(
       o('Builder.parser', 'FloatingParser'),
-      o('useSizeInsteadOfDepth'),
       o('FloatingParser.maxDepth', 15),
     nil),
-    'old-baseline' => l(
-      o('Builder.parser', 'tables.baseline.TableBaselineParser'),
-    nil),
+    'baseline' => o('Builder.parser', 'tables.baseline.TableBaselineParser'),
     'serialized' => o('Builder.parser', 'tables.serialize.SerializedParser'),
     # ACL 2016
     'grow-dpd' => l(
       o('Builder.parser', 'tables.dpd.DPDParser'),
-      o('useSizeInsteadOfDepth'),
       o('FloatingParser.maxDepth', 8),
     nil),
     'grow-float' => l(
       o('Builder.parser', 'FloatingParser'),
-      o('useSizeInsteadOfDepth'),
       o('FloatingParser.maxDepth', 8),
       o('FloatingParser.betaReduce'), o('initialFloatingHasZeroDepth'),
     nil),
     'grow-mix' => l(
       o('Builder.parser', 'MixParser'),
       o('MixParser.parsers', 'FloatingParser', 'tables.serialize.SerializedParser:train-0xc'),
-      o('useSizeInsteadOfDepth'),
       o('FloatingParser.maxDepth', 8),
       o('FloatingParser.betaReduce'), o('initialFloatingHasZeroDepth'),
     nil),
+    # EMNLP 2017
+    'cprune' => l(
+      o('Builder.parser', 'cprune.CPruneFloatingParser'),
+      o('FloatingParser.maxDepth', 15),
+      o('maxNumNeighbors', 40),
+      o('maxPredictedPatterns', 1000),
+    nil),
   }),
-  o('Parser.verbose', 1),
+  o('Parser.verbose', 0),
   letDefault(:pruning, 1),
   sel(:pruning,
     l(),
     l(
       o('DerivationPruner.pruningStrategies', *tablesPruningStrategies),
       o('DerivationPruner.pruningComputers', 'tables.TableDerivationPruningComputer'),
-      o('recursivePruning', false),
     nil),
   nil),
   # Grammar
   tablesGrammarPaths,
-  letDefault(:fuzzy, 1),
-  sel(:fuzzy,
-    l(),
-    l(o('fuzzyMatchSubstring'), o('fuzzyMatchMaxEditDistanceRatio', 0.3), o('alsoReturnUnion'), o('alsoMatchPart')),
-  nil),
+  letDefault(:fuzzy, 'original'),
+  sel(:fuzzy, {
+    'original' => o('FuzzyMatcher.fuzzyMatcher', 'tables.match.OriginalMatcher'),
+    'editdist-exact' => l(
+      o('FuzzyMatcher.fuzzyMatcher', 'tables.match.EditDistanceFuzzyMatcher'),
+      o('fuzzyMatchMaxEditDistanceRatio', 0.0),
+    nil),
+    'editdist-fuzzy' => l(
+      o('FuzzyMatcher.fuzzyMatcher', 'tables.match.EditDistanceFuzzyMatcher'),
+      o('fuzzyMatchSubstring'), o('fuzzyMatchMaxEditDistanceRatio', 0.15),
+      o('alsoMatchPart'),
+    nil),
+  }),
   letDefault(:normalize, 1),
   sel(:normalize,
     l(),
-    l(o('genericDateValue'), o('numberCanStartAnywhere'), o('num2CanStartAnywhere'), o('NumberFn.alsoTestByIsolatedNER')),
+    l(o('genericDateValue'), o('numberCanStartAnywhere'), o('num2CanStartAnywhere')),
   nil),
   letDefault(:anchor, 1),
   sel(:anchor, {
@@ -712,19 +746,24 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
   tablesDataPaths,
   # Verbosity
   o('FeatureVector.ignoreZeroWeight'),
-  o('maxPrintedPredictions', 10), o('maxPrintedTrue', 10), o('logFeaturesLimit', 10),
+  o('logFeaturesLimit', 10),
   o('LambdaDCSException.noErrorMessage'),
-  letDefault(:verbose, 2),
+  letDefault(:verbose, 0),
   sel(:verbose,
-    l(),
     l(
+      o('maxPrintedPredictions', 1), o('maxPrintedTrue', 1),
+    nil),
+    l(
+      o('maxPrintedPredictions', 10), o('maxPrintedTrue', 10),
       o('putCellNameInCanonicalUtterance'), o('showUtterance'),
     nil),
     l(
+      o('maxPrintedPredictions', 10), o('maxPrintedTrue', 10),
       o('putCellNameInCanonicalUtterance'), o('showUtterance'),
       o('summarizeRuleTime'), o('summarizeDenotations'),
     nil),
     l(
+      o('maxPrintedPredictions', 10), o('maxPrintedTrue', 10),
       o('putCellNameInCanonicalUtterance'), o('showUtterance'),
       o('summarizeRuleTime'), o('summarizeDenotations'),
       o('showRules'),
@@ -750,6 +789,7 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
       o('combineFromFloatingParser'),
       o('maxTrainIters', 3),
       o('showValues', false), o('showFirstValue'),
+      o('customExpectedCounts', 'TOP'),
     nil),
     l(
       # for dumping derivations (@class=dump)
@@ -773,13 +813,13 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
     'some' => l(     # Add your own features! (only set up the feature computers)
       o('FeatureExtractor.featureComputers', 'tables.features.PhrasePredicateFeatureComputer tables.features.PhraseDenotationFeatureComputer'.split),
     nil),
-    'all' => l(      # All features
+    'all' => l(      # All ACL 2015 features
       o('FeatureExtractor.featureDomains', 'custom-denotation phrase-predicate phrase-denotation headword-denotation missing-predicate'.split),
       o('FeatureExtractor.featureComputers', 'tables.features.PhrasePredicateFeatureComputer tables.features.PhraseDenotationFeatureComputer'.split),
     nil),
-    'all-fragment' => l(
-      o('FeatureExtractor.featureDomains', 'custom-denotation phrase-denotation headword-denotation phrase-fragment'.split),
-      o('FeatureExtractor.featureComputers', 'tables.features.PhraseDenotationFeatureComputer tables.features.PhraseFragmentFeatureComputer'.split),
+    'more' => l(     # All ACL 2015 features + more experimental features
+      o('FeatureExtractor.featureDomains', 'custom-denotation phrase-predicate phrase-denotation headword-denotation missing-predicate anchored-entity'.split),
+      o('FeatureExtractor.featureComputers', 'tables.features.PhrasePredicateFeatureComputer tables.features.PhraseDenotationFeatureComputer tables.features.AnchorFeatureComputer'.split),
     nil),
     'baseline' => l( # For the baseline classifier
       o('FeatureExtractor.featureDomains', 'custom-denotation phrase-denotation headword-denotation table-baseline'.split),
@@ -802,7 +842,6 @@ addMode('tables', 'QA on HTML tables', lambda { |e| l(
     'none' => l(),
     'careful' => l(
       o('maxNforLexicalizeAllPairs', 2),
-      o('lookUnderCellProperty'), o('useGenericCellType'),
       o('computeFuzzyMatchPredicates'),
     nil),
   }),
@@ -832,7 +871,7 @@ def tablesGrammarPaths
         nil),
         'combined-more' => l(
           o('Grammar.inPaths', "#{baseDir}combined.grammar"),
-          o('Grammar.tags', *'alternative movement comparison count aggregate superlative arithmetic merge u-superlative neq'.split),
+          o('Grammar.tags', *'alternative movement comparison count aggregate superlative arithmetic merge v-superlative'.split),
         nil),
         'combined-trigger' => l(    # Use trigger words for operations
           o('Grammar.inPaths', "#{baseDir}combined.grammar"),
@@ -849,6 +888,17 @@ def tablesGrammarPaths
           o('Grammar.tags', *'scoped merge-and arithmetic comparison alternative neq yearrange part closedclass scoped-2args-merge-and'.split),
           let(:anchor, 2),
         nil),
+        'grow-strict' => l(
+          o('Grammar.inPaths', "#{baseDir}grow.grammar"),
+          o('Grammar.binarizeRules', false),
+          o('Grammar.tags', *'scoped merge-and arithmetic comparison alternative neq yearrange part closedclass-generic scoped-2args-merge-and'.split),
+          let(:anchor, 2),
+        nil),
+        # EMNLP 2017
+        'extended' => l(
+          o('Grammar.inPaths', "#{baseDir}extended.grammar"),
+          o('Grammar.tags', *'alternative movement comparison count aggregate superlative arithmetic merge v-superlative'.split),
+        nil),
       }),
     nil)
   }
@@ -856,16 +906,18 @@ end
 
 def tablesDataPaths
   lambda { |e|
-    baseDir = ['lib/data/tables/data/', 'WikiTableQuestions/data/'][e[:cldir]]
-    csvDir = ['lib/data/tables/', 'WikiTableQuestions/'][e[:cldir]]
+    baseDir = ['lib/data/WikiTableQuestions/data/', 'WikiTableQuestions/data/'][e[:cldir]]
+    csvDir = ['lib/data/WikiTableQuestions/', 'WikiTableQuestions/'][e[:cldir]]
+    nnDir = ['lib/data/nn_0/', 'nn_0/'][e[:cldir]]
     datasets = {
       'none' => l(),
+      'train' => o('Dataset.inPaths', "train,#{baseDir}training.examples"),
       # Pristine test test
       'test' => l(
         o('Dataset.inPaths',
           "train,#{baseDir}training.examples",
           "test,#{baseDir}pristine-unseen-tables.examples"),
-        o('Dataset.trainFrac', 1), o('Dataset.devFrac', 0),
+        o('neighborFilePath', "#{nnDir}/exact_nearest_neighbors.all"),
       nil),
       # @data=annotated can be used with @class=check only
       'annotated' => o('Dataset.inPaths', "train,#{baseDir}annotated-all.examples"),
@@ -878,12 +930,16 @@ def tablesDataPaths
           "train,#{baseDir}random-split-#{x}-train.examples",
           "dev,#{baseDir}random-split-#{x}-dev.examples",
         nil),
+        o('neighborFilePath', "#{nnDir}/exact_nearest_neighbors.seed-#{x}.train"),
       nil)
     end
     # That's it!
     l(
-      o('splitDevFromTrain', false),
       o('TableKnowledgeGraph.baseCSVDir', csvDir),
+      # To use the normalized values from the tagged file, which were checked by hand,
+      # add @useTaggedFile=1
+      letDefault(:useTaggedFile, 0),
+      selo(:useTaggedFile, 'TableValuePreprocessor.taggedFiles', '', "#{csvDir}/tagged/data/"),
       sel(:data, datasets),
     nil)
   }
@@ -901,8 +957,9 @@ def tablesPruningStrategies
     "mistypedMerge",
     ### Strategies that depend on the children's formulas
     "doubleNext",
-    "doubleSummarizers",
+    "multipleSuperlatives",
     "sameMerge",
+    "forwardBackward",
     "unsortedMerge",
     "typeRowMerge",
   nil].compact
@@ -912,10 +969,8 @@ end
 ############################################################
 # {2015-01-18} Generate utterances [Percy]
 addMode('genovernight', 'Generate utterances for overnight semantic parsing', lambda { |e| l(
-  'fig/bin/qcreate',
-  letDefault(:gen, 0),
-  sel(:gen, rlwrap, l()),
-  'java', '-Dmodules=core,overnight', '-Xmx10g', '-cp', 'libsempre/*:lib/*', '-ea', 'edu.stanford.nlp.sempre.overnight.GenerationMain',
+  header('core,overnight'),
+  'edu.stanford.nlp.sempre.overnight.GenerationMain',
   figOpts,
   o('JoinFn.typeInference', true),
   o('JoinFn.specializedTypeCheck', false),
@@ -931,6 +986,7 @@ addMode('genovernight', 'Generate utterances for overnight semantic parsing', la
   o('FeatureExtractor.featureComputers','overnight.OvernightFeatureComputer'),
   o('OvernightFeatureComputer.featureDomains', ''),
   o('OvernightFeatureComputer.itemAnalysis',false),
+  letDefault(:gen, 1),
   sel(:gen,
     l( # For debugging the grammar
       o('FeatureExtractor.featureDomains', 'denotation'),
@@ -973,6 +1029,72 @@ addMode('genovernight-wrapper', 'Generate utterances for overnight semantic pars
   lambda { |e| system 'mkdir -p genovernight.out'; o('execDir', 'genovernight.out/' + e[:domain]) },
 nil) })
 
+addMode('geo880', 'Semantic parsing on the geo880 dataset', lambda { |e| l(
+  # Usual header
+  header('core,tables,corenlp,geo880'),
+  'edu.stanford.nlp.sempre.Main',
+  # Fig parameters
+  figOpts,
+  o('executor', 'tables.lambdadcs.LambdaDCSExecutor'),
+  o('JoinFn.specializedTypeCheck', false), o('JoinFn.typeInference', false),
+  # Parser
+  o('Builder.parser', 'BeamParser'),
+  o('Parser.coarsePrune'),
+ 
+  # Evaluation
+  o('Builder.valueEvaluator', 'geo880.Geo880ValueEvaluator'),
+
+  # Grammar
+  o('Grammar.inPaths','lib/data/geo880/geo880.grammar'),
+
+  # Type hierarchy
+  o('Geo880TypeLookup.typeHierarchyPath', 'lib/data/geo880/geo880.type_hierarchy'),
+  o('TypeInference.typeLookup','geo880.Geo880TypeLookup'),
+
+  # Yrkvpba
+  o('SimpleLexicon.inPaths', 'lib/data/geo880/geo880.lexicon'),
+  
+  # Learner
+  o('Learner.maxTrainIters', 3),
+
+  # Dataset
+  letDefault(:data, 0),
+  sel(:data,
+    l(o('Dataset.inPaths', 'train,lib/data/geo880/geo880-train.preprocessed.examples'), unbalancedTrainDevSplit), # (0) train 0.8, dev 0.2
+    l(o('Dataset.inPaths', 'train,lib/data/geo880/geo880-train.examples', 'test,lib/data/geo880/geo880-test.preprocessed/examples')), # (1) Don't run on test yet!
+  nil),
+  # Load the graph
+  o('Dataset.globalGraphPath', 'lib/data/geo880/geo880.kg'),
+  # Verbosity
+  letDefault(:verbose, 0),
+  sel(:verbose,
+    l(),
+    l(
+      o('showRules'),
+      o('Parser.verbose', 2),
+      o('JoinFn.verbose', 3),
+      o('JoinFn.showTypeCheckFailures'),
+    nil),
+  nil),
+  # Language Analyzer
+  l(o('LanguageAnalyzer', 'corenlp.CoreNLPAnalyzer'), o('annotators', *'tokenize ssplit pos lemma ner'.split)),
+  # Regularization
+  letDefault(:l1, 0),
+  sel(:l1,
+    l(),
+    l(o('Params.l1Reg','lazy'), o('Params.l1RegCoeff', '3e-5')),
+    l(o('Params.l1Reg','lazy'), selo(nil, 'Params.l1RegCoeff', 0, 0.00001, 0.0001, 0.001, 0.01)),
+  nil),
+  # Features
+  letDefault(:feat, 'freebase'),
+  sel(:feat, {
+    'none' => l(),   # No features (random)
+    'freebase' => l(
+      o('FeatureExtractor.featureDomains', 'rule opCount constant whType span lemmaAndBinaries denotation lexAlign joinPos skipPos'.split),
+#      o('FeatureExtractor.featureDomains', 'rule opCount constant whType lemmaAndBinaries denotation lexAlign joinPos skipPos'.split),
+    nil),
+  }),
+nil) })
 
 ############################################################
 

scripts/agenda-stats

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 ARGV.each { |e|
   e = e.sub(/\.exec$/, '')

scripts/extract-module-classes.rb

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 # Input: src
 # Output: module-classes.txt

scripts/find-hard-coded-paths.rb

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 # Heuristically find all hard-coded paths in the source code.
 # There should be no absolute paths.

scripts/fix-checkstyle.rb

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 # Hacky script for automatically fixing style errors.  This script is far from
 # perfect and you should manually inspect all changes before making changes.

scripts/verify-code-loop.rb

@@ -1,4 +1,4 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 # Verifies that the codebase is sane (compiles, doesn't crash, gets reasonable
 # accuracy) every once in a while.  If something fails, an email is sent out

src/edu/stanford/nlp/sempre/ActionFormula.java

@@ -0,0 +1,130 @@
+package edu.stanford.nlp.sempre;
+
+import java.util.List;
+
+import com.google.common.base.Function;
+import com.google.common.collect.Lists;
+
+import fig.basic.LispTree;
+
+/**
+ * An ActionFormula represent a compositional action used in the interactive
+ * package : is used as a prefix to denote an ActionFormula primitive (:
+ * actioname args) sequential (:s ActionFormula ActionFormula ...) repeat (:loop
+ * Number ActionFormula) conditional (:if Set ActionFormula) block scoping (:blk
+ * ActionFormula)
+ * 
+ * @author sidaw
+ */
+public class ActionFormula extends Formula {
+  public enum Mode {
+    primitive(":"), // (: remove *)
+    sequential(":s"), // (:s (: add red top) (: remove this))
+    repeat(":loop"), // (:loop (count (has color green)) (: add red top))
+    conditional(":if"), // (:if (count (has color green)) (: add red top))
+    whileloop(":while"), // (:while (count (has color green)) (: add red top))
+    forset(":for"), // (:for (and this (color red)) (:s (: add red top) (: add
+                    // yellow top) (: remove)))
+    foreach(":foreach"), // (:foreach * (add ((reverse color) this) top))
+
+    // primitives for declaring variables
+    // let(":let"), // (:let X *),
+    // set(":set"), // (:set X *)
+
+    block(":blk"), // start a block of code (like {}) with a new scope
+    blockr(":blkr"), // also return a result after finishing the block
+    isolate(":isolate"), other(":?");
+
+    private final String value;
+
+    Mode(String value) {
+      this.value = value;
+    }
+
+    @Override
+    public String toString() {
+      return this.value;
+    }
+  };
+
+  public final Mode mode;
+  public final List<Formula> args;
+
+  public ActionFormula(Mode mode, List<Formula> args) {
+    this.mode = mode;
+    this.args = args;
+  }
+
+  public static Mode parseMode(String mode) {
+    if (mode == null)
+      return null;
+    for (Mode m : Mode.values()) {
+      // LogInfo.logs("mode string %s \t== %s \t!= %s", m.toString(), mode,
+      // m.name());
+      if (m.toString().equals(mode))
+        return m;
+    }
+    if (mode.startsWith(":"))
+      throw new RuntimeException("Unsupported ActionFormula mode");
+    return null;
+  }
+
+  @Override
+  public LispTree toLispTree() {
+    LispTree tree = LispTree.proto.newList();
+    tree.addChild(this.mode.toString());
+    for (Formula arg : args)
+      tree.addChild(arg.toLispTree());
+    return tree;
+  }
+
+  @Override
+  public void forEach(Function<Formula, Boolean> func) {
+    if (!func.apply(this)) {
+      for (Formula arg : args)
+        arg.forEach(func);
+    }
+  }
+
+  @Override
+  public Formula map(Function<Formula, Formula> transform) {
+    Formula result = transform.apply(this);
+    if (result != null)
+      return result;
+    List<Formula> newArgs = Lists.newArrayList();
+    for (Formula arg : args)
+      newArgs.add(arg.map(transform));
+    return new ActionFormula(this.mode, newArgs);
+  }
+
+  @Override
+  public List<Formula> mapToList(Function<Formula, List<Formula>> transform, boolean alwaysRecurse) {
+    List<Formula> res = transform.apply(this);
+    if (res.isEmpty() || alwaysRecurse) {
+      for (Formula arg : args)
+        res.addAll(arg.mapToList(transform, alwaysRecurse));
+    }
+    return res;
+  }
+
+  @SuppressWarnings({ "equalshashcode" })
+  @Override
+  public boolean equals(Object thatObj) {
+    if (!(thatObj instanceof ActionFormula))
+      return false;
+    ActionFormula that = (ActionFormula) thatObj;
+    if (!this.mode.equals(that.mode))
+      return false;
+    if (!this.args.equals(that.args))
+      return false;
+    return true;
+  }
+
+  @Override
+  public int computeHashCode() {
+    int hash = 0x7ed55d16;
+    hash = hash * 0xd3a2646c + mode.hashCode();
+    hash = hash * 0xd3a2646c + args.hashCode();
+    return hash;
+  }
+}

src/edu/stanford/nlp/sempre/BooleanValue.java

@@ -19,6 +19,9 @@ public class BooleanValue extends Value {
     return tree;
   }
 
+  @Override public String sortString() { return "" + value; }
+  @Override public String pureString() { return "" + value; }
+
   @Override public int hashCode() { return Boolean.valueOf(value).hashCode(); }
   @Override public boolean equals(Object o) {
     if (this == o) return true;

src/edu/stanford/nlp/sempre/ChartParserState.java

@@ -22,7 +22,7 @@ public abstract class ChartParserState extends ParserState {
   protected final Map<String, List<Derivation>>[][] chart;
 
   // For visualizing how chart is filled
-  List<CatSpan> chartFillingList = new ArrayList<>();
+  protected List<CatSpan> chartFillingList = new ArrayList<>();
 
   protected String[][] phrases; // the phrases in the example
 
@@ -74,7 +74,7 @@ public abstract class ChartParserState extends ParserState {
     }
   }
 
-  void addToChart(Derivation deriv) {
+  protected void addToChart(Derivation deriv) {
     if (parser.verbose(3)) LogInfo.logs("addToChart %s: %s", deriv.cat, deriv);
 
     if (Parser.opts.pruneErrorValues && deriv.value instanceof ErrorValue) return;
@@ -90,6 +90,10 @@ public abstract class ChartParserState extends ParserState {
     }
   }
 
+  public Map<String, List<Derivation>>[][] getChart() {
+    return chart;
+  }
+
   // TODO(joberant): move to visualization utility class
   public static class CatSpan {
     @JsonProperty

src/edu/stanford/nlp/sempre/ContextValue.java

@@ -68,7 +68,7 @@ public class ContextValue extends Value {
   }
 
   public ContextValue(KnowledgeGraph graph) {
-    this(null, null, null, graph);
+    this(null, null, new ArrayList(), graph);
   }
 
   // Example:
@@ -107,8 +107,11 @@ public class ContextValue extends Value {
       tree.addChild(LispTree.proto.newList("user", user));
     if (date != null)
       tree.addChild(date.toLispTree());
+		// When logging examples, logging the entire graph takes too much screen space.
+		// I don't think that we ever deserialize a graph from a serialized context,
+		// so this should be fine.
     if (graph != null)
-      tree.addChild(graph.toLispTree());
+      tree.addChild(graph.toShortLispTree());
     for (Exchange e : exchanges)
       tree.addChild(LispTree.proto.newList("exchange", e.toLispTree()));
     return tree;

src/edu/stanford/nlp/sempre/Dataset.java

@@ -38,6 +38,9 @@ public class Dataset {
 
     @Option(gloss = "Only keep examples which have at most this number of tokens")
     public int maxTokens = Integer.MAX_VALUE;
+
+    @Option(gloss = "Path to a knowledge graph that will be uploaded as global context")
+    public String globalGraphPath;
   }
 
   public static Options opts = new Options();
@@ -96,10 +99,22 @@ public class Dataset {
         return;
       }
     }
-
     readLispTreeFromPathPairs(pathPairs);
+    updateGlobalContext();
   }
 
+  private void updateGlobalContext() {
+    if (opts.globalGraphPath != null) {
+      KnowledgeGraph graph = NaiveKnowledgeGraph.fromFile(opts.globalGraphPath);
+      for (String group : allExamples.keySet()) {
+        for (Example ex : allExamples.get(group)) {
+          ex.setContext(new ContextValue(graph));
+        }
+      }
+    }
+  }
+
+
   private void readJsonFromPathPairs(List<Pair<String, String>> pathPairs) {
     List<GroupInfo> groups = Lists.newArrayListWithCapacity(pathPairs.size());
     for (Pair<String, String> pathPair : pathPairs) {

src/edu/stanford/nlp/sempre/DateValue.java

@@ -86,6 +86,7 @@ public class DateValue extends Value {
         + "-" + (month == -1 ? "xx" : String.format("%02d", month))
         + "-" + (day == -1 ? "xx" : String.format("%02d", day));
   }
+  @Override public String pureString() { return isoString(); }
 
   @Override public int hashCode() {
     int hash = 0x7ed55d16;

src/edu/stanford/nlp/sempre/DefaultDerivationPruningComputer.java

@@ -1,10 +1,9 @@
 package edu.stanford.nlp.sempre;
 
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.Map;
+import java.util.*;
 
-import fig.basic.LogInfo;
+import fig.basic.LispTree;
+import fig.basic.Option;
 
 /**
  * Common pruning strategies that can be used in many semantic parsing tasks.
@@ -12,6 +11,11 @@ import fig.basic.LogInfo;
  * @author ppasupat
  */
 public class DefaultDerivationPruningComputer extends DerivationPruningComputer {
+  public static class Options {
+    @Option(gloss = "(for badSummarizerHead) allow count on sets of size 1")
+    public boolean allowCountOne = false;
+  }
+  public static Options opts = new Options();
 
   public DefaultDerivationPruningComputer(DerivationPruner pruner) {
     super(pruner);
@@ -22,6 +26,7 @@ public class DefaultDerivationPruningComputer extends DerivationPruningComputer
   public static final String nonLambdaError = "nonLambdaError";
   public static final String tooManyValues = "tooManyValues";
   public static final String doubleSummarizers = "doubleSummarizers";
+  public static final String multipleSuperlatives = "multipleSuperlatives";
   public static final String sameMerge = "sameMerge";
   public static final String mistypedMerge = "mistypedMerge";
   public static final String unsortedMerge = "unsortedMerge";
@@ -32,49 +37,116 @@ public class DefaultDerivationPruningComputer extends DerivationPruningComputer
     return Arrays.asList(
         atomic,
         emptyDenotation, nonLambdaError, tooManyValues,
-        doubleSummarizers, sameMerge, mistypedMerge, unsortedMerge, badSummarizerHead);
+        doubleSummarizers, multipleSuperlatives,
+        sameMerge, mistypedMerge, unsortedMerge, badSummarizerHead);
   }
 
-  // ============================================================
-  // Formula-based pruning
-  // ============================================================
-
   @Override
-  public String isPrunedWithoutExecution(Derivation deriv) {
+  public String isPruned(Derivation deriv) {
     // atomic: Prune atomic formula at root.
     //   e.g., Prevent "Who was taller, Lincoln or Obama" --> fb:en.lincoln generated from lexicon without any computation   
     if (containsStrategy(atomic)) {
-      if (deriv.isRoot(pruner.ex.numTokens()) && deriv.formula instanceof ValueFormula)
+      if (deriv.isRoot(ex.numTokens()) && deriv.formula instanceof ValueFormula)
         return atomic;
     }
-    return null;
-  }
-
-  // ============================================================
-  // Denotation-based Pruning
-  // ============================================================
-
-  @Override
-  public String isPrunedGeneral(Derivation deriv) {
     // emptyDenotation: Prune if the denotation is empty
     if (containsStrategy(emptyDenotation)) {
+      deriv.ensureExecuted(parser.executor, ex.context);
       if (deriv.value instanceof ListValue && ((ListValue) deriv.value).values.isEmpty())
         return emptyDenotation;
     }
     // nonLambdaError: Prune if the denotation is an error and the formula is not a partial formula
     if (containsStrategy(nonLambdaError) && !isLambdaFormula(deriv.formula)) {
-      if (deriv.value instanceof ErrorValue) {
-        if (DerivationPruner.opts.pruningVerbosity >= 5)
-          LogInfo.logs("NonLambdaError: %s => %s", deriv.formula, deriv.value);
+      deriv.ensureExecuted(parser.executor, ex.context);
+      if (deriv.value instanceof ErrorValue)
         return nonLambdaError;
-      }
     }
     // tooManyValues: Prune if the denotation has too many values (at $ROOT only)
-    if (containsStrategy(tooManyValues) && deriv.isRoot(pruner.ex.numTokens())) {
+    if (containsStrategy(tooManyValues) && deriv.isRoot(ex.numTokens())) {
       if (!(deriv.value instanceof ListValue) ||
           ((ListValue) deriv.value).values.size() > DerivationPruner.opts.maxNumValues)
         return tooManyValues;
     }
+    // doubleSummarizers: Prune when two summarizers (aggregate or superlative) are directly nested
+    // e.g., in (sum (avg ...)) and (min (argmax ...)), the outer operation is redundant
+    if (containsStrategy(doubleSummarizers)) {
+      Formula innerFormula = null;
+      if (deriv.formula instanceof SuperlativeFormula)
+        innerFormula = ((SuperlativeFormula) deriv.formula).head;
+      else if (deriv.formula instanceof AggregateFormula)
+        innerFormula = ((AggregateFormula) deriv.formula).child;
+      if (innerFormula != null &&
+          (innerFormula instanceof SuperlativeFormula || innerFormula instanceof AggregateFormula))
+        return doubleSummarizers;
+    }
+    // multipleSuperlatives: Prune when more than one superlatives are used
+    // (don't need to be adjacent)
+    if (containsStrategy(multipleSuperlatives)) {
+      List<LispTree> stack = new ArrayList<>();
+      int count = 0;
+      stack.add(deriv.formula.toLispTree());
+      while (!stack.isEmpty()) {
+        LispTree tree = stack.remove(stack.size() - 1);
+        if (tree.isLeaf()) {
+          if ("argmax".equals(tree.value) || "argmin".equals(tree.value)) {
+            count++;
+            if (count >= 2)
+              return multipleSuperlatives;
+          }
+        } else {
+          for (LispTree subtree : tree.children)
+            stack.add(subtree);
+        }
+      }
+    }
+    // sameMerge: Prune merge formulas with two identical children
+    if (containsStrategy(sameMerge) && deriv.formula instanceof MergeFormula) {
+      MergeFormula merge = (MergeFormula) deriv.formula;
+      if (merge.child1.equals(merge.child2))
+        return sameMerge;
+    }
+    // mistypedMerge: Prune merge formulas with children of different types
+    if (containsStrategy(mistypedMerge) && deriv.formula instanceof MergeFormula) {
+      MergeFormula merge = (MergeFormula) deriv.formula;
+      SemType type1 = TypeInference.inferType(merge.child1, true);
+      SemType type2 = TypeInference.inferType(merge.child2, true);
+      if (!type1.meet(type2).isValid())
+        return mistypedMerge;
+    }
+    // unsortedMerge: Prune merge formulas where the children's string forms are not lexicographically sorted.
+    //   Will remove redundant (and Y X) when (and Y X) is already present.
+    if (containsStrategy(unsortedMerge) && deriv.formula instanceof MergeFormula) {
+      MergeFormula merge = (MergeFormula) deriv.formula;
+      String child1 = merge.child1.toString(), child2 = merge.child2.toString();
+      if (child1.compareTo(child2) >= 0)
+        return unsortedMerge;
+    }
+    // badSummarizerHead: Prune if the head of a superlative or a non-count aggregate
+    // is empty or is a single object
+    if (containsStrategy(badSummarizerHead)) {
+      Formula innerFormula = null;
+      boolean isCount = false;
+      if (deriv.formula instanceof SuperlativeFormula)
+        innerFormula = ((SuperlativeFormula) deriv.formula).head;
+      else if (deriv.formula instanceof AggregateFormula) {
+        innerFormula = ((AggregateFormula) deriv.formula).child;
+        if (((AggregateFormula) deriv.formula).mode == AggregateFormula.Mode.count)
+          isCount = true;
+      }
+      if (innerFormula != null) {
+        try {
+          TypeInference.inferType(innerFormula);
+          Value innerValue = parser.executor.execute(innerFormula, ex.context).value;
+          if (innerValue instanceof ListValue) {
+            int size = ((ListValue) innerValue).values.size();
+            if (size == 0 || (size == 1 && !(opts.allowCountOne && isCount)))
+              return badSummarizerHead;
+          }
+        } catch (Exception e) {
+          // TypeInference fails; probably because of free variables. No need to do anything.
+        }
+      }
+    }
     return null;
   }
 
@@ -86,74 +158,4 @@ public class DefaultDerivationPruningComputer extends DerivationPruningComputer
     return false;
   }
 
-  // ============================================================
-  // Recursively prune formulas
-  // ============================================================
-
-  @Override
-  public String isPrunedRecursive(Derivation deriv, Formula subformula, Map<String, Object> state) {
-    // doubleSummarizers: Prune when two summarizers (aggregate or superlative) are directly nested
-    // e.g., in (sum (avg ...)) and (min (argmax ...)), the outer operation is redundant
-    if (containsStrategy(doubleSummarizers)) {
-      Formula innerFormula = null;
-      if (subformula instanceof SuperlativeFormula)
-        innerFormula = ((SuperlativeFormula) subformula).head;
-      else if (subformula instanceof AggregateFormula)
-        innerFormula = ((AggregateFormula) subformula).child;
-      if (innerFormula != null &&
-          (innerFormula instanceof SuperlativeFormula || innerFormula instanceof AggregateFormula))
-        return doubleSummarizers;
-    }
-    // sameMerge: Prune merge formulas with two identical children
-    if (containsStrategy(sameMerge) && subformula instanceof MergeFormula) {
-      MergeFormula merge = (MergeFormula) subformula;
-      if (merge.child1.equals(merge.child2))
-        return sameMerge;
-    }
-    // mistypedMerge: Prune merge formulas with children of different types
-    if (containsStrategy(mistypedMerge) && subformula instanceof MergeFormula) {
-      MergeFormula merge = (MergeFormula) subformula;
-      SemType type1 = TypeInference.inferType(merge.child1, true);
-      SemType type2 = TypeInference.inferType(merge.child2, true);
-      if (!type1.meet(type2).isValid())
-        return mistypedMerge;
-    }
-    // unsortedMerge: Prune merge formulas where the children's string forms are not lexicographically sorted.
-    //   Will remove redundant (and Y X) when (and Y X) is already present.
-    if (containsStrategy(unsortedMerge) && subformula instanceof MergeFormula) {
-      MergeFormula merge = (MergeFormula) subformula;
-      String child1 = merge.child1.toString(), child2 = merge.child2.toString();
-      if (child1.compareTo(child2) >= 0)
-        return unsortedMerge;
-    }
-    // badSummarizerHead: Prune if the head of a superlative or a non-count aggregate
-    // is empty or is a single object
-    if (containsStrategy(badSummarizerHead) && DerivationPruner.opts.ensureExecuted) {
-      Formula innerFormula = null;
-      boolean isCount = false;
-      if (subformula instanceof SuperlativeFormula)
-        innerFormula = ((SuperlativeFormula) subformula).head;
-      else if (subformula instanceof AggregateFormula) {
-        innerFormula = ((AggregateFormula) subformula).child;
-        if (((AggregateFormula) subformula).mode == AggregateFormula.Mode.count)
-          isCount = true;
-      }
-      if (innerFormula != null) {
-        try {
-          TypeInference.inferType(innerFormula);
-          Value innerValue = pruner.parser.executor.execute(innerFormula, pruner.ex.context).value;
-          if (innerValue instanceof ListValue) {
-            int size = ((ListValue) innerValue).values.size();
-            if (size == 0 || (size == 1 && !(DerivationPruner.opts.allowCountOne && isCount)))
-              return badSummarizerHead;
-          }
-        } catch (Exception e) {
-          // TypeInference fails; probably because of free variables. No need to do anything.
-        }
-      }
-    }
-    return null;
-  }
-
-
 }

src/edu/stanford/nlp/sempre/Derivation.java

@@ -24,8 +24,14 @@ public class Derivation implements SemanticFn.Callable, HasScore {
     public boolean showRules = false;
     @Option(gloss = "When printing derivations, to show canonical utterance")
     public boolean showUtterance = false;
+    @Option(gloss = "When printing derivations, show the category")
+    public boolean showCat = false;
     @Option(gloss = "When executing, show formulae (for debugging)")
     public boolean showExecutions = false;
+    @Option(gloss = "Pick the comparator used to sort derivations")
+    public String derivComparator = "ScoredDerivationComparator";
+    @Option(gloss = "bonus score for being all anchored")
+    public double anchoredBonus = 0.0;
   }
 
   public static Options opts = new Options();
@@ -40,8 +46,25 @@ public class Derivation implements SemanticFn.Callable, HasScore {
   // Floating cell information
   // TODO(yushi): make fields final
   public String canonicalUtterance;
+  public boolean allAnchored = true;
   private int[] numAnchors;     // Number of times each token was anchored
 
+  /**
+  * Information for grammar induction.
+  * For each descendant derivation of the body, this class tracks where and what in the head it matches
+  * GrammarInfo.start, GrammarInfo.end refer to matching positions in the head, as opposed to the body
+  * @author sidaw
+  **/
+  public class GrammarInfo {
+    public boolean anchored = false;
+    public boolean matched = false;
+    public int start = -1, end = -1;
+    public Formula formula;
+    public List<Derivation> matches = new ArrayList<>();
+  }
+  public GrammarInfo grammarInfo = new GrammarInfo();
+
+
   // If this derivation is composed of other derivations
   public final Rule rule;  // Which rule was used to produce this derivation?  Set to nullRule if not.
   public final List<Derivation> children;  // Corresponds to the RHS of the rule.
@@ -63,7 +86,7 @@ public class Derivation implements SemanticFn.Callable, HasScore {
   // Information for scoring
   private final FeatureVector localFeatureVector;  // Features
   double score = Double.NaN;  // Weighted combination of features
-  public double prob = Double.NaN;  // Probability (normalized exp of score).
+  double prob = Double.NaN;  // Probability (normalized exp of score).
 
   // Used during parsing (by FeatureExtractor, SemanticFn) to cache arbitrary
   // computation across different sub-Derivations.
@@ -93,7 +116,9 @@ public class Derivation implements SemanticFn.Callable, HasScore {
   // we can break ties consistently for reproducible results.
   long creationIndex;
   public static long numCreated = 0;  // Incremented for each derivation we create.
-  public static final Comparator<Derivation> derivScoreComparator = new ScoredDerivationComparator();
+  @SuppressWarnings("unchecked")
+  public static final Comparator<Derivation> derivScoreComparator =
+      (Comparator<Derivation>)Utils.newInstanceHard(SempreUtils.resolveClassName("Derivation$" + opts.derivComparator));
 
   public static final List<Derivation> emptyList = Collections.emptyList();
 
@@ -241,7 +266,7 @@ public class Derivation implements SemanticFn.Callable, HasScore {
   public void addFeatures(FeatureVector fv) { this.localFeatureVector.add(fv); }
 
   public double localScore(Params params) {
-    return localFeatureVector.dotProduct(params);
+    return localFeatureVector.dotProduct(params) + (this.allAnchored()? opts.anchoredBonus : 0.0);
   }
 
   // SHOULD NOT BE USED except during test time if the memory is desperately needed.
@@ -312,6 +337,9 @@ public class Derivation implements SemanticFn.Callable, HasScore {
     if (opts.showUtterance && canonicalUtterance != null) {
       tree.addChild(LispTree.proto.newList("canonicalUtterance", canonicalUtterance));
     }
+    if (opts.showCat && cat != null) {
+      tree.addChild(LispTree.proto.newList("cat", cat));
+    }
     return tree;
   }
 
@@ -432,6 +460,25 @@ public class Derivation implements SemanticFn.Callable, HasScore {
     }
   }
 
+  //Used to compare derivations by score, prioritizing the fully anchored.
+  public static class AnchorPriorityScoreComparator implements Comparator<Derivation> {
+    @Override
+    public int compare(Derivation deriv1, Derivation deriv2) {
+      boolean deriv1Core = deriv1.allAnchored();
+      boolean deriv2Core = deriv2.allAnchored();
+
+      if (deriv1Core && !deriv2Core) return -1;
+      if (deriv2Core && !deriv1Core) return +1;
+
+      if (deriv1.score > deriv2.score) return -1;
+      if (deriv1.score < deriv2.score) return +1;
+      // Ensure reproducible randomness
+      if (deriv1.creationIndex < deriv2.creationIndex) return -1;
+      if (deriv1.creationIndex > deriv2.creationIndex) return +1;
+      return 0;
+    }
+  }
+
   // for debugging
   public void printDerivationRecursively() {
     LogInfo.logs("Deriv: %s(%s,%s) %s", cat, start, end, formula);
@@ -517,4 +564,16 @@ public class Derivation implements SemanticFn.Callable, HasScore {
     Formula reduced = Formulas.betaReduction(formula);
     return new Builder().withAllFrom(this).formula(reduced).createDerivation();
   }
+
+  public boolean allAnchored() {
+    if (rule.isInduced() || !this.allAnchored) {
+      this.allAnchored = false;
+      return false;
+    } else {
+      for (Derivation child : children) {
+        if (child.allAnchored() == false) return false;
+      }
+      return true;
+    }
+  }
 }

src/edu/stanford/nlp/sempre/DerivationPruner.java

@@ -2,8 +2,6 @@ package edu.stanford.nlp.sempre;
 
 import java.util.*;
 
-import com.google.common.base.Function;
-
 import fig.basic.*;
 
 /**
@@ -24,12 +22,6 @@ public class DerivationPruner {
     @Option public int pruningVerbosity = 0;
     @Option(gloss = "(for tooManyValues) maximum denotation size of the final formula")
     public int maxNumValues = 10;
-    @Option(gloss = "(for badSummarizerHead) allow count on sets of size 1")
-    public boolean allowCountOne = false;
-    @Option(gloss = "Ensure that deriv.value contains the executed denotation.")
-    public boolean ensureExecuted = true;
-    @Option(gloss = "If false, apply recursive pruning strategies only at the outermost layer of the formula")
-    public boolean recursivePruning = true;
   }
   public static Options opts = new Options();
 
@@ -92,17 +84,9 @@ public class DerivationPruner {
    */
   public boolean isPruned(Derivation deriv) {
     if (opts.pruningStrategies.isEmpty() && pruningComputers.isEmpty()) return false;
-    if (isPrunedWithoutExecution(deriv)) return true;
-    if (isPrunedGeneral(deriv)) return true;
-    if (isPrunedRecursive(deriv)) return true;
-    return false;
-  }
-
-  // Prune without executing the formula
-  boolean isPrunedWithoutExecution(Derivation deriv) {
     String matchedStrategy;
     for (DerivationPruningComputer computer : pruningComputers) {
-      if ((matchedStrategy = computer.isPrunedWithoutExecution(deriv)) != null) {
+      if ((matchedStrategy = computer.isPruned(deriv)) != null) {
         if (opts.pruningVerbosity >= 2)
           LogInfo.logs("PRUNED [%s] %s", matchedStrategy, deriv.formula);
         return true;
@@ -111,68 +95,6 @@ public class DerivationPruner {
     return false;
   }
 
-  // Prune based on the denotation (general)
-  boolean isPrunedGeneral(Derivation deriv) {
-    if (!opts.ensureExecuted) return false;
-    deriv.ensureExecuted(parser.executor, ex.context);
-    String matchedStrategy;
-    for (DerivationPruningComputer computer : pruningComputers) {
-      if ((matchedStrategy = computer.isPrunedGeneral(deriv)) != null) {
-        if (opts.pruningVerbosity >= 2)
-          LogInfo.logs("PRUNED [%s] %s", matchedStrategy, deriv.formula);
-        return true;
-      }
-    }
-    return false;
-  }
-
-  // Prune based on subformula
-  boolean isPrunedRecursive(Derivation deriv) {
-    if (!opts.recursivePruning) {
-      // If recursivePruning flag is turned off, only look at the outermost layer.
-      if (opts.ensureExecuted)
-        deriv.ensureExecuted(parser.executor, ex.context);
-      String matchedStrategy;
-      for (DerivationPruningComputer computer : pruningComputers) {
-        if ((matchedStrategy = computer.isPrunedRecursive(deriv, deriv.formula, new HashMap<>())) != null) {
-          if (opts.pruningVerbosity >= 2)
-            LogInfo.logs("PRUNED [%s] %s", matchedStrategy, deriv.formula);
-          return true;
-        }
-      }
-    } else {
-      PruningRecurser recurser = new PruningRecurser(deriv);
-      deriv.formula.forEach(recurser);
-      if (recurser.matchedStrategy != null) {
-        if (opts.pruningVerbosity >= 2)
-          LogInfo.logs("PRUNED [%s] %s", recurser.matchedStrategy, deriv.formula);
-        return true;
-      }
-    }
-    return false;
-  }
-
-  /**
-   * Helper class for traversing the formula recursively.
-   */
-  class PruningRecurser implements Function<Formula, Boolean> {
-    final Derivation deriv;
-    final Map<String, Object> state = new HashMap<>();
-    String matchedStrategy = null;
-
-    public PruningRecurser(Derivation deriv) { this.deriv = deriv; }
-
-    @Override
-    public Boolean apply(Formula f) {
-      if (matchedStrategy != null) return true;     // Don't process further
-      for (DerivationPruningComputer computer : pruningComputers) {
-        if ((matchedStrategy = computer.isPrunedRecursive(deriv, f, state)) != null)
-          return true;   // Don't process further
-      }
-      return false;   // Recurse
-    }
-  }
-
   /**
    * Run isPruned with a (temporary) custom set of allowed pruning strategies.
    * If customAllowedPruningStrategies is null, all strategies are allowed.

src/edu/stanford/nlp/sempre/DerivationPruningComputer.java

@@ -1,7 +1,6 @@
 package edu.stanford.nlp.sempre;
 
 import java.util.Collection;
-import java.util.Map;
 
 /**
  * Used to prune formulas during parsing.
@@ -14,9 +13,13 @@ import java.util.Map;
 public abstract class DerivationPruningComputer {
 
   protected final DerivationPruner pruner;
+  protected final Parser parser;
+  protected final Example ex;
 
   public DerivationPruningComputer(DerivationPruner pruner) {
     this.pruner = pruner;
+    this.parser = pruner.parser;
+    this.ex = pruner.ex;
   }
 
   /**
@@ -30,38 +33,11 @@ public abstract class DerivationPruningComputer {
   }
 
   /**
-   * Prune the derivation without executing the formula.
+   * Prune the derivation.
    *
    * To add pruning strategies, override this method.
    * Return the strategy name to prune the formula, and null otherwise.
-   * This method should not execute the formula.
    */
-  public String isPrunedWithoutExecution(Derivation deriv) {
-    return null;
-  }
+  public abstract String isPruned(Derivation deriv);
 
-  /**
-   * Prune the derivation (general).
-   *
-   * The formula will already be executed (i.e., deriv.value is set).
-   *
-   * To add pruning strategies, override this method.
-   * Return the strategy name to prune the formula, and null otherwise.
-   */
-  public String isPrunedGeneral(Derivation deriv) {
-    return null;
-  }
-
-  /**
-   * Prune the derivation based on the subformula.
-   *
-   * DerivationPruner will traverse the formula and call this method on each subformula.
-   * The method can store temporary data in |state|.
-   *
-   * To add pruning strategies, override this method.
-   * Return the strategy name to prune the formula, and null otherwise.
-   */
-  public String isPrunedRecursive(Derivation deriv, Formula subformula, Map<String, Object> state) {
-    return null;
-  }
 }

src/edu/stanford/nlp/sempre/Example.java

@@ -198,7 +198,7 @@ public class Example {
 
   public void preprocess() {
     this.languageInfo = LanguageAnalyzer.getSingleton().analyze(this.utterance);
-    this.targetValue = TargetValuePreprocessor.getSingleton().preprocess(this.targetValue);
+    this.targetValue = TargetValuePreprocessor.getSingleton().preprocess(this.targetValue, this);
   }
 
   public void log() {
@@ -218,6 +218,21 @@ public class Example {
     LogInfo.end_track();
   }
 
+  public void logWithoutContext() {
+    LogInfo.begin_track("Example: %s", utterance);
+    LogInfo.logs("Tokens: %s", getTokens());
+    LogInfo.logs("Lemmatized tokens: %s", getLemmaTokens());
+    LogInfo.logs("POS tags: %s", languageInfo.posTags);
+    LogInfo.logs("NER tags: %s", languageInfo.nerTags);
+    LogInfo.logs("NER values: %s", languageInfo.nerValues);
+    if (targetFormula != null)
+      LogInfo.logs("targetFormula: %s", targetFormula);
+    if (targetValue != null)
+      LogInfo.logs("targetValue: %s", targetValue);
+    LogInfo.logs("Dependency children: %s", languageInfo.dependencyChildren);
+    LogInfo.end_track();
+  }
+
   public List<Derivation> getCorrectDerivations() {
     List<Derivation> res = new ArrayList<>();
     for (Derivation deriv : predDerivations) {

src/edu/stanford/nlp/sempre/ExampleUtils.java

@@ -119,6 +119,29 @@ public final class ExampleUtils {
     out.close();
   }
 
+  public static void writePredictionTSV(int iter, String group, Example ex) {
+    String basePath = "preds-iter" + iter + "-" + group + ".tsv";
+    String outPath = Execution.getFile(basePath);
+    if (outPath == null) return;
+    PrintWriter out = IOUtils.openOutAppendHard(outPath);
+
+    List<String> fields = new ArrayList<>();
+    fields.add(ex.id);
+
+    if (!ex.predDerivations.isEmpty()) {
+      Derivation deriv = ex.predDerivations.get(0);
+      if (deriv.value instanceof ListValue) {
+        List<Value> values = ((ListValue) deriv.value).values;
+        for (Value v : values) {
+          fields.add(v.pureString().replaceAll("\\s+", " ").trim());
+        }
+      }
+    }
+
+    out.println(String.join("\t", fields));
+    out.close();
+  }
+
   //read lisptree and write json
   public static void main(String[] args) {
     Dataset dataset = new Dataset();

src/edu/stanford/nlp/sempre/FeatureExtractor.java

@@ -211,10 +211,12 @@ public class FeatureExtractor {
     List<String> nonEntityLemmas = new LinkedList<>();
     extractNonEntityLemmas(ex, deriv, nonEntityLemmas);
     List<String> binaries = extractBinaries(deriv.formula);
-    String binariesStr = Joiner.on('_').join(binaries);
-    for (String nonEntityLemma : nonEntityLemmas) {
-      deriv.addFeature("lemmaAndBinaries", "nonEntitylemmas=" + nonEntityLemma +
-              ",binaries=" + binariesStr);
+    if (!binaries.isEmpty()) {
+      String binariesStr = Joiner.on('_').join(binaries);
+      for (String nonEntityLemma : nonEntityLemmas) {
+        deriv.addFeature("lemmaAndBinaries", "nonEntitylemmas=" + nonEntityLemma +
+          ",binaries=" + binariesStr);
+      }
     }
   }
 

src/edu/stanford/nlp/sempre/FloatingParser.java

@@ -84,18 +84,34 @@ public class FloatingParser extends Parser {
     @Option(gloss = "DEBUG: Print amount of time spent on each rule")
     public boolean summarizeRuleTime = false;
     @Option(gloss = "Stop the parser if it has used more than this amount of time (in seconds)")
-    public int maxFloatingParsingTime = 600;
+    public int maxFloatingParsingTime = Integer.MAX_VALUE;
   }
 
   public static Options opts = new Options();
 
-  protected List<Rule> orderedFloatingRules;
-  public List<Rule> getOrderedFloatingRules() { return orderedFloatingRules; }
+  public boolean earlyStopOnConsistent = false;
+  public int earlyStopOnNumDerivs = -1;
 
   public FloatingParser(Spec spec) {
     super(spec);
   }
 
+  /**
+   * Set early stopping criteria
+   *
+   * @param onConsistent
+   *    Stop when a consistent derivation is found. (Only triggered when computeExpectedCounts = true)
+   * @param onNumDerivs
+   *    Stop when the number of featurized derivations exceed this number (set to -1 to disable)
+   * @return
+   *    this
+   */
+  public FloatingParser setEarlyStopping(boolean onConsistent, int onNumDerivs) {
+    this.earlyStopOnConsistent = onConsistent;
+    this.earlyStopOnNumDerivs = onNumDerivs;
+    return this;
+  }
+
   /**
    * computeCatUnaryRules, but do not topologically sort floating rules
    */
@@ -379,39 +395,54 @@ class FloatingParserState extends ParserState {
       StopWatch stopWatch = new StopWatch().start();
       String rhs1 = rule.rhs.get(0);
       String rhs2 = rule.rhs.get(1);
-      if (!Rule.isCat(rhs1) || !Rule.isCat(rhs2))
-        throw new RuntimeException("Floating rules with > 1 arguments cannot have tokens on the RHS: " + rule);
 
-      if (FloatingParser.opts.useSizeInsteadOfDepth) {
-        derivLoop:
-          for (int depth1 = 0; depth1 < depth; depth1++) {  // sizes must add up to depth-1 (actually size-1)
-            int depth2 = depth - 1 - depth1;
-            for (ChildDerivationsGroup group : getFilteredDerivations(rule, floatingCell(rhs1, depth1), floatingCell(rhs2, depth2)))
-              for (Derivation deriv1 : group.derivations1)
-                for (Derivation deriv2 : group.derivations2)
-                  if (!applyFloatingRule(rule, depth, deriv1, deriv2, deriv1.canonicalUtterance + " " + deriv2.canonicalUtterance))
-                    break derivLoop;
-          }
-      } else {
-        {
-          derivLoop:
-            for (int subDepth = 0; subDepth < depth; subDepth++) {  // depth-1 <=depth-1
-              for (ChildDerivationsGroup group : getFilteredDerivations(rule, floatingCell(rhs1, depth - 1), floatingCell(rhs2, subDepth)))
-                for (Derivation deriv1 : group.derivations1)
-                  for (Derivation deriv2 : group.derivations2)
-                    if (!applyFloatingRule(rule, depth, deriv1, deriv2, deriv1.canonicalUtterance + " " + deriv2.canonicalUtterance))
-                      break derivLoop;
-            }
+      if (!Rule.isCat(rhs1) && !Rule.isCat(rhs2)) {  // token token
+        if (depth == (FloatingParser.opts.initialFloatingHasZeroDepth ? 0 : 1)) {
+          applyFloatingRule(rule, depth, null, null, rhs1 + " " + rhs2);
         }
-        {
+
+      } else if (!Rule.isCat(rhs1) && Rule.isCat(rhs2)) {  // token $Cat
+        List<Derivation> derivations = getDerivations(floatingCell(rhs2, depth - 1));
+        for (Derivation deriv : derivations)
+          applyFloatingRule(rule, depth, deriv, null, rhs1 + " " + deriv.canonicalUtterance);
+
+      } else if (Rule.isCat(rhs1) && !Rule.isCat(rhs2)) {  // $Cat token
+        List<Derivation> derivations = getDerivations(floatingCell(rhs1, depth - 1));
+        for (Derivation deriv : derivations)
+          applyFloatingRule(rule, depth, deriv, null, deriv.canonicalUtterance + " " + rhs2);
+
+      } else {  // $Cat $Cat
+        if (FloatingParser.opts.useSizeInsteadOfDepth) {
           derivLoop:
-            for (int subDepth = 0; subDepth < depth - 1; subDepth++) {  // <depth-1 depth-1
-              for (ChildDerivationsGroup group : getFilteredDerivations(rule, floatingCell(rhs1, subDepth), floatingCell(rhs2, depth - 1)))
+            for (int depth1 = 0; depth1 < depth; depth1++) {  // sizes must add up to depth-1 (actually size-1)
+              int depth2 = depth - 1 - depth1;
+              for (ChildDerivationsGroup group : getFilteredDerivations(rule, floatingCell(rhs1, depth1), floatingCell(rhs2, depth2)))
                 for (Derivation deriv1 : group.derivations1)
                   for (Derivation deriv2 : group.derivations2)
                     if (!applyFloatingRule(rule, depth, deriv1, deriv2, deriv1.canonicalUtterance + " " + deriv2.canonicalUtterance))
                       break derivLoop;
             }
+        } else {
+          {
+            derivLoop:
+              for (int subDepth = 0; subDepth < depth; subDepth++) {  // depth-1 <=depth-1
+                for (ChildDerivationsGroup group : getFilteredDerivations(rule, floatingCell(rhs1, depth - 1), floatingCell(rhs2, subDepth)))
+                  for (Derivation deriv1 : group.derivations1)
+                    for (Derivation deriv2 : group.derivations2)
+                      if (!applyFloatingRule(rule, depth, deriv1, deriv2, deriv1.canonicalUtterance + " " + deriv2.canonicalUtterance))
+                        break derivLoop;
+              }
+          }
+          {
+            derivLoop:
+              for (int subDepth = 0; subDepth < depth - 1; subDepth++) {  // <depth-1 depth-1
+                for (ChildDerivationsGroup group : getFilteredDerivations(rule, floatingCell(rhs1, subDepth), floatingCell(rhs2, depth - 1)))
+                  for (Derivation deriv1 : group.derivations1)
+                    for (Derivation deriv2 : group.derivations2)
+                      if (!applyFloatingRule(rule, depth, deriv1, deriv2, deriv1.canonicalUtterance + " " + deriv2.canonicalUtterance))
+                        break derivLoop;
+              }
+          }
         }
       }
       ruleTime.put(rule, ruleTime.getOrDefault(rule, 0L) + stopWatch.stop().ms);
@@ -438,70 +469,107 @@ class FloatingParserState extends ParserState {
       derivations.addAll(myDerivations);
   }
 
+  /**
+   * Build derivations in a thread to allow timeout.
+   */
+  class DerivationBuilder implements Runnable {
+    @Override public void run() {
+      // Base case ($TOKEN, $PHRASE)
+      for (Derivation deriv : gatherTokenAndPhraseDerivations()) {
+        addToChart(anchoredCell(deriv.cat, deriv.start, deriv.end), deriv);
+        addToChart(floatingCell(deriv.cat, 0), deriv);
+      }
+
+      Set<String> categories = new HashSet<>();
+      for (Rule rule : parser.grammar.rules)
+        categories.add(rule.lhs);
+
+      if (Parser.opts.verbose >= 1)
+        LogInfo.begin_track_printAll("Anchored");
+      // Build up anchored derivations (like the BeamParser)
+      int numTokens = ex.numTokens();
+      for (int len = 1; len <= numTokens; len++) {
+        for (int i = 0; i + len <= numTokens; i++)  {
+          buildAnchored(i, i + len);
+          for (String cat : categories) {
+            String cell = anchoredCell(cat, i, i + len).toString();
+            pruneCell(cell, chart.get(cell));
+          }
+        }
+      }
+      if (Parser.opts.verbose >= 1)
+        LogInfo.end_track();
+
+      // Build up floating derivations
+      for (int depth = (FloatingParser.opts.initialFloatingHasZeroDepth ? 0 : 1); depth <= FloatingParser.opts.maxDepth; depth++) {
+        if (Parser.opts.verbose >= 1)
+          LogInfo.begin_track_printAll("%s = %d", FloatingParser.opts.useSizeInsteadOfDepth ? "SIZE" : "DEPTH", depth);
+        buildFloating(depth);
+        for (String cat : categories) {
+          String cell = floatingCell(cat, depth).toString();
+          pruneCell(cell, chart.get(cell));
+        }
+        if (Parser.opts.verbose >= 1)
+          LogInfo.end_track();
+        // Early stopping
+        if (computeExpectedCounts && ((FloatingParser) parser).earlyStopOnConsistent) {
+          // Consistent derivation found?
+          String cell = floatingCell(Rule.rootCat, depth).toString();
+          List<Derivation> rootDerivs = chart.get(cell);
+          if (rootDerivs != null) {
+            for (Derivation rootDeriv : rootDerivs) {
+              rootDeriv.ensureExecuted(parser.executor, ex.context);
+              if (parser.valueEvaluator.getCompatibility(ex.targetValue, rootDeriv.value) == 1) {
+                LogInfo.logs("Early stopped: consistent derivation found at depth = %d", depth);
+                return;
+              }
+            }
+          }
+        }
+        if (((FloatingParser) parser).earlyStopOnNumDerivs > 0) {
+          // Too many derivations generated?
+          if (numOfFeaturizedDerivs > ((FloatingParser) parser).earlyStopOnNumDerivs) {
+            LogInfo.logs("Early stopped: number of derivations exceeded at depth = %d", depth);
+            return;
+          }
+        }
+      }
+    }
+  }
+
+  public void buildDerivations() {
+    DerivationBuilder derivBuilder = new DerivationBuilder();
+    if (FloatingParser.opts.maxFloatingParsingTime == Integer.MAX_VALUE) {
+      derivBuilder.run();
+    } else {
+      Thread parsingThread = new Thread(derivBuilder);
+      parsingThread.start();
+      try {
+        parsingThread.join(FloatingParser.opts.maxFloatingParsingTime * 1000);
+        if (parsingThread.isAlive()) {
+          // This will only interrupt first or second passes, not the final candidate collection.
+          LogInfo.warnings("Parsing time exceeded %d seconds. Will now interrupt ...", FloatingParser.opts.maxFloatingParsingTime);
+          timeout = true;
+          parsingThread.interrupt();
+          parsingThread.join();
+        }
+      } catch (InterruptedException e) {
+        e.printStackTrace();
+        LogInfo.fails("FloatingParser error: %s", e);
+      }
+    }
+    evaluation.add("timeout", timeout);
+  }
+
+  // ============================================================
+  // Main entry point
+  // ============================================================
+
   @Override public void infer() {
     LogInfo.begin_track_printAll("FloatingParser.infer()");
     ruleTime = new HashMap<>();
 
-    // Base case ($TOKEN, $PHRASE)
-    for (Derivation deriv : gatherTokenAndPhraseDerivations()) {
-      addToChart(anchoredCell(deriv.cat, deriv.start, deriv.end), deriv);
-      addToChart(floatingCell(deriv.cat, 0), deriv);
-    }
-
-    Set<String> categories = new HashSet<>();
-    for (Rule rule : parser.grammar.rules)
-      categories.add(rule.lhs);
-
-    if (Parser.opts.verbose >= 1)
-      LogInfo.begin_track_printAll("Anchored");
-    // Build up anchored derivations (like the BeamParser)
-    int numTokens = ex.numTokens();
-    for (int len = 1; len <= numTokens; len++) {
-      for (int i = 0; i + len <= numTokens; i++)  {
-        buildAnchored(i, i + len);
-        for (String cat : categories) {
-          String cell = anchoredCell(cat, i, i + len).toString();
-          pruneCell(cell, chart.get(cell));
-        }
-      }
-    }
-    if (Parser.opts.verbose >= 1)
-      LogInfo.end_track();
-
-    // Build up floating derivations
-    // Timeout if taking too long
-    timeout = false;
-    Thread parsingThread = new Thread(new Runnable() {
-      @Override
-      public void run() {
-        for (int depth = (FloatingParser.opts.initialFloatingHasZeroDepth ? 0 : 1); depth <= FloatingParser.opts.maxDepth; depth++) {
-          if (Parser.opts.verbose >= 1)
-            LogInfo.begin_track_printAll("%s = %d", FloatingParser.opts.useSizeInsteadOfDepth ? "SIZE" : "DEPTH", depth);
-          buildFloating(depth);
-          for (String cat : categories) {
-            String cell = floatingCell(cat, depth).toString();
-            pruneCell(cell, chart.get(cell));
-          }
-          if (Parser.opts.verbose >= 1)
-            LogInfo.end_track();
-        }
-      }
-    });
-    parsingThread.start();
-    try {
-      parsingThread.join(FloatingParser.opts.maxFloatingParsingTime * 1000);
-      if (parsingThread.isAlive()) {
-        // This will only interrupt first or second passes, not the final candidate collection.
-        LogInfo.warnings("Parsing time exceeded %d seconds. Will now interrupt ...", FloatingParser.opts.maxFloatingParsingTime);
-        timeout = true;
-        parsingThread.interrupt();
-        parsingThread.join();
-      }
-    } catch (InterruptedException e) {
-      e.printStackTrace();
-      LogInfo.fails("DPParser error: %s", e);
-    }
-    evaluation.add("timeout", timeout);
+    buildDerivations();
 
     if (FloatingParser.opts.summarizeRuleTime) summarizeRuleTime();
 

src/edu/stanford/nlp/sempre/Formulas.java

@@ -74,6 +74,16 @@ public abstract class Formulas {
       if (mode != null)
         return new ArithmeticFormula(mode, fromLispTree(tree.child(1)), fromLispTree(tree.child(2)));
     }
+    
+    { // ActionFormula
+      ActionFormula.Mode mode = ActionFormula.parseMode(func);
+      if (mode != null) {
+        List<Formula> args = Lists.newArrayList();
+        for (int i = 1; i < tree.children.size(); i++)
+          args.add(fromLispTree(tree.child(i)));
+        return new ActionFormula(mode, args);
+      }
+    }
 
     // Default is join: (fb:type.object.type fb:people.person)
     if (tree.children.size() != 2)
@@ -301,7 +311,7 @@ public abstract class Formulas {
     // recurse
     if (f instanceof AggregateFormula) {
       AggregateFormula aggFormula = (AggregateFormula) f;
-      extractSubpartsRecursive(aggFormula, res);
+      extractSubpartsRecursive(aggFormula.child, res);
     } else if (f instanceof CallFormula) {
       CallFormula callFormula = (CallFormula) f;
       extractSubpartsRecursive(callFormula.func, res);

src/edu/stanford/nlp/sempre/FuzzyMatchFn.java

@@ -39,6 +39,9 @@ public class FuzzyMatchFn extends SemanticFn {
     }
   }
 
+  public FuzzyMatchFnMode getMode() { return mode; }
+  public boolean getMatchAny() { return matchAny; }
+
   @Override
   public DerivationStream call(Example ex, Callable c) {
     return new LazyFuzzyMatchFnDerivs(ex, c, mode, matchAny);

src/edu/stanford/nlp/sempre/Grammar.java

@@ -35,6 +35,8 @@ public class Grammar {
     @Option(gloss = "Variables which are used to interpret the grammar file")
     public List<String> tags = new ArrayList<>();
     @Option public boolean binarizeRules = true;
+    @Option(gloss = "Specifiy which ApplyFn to use: defaults to JoinFn when null")
+    public String useApplyFn = null;
   }
 
   public static Options opts = new Options();
@@ -42,7 +44,7 @@ public class Grammar {
   // All the rules in the grammar.  Each parser can read these and transform
   // them however the parser wishes.
   // This contains binarized rules
-  ArrayList<Rule> rules = new ArrayList<>();
+  protected ArrayList<Rule> rules = new ArrayList<>();
   public List<Rule> getRules() { return rules; }
 
   Map<String, LispTree> macros = new HashMap<>();  // Map from macro name to its replacement value
@@ -258,7 +260,7 @@ public class Grammar {
     return cat;
   }
 
-  private void interpretRule(LispTree tree) {
+  protected void interpretRule(LispTree tree) {
     if (tree.children.size() < 4)
       throw new RuntimeException("Invalid rule: " + tree);
 
@@ -358,7 +360,7 @@ public class Grammar {
 
   // Generate intermediate categories for binarization.
   public static final String INTERMEDIATE_PREFIX = "$Intermediate";
-  private int freshCatIndex = 0;
+  protected int freshCatIndex = 0;
   private String generateFreshCat() {
     freshCatIndex++;
     return INTERMEDIATE_PREFIX + freshCatIndex;
@@ -366,6 +368,9 @@ public class Grammar {
   public static boolean isIntermediate(String cat) {
     return cat.startsWith(INTERMEDIATE_PREFIX);
   }
+  public int getFreshCatIndex() {
+    return freshCatIndex;
+  }
 
   // Create multiple versions of this rule if there are optional RHS.
   // Restriction: must be able to split the RHS into two halves, each of
@@ -524,13 +529,20 @@ public class Grammar {
 
     String name = tree.child(0).value;
 
-    // Syntactic sugar: (lambda x (var x)) => (JoinFn betaReduce forward (arg0 (lambda x (var x))))
+    // Syntactic sugar: (lambda x (f (var x))) => (useApplyFn (lambda x (f (var x))))
+    // defaults to (lambda x (var x)) => (JoinFn betaReduce forward (arg0 (lambda x (var x))))
     if (name.equals("lambda")) {
       LispTree newTree = LispTree.proto.newList();
-      newTree.addChild("JoinFn");
-      newTree.addChild("betaReduce");
-      newTree.addChild("forward");
-      newTree.addChild(LispTree.proto.newList("arg0", tree));
+
+      if (Grammar.opts.useApplyFn == null) {
+        newTree.addChild("JoinFn");
+        newTree.addChild("betaReduce");
+        newTree.addChild("forward");
+        newTree.addChild(LispTree.proto.newList("arg0", tree));
+      } else {
+        newTree.addChild(Grammar.opts.useApplyFn);
+        newTree.addChild(tree);
+      }
       tree = newTree;
       name = tree.child(0).value;
     }

src/edu/stanford/nlp/sempre/JavaExecutor.java

@@ -1,5 +1,6 @@
 package edu.stanford.nlp.sempre;
 
+import com.google.common.base.Strings;
 import com.google.common.collect.Lists;
 import com.google.common.collect.Maps;
 
@@ -25,7 +26,10 @@ public class JavaExecutor extends Executor {
     @Option(gloss = "Whether to convert NumberValue to int/double") public boolean convertNumberValues = true;
     @Option(gloss = "Print stack trace on exception") public boolean printStackTrace = false;
     // the actual function will be called with the current ContextValue as its last argument if marked by contextPrefix
-    @Option(gloss = "Formula in the grammar whose name startsWith @ is context sensitive")  public String contextPrefix = "@";
+    @Option(gloss = "Formula in the grammar whose name startsWith contextPrefix is context sensitive") 
+    public String contextPrefix = "context:";
+    @Option(gloss = "Reduce verbosity by automatically appending, for example, edu.stanford.nlp.sempre to java calls")
+    public String classPathPrefix = ""; // e.g. "edu.stanford.nlp.sempre";
   }
   public static Options opts = new Options();
 
@@ -94,6 +98,30 @@ public class JavaExecutor extends Executor {
     public static String plus(String a, String b, String c, String d, String e, String f, String g) {
       return a + b + c + d + e + f + g;
     }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h) {
+      return a + b + c + d + e + f + g + h;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i) {
+      return a + b + c + d + e + f + g + h + i;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i, String j) {
+      return a + b + c + d + e + f + g + h + i + j;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i, String j, String k) {
+      return a + b + c + d + e + f + g + h + i + j + k;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i, String j, String k, String l) {
+      return a + b + c + d + e + f + g + h + i + j + k + l;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i, String j, String k, String l, String m) {
+      return a + b + c + d + e + f + g + h + i + j + k + l + m;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i, String j, String k, String l, String m, String n) {
+      return a + b + c + d + e + f + g + h + i + j + k + l + m + n;
+    }
+    public static String plus(String a, String b, String c, String d, String e, String f, String g, String h, String i, String j, String k, String l, String m, String n, String o) {
+      return a + b + c + d + e + f + g + h + i + j + k + l + m + n + o;
+    }
     private static String toString(Object x) {
       if (x instanceof String)
         return (String) x;
@@ -189,6 +217,11 @@ public class JavaExecutor extends Executor {
         id = id.replace(opts.contextPrefix, "");
       }
       id = MapUtils.get(shortcuts, id, id);
+      
+      // classPathPrefix, like edu.stanford.nlp.sempre.interactive
+      if (!Strings.isNullOrEmpty(opts.classPathPrefix) && !id.startsWith(".") && !id.startsWith(opts.classPathPrefix)) {
+        id = opts.classPathPrefix + "." + id;
+      }
 
       if (id.startsWith(".")) // Instance method
         return invoke(id.substring(1), args.get(0), args.subList(1, args.size()).toArray(new Object[0]));

src/edu/stanford/nlp/sempre/JoinFn.java

@@ -23,10 +23,10 @@ public class JoinFn extends SemanticFn {
   public static class Options {
     @Option(gloss = "Verbose") public int verbose = 0;
     @Option public boolean showTypeCheckFailures = false;
-    @Option public boolean typeInference = false;
+    @Option public boolean typeInference = true;
     // TODO(joberant): this flag is for backward compatibility.  If we don't
     // need it for the new results, get rid of it.
-    @Option public boolean specializedTypeCheck = true;
+    @Option public boolean specializedTypeCheck = false;
   }
 
   public static Options opts = new Options();

src/edu/stanford/nlp/sempre/KnowledgeGraph.java

@@ -60,6 +60,7 @@ public abstract class KnowledgeGraph {
   // ============================================================
 
   public abstract LispTree toLispTree();
+  public abstract LispTree toShortLispTree();
   @Override public String toString() { return toLispTree().toString(); }
 
   /** Return all y such that x in firsts and (x,r,y) in graph */

src/edu/stanford/nlp/sempre/Learner.java

@@ -27,6 +27,8 @@ public class Learner {
 
     @Option(gloss = "Write predDerivations to examples file (huge)")
     public boolean outputPredDerivations = false;
+    @Option(gloss = "Write predicted values to a TSV file")
+    public boolean outputPredValues = false;
 
     @Option(gloss = "Dump all features and compatibility scores")
     public boolean dumpFeaturesAndCompatibility = false;
@@ -149,6 +151,14 @@ public class Learner {
     LogInfo.end_track();
   }
 
+  public void onlineLearnExampleByFormula(Example ex, List<Formula> formulas) {
+    HashMap<String, Double> counts = new HashMap<>();
+    for (Derivation deriv : ex.predDerivations)
+      deriv.compatibility = formulas.contains(deriv.formula)? 1 : 0;
+    ParserState.computeExpectedCounts(ex.predDerivations, counts);
+    params.update(counts);
+  }
+
   private Evaluation processExamples(int iter, String group,
       List<Example> examples, boolean computeExpectedCounts) {
     Evaluation evaluation = new Evaluation();
@@ -214,8 +224,11 @@ public class Learner {
           addFeedback(ex);
 
         // Write out examples and predictions
-        if (opts.outputPredDerivations && Builder.opts.parser.equals("FloatingParser")) {
-          ExampleUtils.writeParaphraseSDF(iter, group, ex, opts.outputPredDerivations);
+        if (opts.outputPredDerivations) {
+          ExampleUtils.writeParaphraseSDF(iter, group, ex, true);
+        }
+        if (opts.outputPredValues) {
+          ExampleUtils.writePredictionTSV(iter, group, ex);
         }
 
         // To save memory
@@ -299,6 +312,7 @@ public class Learner {
     // evaluation.add(LexiconFn.lexEval);
     evaluation.logStats(prefix);
     evaluation.putOutput(prefix);
+    evaluation.putOutput(prefix.replaceAll("iter=", "").replace('.', '_'));
   }
 
   private void printLearnerEventsIter(Example ex, int iter, String group) {

src/edu/stanford/nlp/sempre/Main.java

@@ -1,5 +1,8 @@
 package edu.stanford.nlp.sempre;
 
+import java.lang.reflect.Constructor;
+
+
 import fig.basic.Option;
 import fig.exec.Execution;
 
@@ -11,6 +14,7 @@ import fig.exec.Execution;
 public class Main implements Runnable {
   @Option public boolean interactive = false;
   @Option public boolean server = false;
+  @Option public String masterType = "edu.stanford.nlp.sempre.Master";
 
   public void run() {
     Builder builder = new Builder();
@@ -22,16 +26,24 @@ public class Main implements Runnable {
     Learner learner = new Learner(builder.parser, builder.params, dataset);
     learner.learn();
 
-    if (server) {
-      Master master = new Master(builder);
-      Server server = new Server(master);
-      server.run();
+    if (server || interactive) {
+      Master master = createMaster(masterType, builder);
+      if (server)
+        master.runServer();
+      if (interactive)
+        master.runInteractivePrompt();
     }
+  }
 
-    if (interactive) {
-      Master master = new Master(builder);
-      master.runInteractivePrompt();
+  public Master createMaster(String masterType, Builder builder) {
+    try {
+      Class<?> masterClass = Class.forName(masterType);
+      Constructor<?> constructor = masterClass.getConstructor(Builder.class);
+      return (Master)constructor.newInstance(builder);
+    } catch (Throwable t) {
+      t.printStackTrace();
     }
+    return null;
   }
 
   public static void main(String[] args) {

src/edu/stanford/nlp/sempre/Master.java

@@ -4,6 +4,7 @@ import com.google.common.base.Joiner;
 import com.google.common.base.Strings;
 import com.google.common.collect.Lists;
 import fig.basic.*;
+import jline.console.ConsoleReader;
 
 import java.io.IOException;
 import java.io.PrintWriter;
@@ -40,22 +41,24 @@ public class Master {
     @Option(gloss = "Write out new grammar rules")
     public String newGrammarPath;
   }
-
   public static Options opts = new Options();
-
+  
   public class Response {
     // Example that was parsed, if any.
-    Example ex;
+    public Example ex;
 
     // Which derivation we're selecting to show
     int candidateIndex = -1;
 
     // Detailed information
-    List<String> lines = new ArrayList<>();
+    public Map<String, Object> stats = new LinkedHashMap<>();
+    public List<String> lines = new ArrayList<>();
 
     public String getFormulaAnswer() {
       if (ex.getPredDerivations().size() == 0)
         return "(no answer)";
+      else if (candidateIndex == -1)
+        return "(not selected)";
       else {
         Derivation deriv = getDerivation();
         return deriv.getFormula() + " => " + deriv.getValue();
@@ -64,6 +67,8 @@ public class Master {
     public String getAnswer() {
       if (ex.getPredDerivations().size() == 0)
         return "(no answer)";
+      else if (candidateIndex == -1)
+        return "(not selected)";
       else {
         Derivation deriv = getDerivation();
         deriv.ensureExecuted(builder.executor, ex.context);
@@ -79,9 +84,9 @@ public class Master {
     }
   }
 
-  private Builder builder;
-  private Learner learner;
-  private HashMap<String, Session> sessions = new LinkedHashMap<>();
+  protected Builder builder;
+  protected Learner learner;
+  protected HashMap<String, Session> sessions = new LinkedHashMap<>();
 
   public Master(Builder builder) {
     this.builder = builder;
@@ -106,7 +111,7 @@ public class Master {
     return session;
   }
 
-  void printHelp() {
+  protected void printHelp() {
     LogInfo.log("Enter an utterance to parse or one of the following commands:");
     LogInfo.log("  (help): show this help message");
     LogInfo.log("  (status): prints out status of the system");
@@ -126,31 +131,32 @@ public class Master {
     LogInfo.log("Press Ctrl-D to exit.");
   }
 
+  public void runServer() {
+    Server server = new Server(this);
+    server.run();;
+  }
+  
   public void runInteractivePrompt() {
     Session session = getSession("stdin");
 
     if (opts.printHelp)
       printHelp();
-
-    while (true) {
-      LogInfo.stdout.print("> ");
-      LogInfo.stdout.flush();
+    try {
+      ConsoleReader reader = new ConsoleReader();
+      reader.setPrompt("> ");
       String line;
-      try {
-        line = LogInfo.stdin.readLine();
-      } catch (IOException e) {
-        throw new RuntimeException(e);
-      }
-      if (line == null) break;
-
-      int indent = LogInfo.getIndLevel();
-      try {
-        processQuery(session, line);
-      } catch (Throwable t) {
-        while (LogInfo.getIndLevel() > indent)
-          LogInfo.end_track();
-        t.printStackTrace();
+      while ((line = reader.readLine()) != null) {
+        int indent = LogInfo.getIndLevel();
+        try {
+          processQuery(session, line);
+        } catch (Throwable t) {
+          while (LogInfo.getIndLevel() > indent)
+            LogInfo.end_track();
+          t.printStackTrace();
+        }
       }
+    } catch (IOException e) {
+      throw new RuntimeException(e);
     }
   }
 
@@ -229,7 +235,7 @@ public class Master {
     builder.parser.parse(builder.params, ex, false);
 
     response.ex = ex;
-    ex.log();
+    ex.logWithoutContext();
     if (ex.predDerivations.size() > 0) {
       response.candidateIndex = 0;
       printDerivation(response.getDerivation());
@@ -377,7 +383,14 @@ public class Master {
       } else {
         session.context = new ContextValue(tree);
       }
-    } else {
+    } else if (command.equals("loadgraph")) {
+      if (tree.children.size() != 2 || !tree.child(1).isLeaf())
+        throw new RuntimeException("Invalid argument: argument should be a file path");
+      KnowledgeGraph graph = NaiveKnowledgeGraph.fromFile(tree.child(1).value);
+      session.context = new ContextValue(session.context.user, session.context.date,
+        session.context.exchanges, graph);
+    }
+    else {
       LogInfo.log("Invalid command: " + tree);
     }
   }

src/edu/stanford/nlp/sempre/NaiveKnowledgeGraph.java

@@ -170,6 +170,10 @@ public class NaiveKnowledgeGraph extends KnowledgeGraph {
     return new NaiveKnowledgeGraph(triples);
   }
 
+  public static KnowledgeGraph fromFile(String path) {
+    return fromLispTree(LispTree.proto.parseFromFile(path).next());
+  }
+
   @Override
   public LispTree toLispTree() {
     LispTree tree = LispTree.proto.newList();
@@ -180,4 +184,16 @@ public class NaiveKnowledgeGraph extends KnowledgeGraph {
     }
     return tree;
   }
+
+  @Override
+  public LispTree toShortLispTree() {
+    if (triples.size() > 1000) {
+      LispTree tree = LispTree.proto.newList();
+      tree.addChild("graph");
+      tree.addChild("NaiveKnowledgeGraph");
+      tree.addChild(("TooManyTriples"));
+      return tree;
+    }
+    return toLispTree();
+  }
 }

src/edu/stanford/nlp/sempre/NameValue.java

@@ -42,6 +42,7 @@ public class NameValue extends Value {
   }
 
   @Override public String sortString() { return id; }
+  @Override public String pureString() { return description == null ? id : description; }
 
   @Override public int hashCode() { return id.hashCode(); }
   @Override public boolean equals(Object o) {

src/edu/stanford/nlp/sempre/NumberFn.java

@@ -2,6 +2,7 @@ package edu.stanford.nlp.sempre;
 
 import java.util.ArrayList;
 import java.util.List;
+
 import fig.basic.*;
 
 /**
@@ -16,6 +17,8 @@ public class NumberFn extends SemanticFn {
     public boolean alsoTestByConversion = false;
     @Option(gloss = "Also test numbers by applying NER on just the phrase")
     public boolean alsoTestByIsolatedNER = false;
+    @Option(gloss = "range of allowed numbers. e.g. null: no limits, Lists.newArrayList(0,100): 0-100 inclusive")
+    public List<Double> allowedRange = null;
   }
   public static Options opts = new Options();
 
@@ -79,6 +82,13 @@ public class NumberFn extends SemanticFn {
           if (value != null) {
             try {
               NumberValue numberValue = new NumberValue(Double.parseDouble(value));
+              if (opts.allowedRange != null) {
+                if (numberValue.value < opts.allowedRange.get(0) || numberValue.value > opts.allowedRange.get(1)) {
+                  LogInfo.warnings("NumberFn: %f is outside of the allowed range %s", numberValue.value, opts.allowedRange);
+                  return null;
+                }
+              }
+
               SemType type = numberValue.value == (int) numberValue.value ? SemType.intType : SemType.floatType;
               return new Derivation.Builder()
                       .withCallable(c)

src/edu/stanford/nlp/sempre/NumberValue.java

@@ -40,6 +40,7 @@ public class NumberValue extends Value {
   }
 
   @Override public String sortString() { return "" + value; }
+  @Override public String pureString() { return "" + value; }
 
   @Override public int hashCode() { return Double.valueOf(value).hashCode(); }
   @Override public boolean equals(Object o) {

src/edu/stanford/nlp/sempre/Parser.java

@@ -56,6 +56,9 @@ public abstract class Parser {
 
     @Option(gloss = "Dump all features (for debugging)")
     public boolean dumpAllFeatures = false;
+
+    @Option(gloss = "Call SetEvaluation during parsing")
+    public boolean callSetEvaluation = true;
   }
 
   public static final Options opts = new Options();
@@ -88,7 +91,7 @@ public abstract class Parser {
   public List<Rule> getCatUnaryRules() { return catUnaryRules; }
 
   // TODO(joberant): move this to a separate class in charge of visualizing charts
-  PrintWriter chartFillOut = null;  // For printing a machine-readable json file
+  public PrintWriter chartFillOut = null;  // For printing a machine-readable json file
 
   public Parser(Spec spec) {
     this.grammar = spec.grammar;
@@ -97,8 +100,8 @@ public abstract class Parser {
     this.valueEvaluator = spec.valueEvaluator;
 
     computeCatUnaryRules();
-    LogInfo.logs("Parser: %d catUnaryRules (sorted), %d nonCatUnaryRules (in trie)",
-        catUnaryRules.size(), grammar.rules.size() - catUnaryRules.size());
+    LogInfo.logs("%s: %d catUnaryRules (sorted), %d nonCatUnaryRules (in trie)",
+        this.getClass().getSimpleName(), catUnaryRules.size(), grammar.rules.size() - catUnaryRules.size());
   }
 
   // If grammar changes, then we might need to update aspects of the parser.
@@ -174,9 +177,10 @@ public abstract class Parser {
     Derivation.sortByScore(ex.predDerivations);
 
     // Evaluate
-    ex.evaluation = new Evaluation();
-    addToEvaluation(state, ex.evaluation);
-
+    if (opts.callSetEvaluation) {
+      ex.evaluation = new Evaluation();
+      addToEvaluation(state, ex.evaluation);
+    }
     // Clean up temporary state used during parsing
     ex.clearTempState();
     for (Derivation deriv : ex.predDerivations)
@@ -206,6 +210,7 @@ public abstract class Parser {
     int correctIndexAfterParse = -1;
     double maxCompatibility = 0.0;
     double[] compatibilities = null;
+    int numCorrect = 0, numPartialCorrect = 0, numIncorrect = 0;
 
     if (ex.targetValue != null) {
       compatibilities = new double[numCandidates];
@@ -217,6 +222,14 @@ public abstract class Parser {
           correctIndex = i;
         // record maximum compatibility for partial oracle
         maxCompatibility = Math.max(compatibilities[i], maxCompatibility);
+        // Count
+        if (compatibilities[i] == 1) {
+          numCorrect++;
+        } else if (compatibilities[i] == 0) {
+          numIncorrect++;
+        } else {
+          numPartialCorrect++;
+        }
       }
       // What if we only had parsed bottom up?
       for (int i = 0; i < numCandidates; i++) {
@@ -330,6 +343,9 @@ public abstract class Parser {
     evaluation.add("numCandidates", numCandidates);  // From this parse
     if (numCandidates > 0)
       evaluation.add("parsedNumCandidates", numCandidates);
+    evaluation.add("numCorrect", numCorrect);
+    evaluation.add("numPartialCorrect", numPartialCorrect);
+    evaluation.add("numIncorrect", numIncorrect);
 
     // Add parsing stats
     evaluation.add(state.evaluation);

src/edu/stanford/nlp/sempre/ParserState.java

@@ -24,7 +24,7 @@ public abstract class ParserState {
   }
   public static Options opts = new Options();
 
-  public enum CustomExpectedCount { NONE, UNIFORM, TOP, RANDOM, }
+  public enum CustomExpectedCount { NONE, UNIFORM, TOP, TOPALT, RANDOM, }
 
   //// Input: specification of how to parse
 
@@ -137,8 +137,8 @@ public abstract class ParserState {
     if (Parser.opts.verbose >= 3) {
       LogInfo.begin_track("ParserState.pruneCell(%s): %d derivations", cellDescription, derivations.size());
       for (Derivation deriv : derivations) {
-        LogInfo.logs("%s(%s,%s): %s %s, [score=%s]", deriv.cat, deriv.start, deriv.end, deriv.formula,
-            deriv.canonicalUtterance, deriv.score);
+        LogInfo.logs("%s(%s,%s): %s %s, [score=%s] allAnchored: %s", deriv.cat, deriv.start, deriv.end, deriv.formula,
+            deriv.canonicalUtterance, deriv.score, deriv.allAnchored());
       }
       LogInfo.end_track();
     }
@@ -230,7 +230,7 @@ public abstract class ParserState {
   }
 
   // Ensure that all the logical forms are executed and compatibilities are computed.
-  protected void ensureExecuted() {
+  public void ensureExecuted() {
     LogInfo.begin_track("Parser.ensureExecuted");
     // Execute predicted derivations to get value.
     for (Derivation deriv : predDerivations) {
@@ -275,7 +275,7 @@ public abstract class ParserState {
     predScores = new double[n];
     // For update schemas that choose one good and one bad candidate to update
     int[] goodAndBad = null;
-    if (opts.customExpectedCounts == CustomExpectedCount.TOP) {
+    if (opts.customExpectedCounts == CustomExpectedCount.TOP || opts.customExpectedCounts == CustomExpectedCount.TOPALT) {
       goodAndBad = getTopDerivations(derivations);
       if (goodAndBad == null) return;
     } else if (opts.customExpectedCounts == CustomExpectedCount.RANDOM) {
@@ -300,6 +300,10 @@ public abstract class ParserState {
           trueScores[i] = (i == goodAndBad[0]) ? 0 : Double.NEGATIVE_INFINITY;
           predScores[i] = (i == goodAndBad[1]) ? 0 : Double.NEGATIVE_INFINITY;
           break;
+        case TOPALT:
+          trueScores[i] = (i == goodAndBad[0]) ? 0 : Double.NEGATIVE_INFINITY;
+          predScores[i] = (i == goodAndBad[0] || i == goodAndBad[1]) ? deriv.score : Double.NEGATIVE_INFINITY;
+          break;
         default:
           throw new RuntimeException("Unknown customExpectedCounts: " + opts.customExpectedCounts);
       }

src/edu/stanford/nlp/sempre/Rule.java

@@ -5,7 +5,9 @@ import com.google.common.collect.Lists;
 import fig.basic.LispTree;
 import fig.basic.Pair;
 
+import java.util.LinkedHashMap;
 import java.util.List;
+import java.util.Map;
 
 /**
  * A rule specifies how to take a right hand of terminals and non-terminals.
@@ -23,11 +25,12 @@ public class Rule {
   public static final String phraseCat = "$PHRASE";  // Sequence of tokens
   public static final String lemmaTokenCat = "$LEMMA_TOKEN";  // Lemmatized version
   public static final String lemmaPhraseCat = "$LEMMA_PHRASE";  // Lemmatized version
-
+  public static final List<String> specialCats = Lists.newArrayList(rootCat, tokenCat, phraseCat, lemmaTokenCat, lemmaPhraseCat);
   public final String lhs;  // Left-hand side: category.
   public final List<String> rhs;  // Right-hand side: sequence of categories (have $ prefix) and tokens.
   public final SemanticFn sem;  // Takes derivations corresponding to RHS categories and produces a set of derivations corresponding to LHS.
   public List<Pair<String, Double>> info;  // Extra info
+  public RuleSource source = null; // for tracking where the rule comes from when they are induced
 
   // Cache the semanticRepn
   public String getSemRepn() {
@@ -46,8 +49,13 @@ public class Rule {
 
   @Override
   public String toString() {
-    if (stringRepn == null)
-      stringRepn = lhs + " -> " + (rhs == null ? "" : Joiner.on(' ').join(rhs)) + " " + sem;
+    if (stringRepn == null) {
+      String semStr = sem == null? "NullSemanticFn" : sem.toString();
+      //int maxLength = 100;
+      //if (semStr.length() > maxLength)
+      //  semStr = String.format("%s...(%d total)", semStr.substring(0,maxLength), semStr.length());
+      stringRepn = lhs + " -> " + (rhs == null ? "" : Joiner.on(' ').join(rhs)) + " " + semStr;
+    }
     return stringRepn;
   }
   private String stringRepn;  // Cache toString()
@@ -95,6 +103,8 @@ public class Rule {
       for (Pair<String, Double> p : info)
         tree.addChild(LispTree.proto.newList(p.getFirst(), "" + p.getSecond()));
     }
+    if (source != null)
+      tree.addChild(source.toJson());
     return tree;
   }
 
@@ -129,4 +139,35 @@ public class Rule {
     else
       return f == 1.0 ? false : !FloatingParser.opts.defaultIsFloating;
   }
+
+  public boolean isInduced() {
+    double a = getInfoTag("induced");
+    if (a == 1.0) return true;
+    return false;
+  }
+
+  @Override
+  public boolean equals(Object o) {
+    if (!(o instanceof Rule)) return false;
+    return ((Rule)o).toString().equals(this.toString());
+  }
+  @Override
+  public int hashCode() {
+    return this.toString().hashCode();
+  }
+
+  public String toJson() {
+    Map<String, Object> jsonMap = new LinkedHashMap<>();
+    jsonMap.put("lhs", lhs);
+    jsonMap.put("rhs", rhs);
+    if (source != null) {
+      jsonMap.put("source", source);
+    }
+    if (info != null) {
+      for (Pair<String, Double> p : info)
+        jsonMap.put(p.getFirst(), p.getSecond());
+    }
+    jsonMap.put("sem", sem.toString());
+    return Json.writeValueAsStringHard(jsonMap);
+  }
 }

src/edu/stanford/nlp/sempre/RuleSource.java

@@ -0,0 +1,48 @@
+package edu.stanford.nlp.sempre;
+
+import java.time.LocalDateTime;
+import java.util.List;
+
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * Tracking where the rule comes from in the grammar induction process.
+ *
+ * @author sidaw
+ */
+
+@JsonIgnoreProperties(ignoreUnknown = true)
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class RuleSource {
+  @JsonProperty
+  public String uid;
+  @JsonProperty
+  public LocalDateTime time;
+  @JsonProperty
+  public String head;
+  @JsonProperty
+  public List<String> body;
+
+  @JsonProperty
+  public int cite = 0;
+  @JsonProperty
+  public int self = 0;
+  @JsonProperty
+  public boolean align = false;
+  @JsonProperty
+  public String alignInfo = "";
+
+  public RuleSource(String uid, String head, List<String> body) {
+    this.uid = uid;
+    this.head = head;
+    this.body = body;
+    this.time = LocalDateTime.now();
+  }
+
+  public String toJson() {
+    return Json.writeValueAsStringHard(this);
+  }
+
+}

src/edu/stanford/nlp/sempre/Session.java

@@ -2,6 +2,11 @@ package edu.stanford.nlp.sempre;
 
 import java.util.ArrayList;
 import java.util.List;
+import java.util.Map;
+
+import com.google.common.base.Strings;
+
+import fig.basic.Option;
 
 /**
  * A Session contains the information specific to a user.
@@ -13,11 +18,23 @@ import java.util.List;
  */
 public class Session {
   public final String id;  // Session id
-  String remoteHost;  // Where we connected from
-  String format;  // html or json
-  ContextValue context;  // Current context used to create new examples
+  public static class Options {
+    // path for default parameters, if using a different set for each session
+    @Option public String inParamsPath;
+  }
+  public String remoteHost;  // Where we connected from
+  public String format;  // html or json
+  public ContextValue context;  // Current context used to create new examples
   Example lastEx;  // Last example that we processed
-
+  
+  // if every user have their own model
+  Params params;
+  Learner learner;
+  public Map<String,String> reqParams;
+  
+  public static Options opts = new Options();
+  
+  // per session parameters
   public Session(String id) {
     this.id = id;
     context = new ContextValue(id, DateValue.now(), new ArrayList<ContextValue.Exchange>());
@@ -58,9 +75,29 @@ public class Session {
       newExchanges.add(context.exchanges.get(i));
     return context.withNewExchange(newExchanges);
   }
-
+  
+  public void useIndependentLearner(Builder builder) {
+    this.params = new Params();
+    if (!Strings.isNullOrEmpty(opts.inParamsPath))
+      this.params.read(opts.inParamsPath);
+    this.learner = new Learner(builder.parser, this.params, new Dataset());
+  }
+ 
   @Override
   public String toString() {
     return String.format("%s: %s; last: %s", id, context, lastEx);
   }
+  
+  // Decides if we write out any logs
+  public boolean isLogging() { return defaultTrue("logging");}
+  public boolean isWritingCitation() { return defaultTrue("cite");}
+  public boolean isWritingGrammar() { return defaultTrue("grammar");}
+  public boolean isLearning() { return defaultTrue("learn");}
+  public boolean isStatsing() { return defaultTrue("stats");}
+  
+  private boolean defaultTrue(String key) {
+    if (this.reqParams == null) return true;
+    if (!this.reqParams.containsKey(key)) return true;
+    return !this.reqParams.get(key).equals("0");
+  }
 }

src/edu/stanford/nlp/sempre/StringValue.java

@@ -20,6 +20,7 @@ public class StringValue extends Value {
   }
 
   @Override public String sortString() { return "\"" + value + "\""; }
+  @Override public String pureString() { return value; }
 
   @Override public int hashCode() { return value.hashCode(); }
   @Override public boolean equals(Object o) {

src/edu/stanford/nlp/sempre/TargetValuePreprocessor.java

@@ -27,10 +27,10 @@ public abstract class TargetValuePreprocessor {
   }
   public static void setSingleton(TargetValuePreprocessor processor) { singleton = processor; }
 
-  public abstract Value preprocess(Value value);
+  public abstract Value preprocess(Value value, Example ex);
 
 }
 
 class IdentityTargetValuePreprocessor extends TargetValuePreprocessor {
-  public Value preprocess(Value value) { return value; }
+  public Value preprocess(Value value, Example ex) { return value; }
 }

src/edu/stanford/nlp/sempre/Trie.java

@@ -1,8 +1,11 @@
 package edu.stanford.nlp.sempre;
 
 import java.util.ArrayList;
-import java.util.HashMap;
 import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.HashMap;
+import java.util.Map;
 import java.util.Set;
 
 /**
@@ -10,19 +13,20 @@ import java.util.Set;
  * @author Percy Liang
  */
 public class Trie {
-  ArrayList<Rule> rules = new ArrayList<>();
-  HashMap<String, Trie> children = new HashMap<>();
+  public ArrayList<Rule> rules = new ArrayList<>();
+  Map<String, Trie> children = new LinkedHashMap<>();
   // Set of LHS categories of all rules in this subtree
-  Set<String> cats = new HashSet<>();
+  public Set<String> cats = new LinkedHashSet<>();
 
-  Trie next(String item) { return children.get(item); }
+  public Trie next(String item) { return children.get(item); }
 
-  void add(Rule rule) { add(rule, 0); }
+  public void add(Rule rule) { add(rule, 0); }
   private void add(Rule rule, int i) {
     cats.add(rule.lhs);
 
     if (i == rule.rhs.size()) {
-      rules.add(rule);
+      if (!rules.contains(rule)) // filter exact match
+        rules.add(rule);
       return;
     }
 

src/edu/stanford/nlp/sempre/TypeInference.java

@@ -227,11 +227,11 @@ public final class TypeInference {
 
     } else if (formula instanceof AggregateFormula) {
       AggregateFormula aggregate = (AggregateFormula) formula;
-      inferType(aggregate.child, env, SemType.anyType);
+      SemType childType = inferType(aggregate.child, env, SemType.anyType);
       if (aggregate.mode == AggregateFormula.Mode.count)
         return check(SemType.numberType.meet(type));
       else
-        return check(SemType.numberOrDateType.meet(type));
+        return check(SemType.numberOrDateType.meet(type).meet(childType));
 
     } else if (formula instanceof ArithmeticFormula) {
       ArithmeticFormula arith = (ArithmeticFormula) formula;
@@ -270,6 +270,9 @@ public final class TypeInference {
       for (int i = 0; i < info.argTypes.size(); i++)
         inferType(call.args.get(i), env, info.argTypes.get(i));
       return check(type.meet(info.retType));
+    } else if (formula instanceof ActionFormula) {
+      initCallTypeInfo();
+      return SemType.anyType;
     } else {
       throw new RuntimeException("Can't infer type of formula: " + formula);
     }

src/edu/stanford/nlp/sempre/UriValue.java

@@ -20,6 +20,9 @@ public class UriValue extends Value {
     return tree;
   }
 
+  @Override public String sortString() { return "" + value; }
+  @Override public String pureString() { return "" + value; }
+
   @Override public int hashCode() { return value.hashCode(); }
   @Override public boolean equals(Object o) {
     if (this == o) return true;

src/edu/stanford/nlp/sempre/Value.java

@@ -24,6 +24,9 @@ public abstract class Value {
   // (optional) String used for sorting Values. The default is to call toString()
   public String sortString() { return toString(); }
 
+  // (optional) String without the LispTree structure. The default is to call toString()
+  public String pureString() { return toString(); }
+
   @JsonCreator
   public static Value fromString(String str) {
     return Values.fromLispTree(LispTree.proto.parseFromString(str));

src/edu/stanford/nlp/sempre/cprune/CPruneDerivInfo.java

@@ -0,0 +1,13 @@
+package edu.stanford.nlp.sempre.cprune;
+
+import java.util.Map;
+import java.util.List;
+
+public class CPruneDerivInfo {
+
+  public Map<String, Symbol> treeSymbols;
+  public Map<String, Symbol> ruleSymbols;
+  public List<String> customRuleStrings;
+  public boolean containsCrossReference;
+
+}

src/edu/stanford/nlp/sempre/cprune/CPruneFloatingParser.java

@@ -0,0 +1,119 @@
+package edu.stanford.nlp.sempre.cprune;
+
+import java.util.List;
+
+import edu.stanford.nlp.sempre.*;
+import fig.basic.LogInfo;
+
+/**
+ * A parser that first tries to exploit the macro grammar and only fall back to full search when needed.
+ */
+public class CPruneFloatingParser extends FloatingParser {
+
+  FloatingParser exploreParser;
+
+  public CPruneFloatingParser(Spec spec) {
+    super(spec);
+    exploreParser = new FloatingParser(spec).setEarlyStopping(true, CollaborativePruner.opts.maxDerivations);
+  }
+
+  @Override
+  public void onBeginDataGroup(int iter, int numIters, String group) {
+    if (CollaborativePruner.uidToCachedNeighbors == null) {
+      CollaborativePruner.customGrammar.init(grammar);
+      CollaborativePruner.loadNeighbors();
+    }
+    CollaborativePruner.stats.reset(iter + "." + group);
+  }
+
+  @Override
+  public ParserState newParserState(Params params, Example ex, boolean computeExpectedCounts) {
+    return new CPruneFloatingParserState(this, params, ex, computeExpectedCounts);
+  }
+
+}
+
+class CPruneFloatingParserState extends ParserState {
+
+  public CPruneFloatingParserState(Parser parser, Params params, Example ex, boolean computeExpectedCounts) {
+    super(parser, params, ex, computeExpectedCounts);
+  }
+
+  @Override
+  public void infer() {
+    LogInfo.begin_track("CPruneFloatingParser.infer()");
+    boolean exploitSucceeds = exploit();
+    if (computeExpectedCounts) {
+      LogInfo.begin_track("Summary of Collaborative Pruning");
+      LogInfo.logs("Exploit succeeds: " + exploitSucceeds);
+      LogInfo.logs("Exploit success rate: " + CollaborativePruner.stats.successfulExploit + "/" + CollaborativePruner.stats.totalExploit);
+      LogInfo.end_track();
+    }
+    // Explore only on the first training iteration
+    if (CollaborativePruner.stats.iter.equals("0.train") && computeExpectedCounts && !exploitSucceeds
+        && (CollaborativePruner.stats.totalExplore <= CollaborativePruner.opts.maxExplorationIters)) {
+      explore();
+      LogInfo.logs("Consistent pattern: " + CollaborativePruner.getConsistentPattern(ex));
+      LogInfo.logs("Explore success rate: " + CollaborativePruner.stats.successfulExplore + "/" + CollaborativePruner.stats.totalExplore);
+    }
+    LogInfo.end_track();
+  }
+
+  public void explore() {
+    LogInfo.begin_track("Explore");
+    CollaborativePruner.initialize(ex, CollaborativePruner.Mode.EXPLORE);
+    ParserState exploreParserState = ((CPruneFloatingParser) parser).exploreParser.newParserState(params, ex, computeExpectedCounts);
+    exploreParserState.infer();
+    predDerivations.clear();
+    predDerivations.addAll(exploreParserState.predDerivations);
+    expectedCounts = exploreParserState.expectedCounts;
+    if (computeExpectedCounts) {
+      for (Derivation deriv : predDerivations)
+        CollaborativePruner.updateConsistentPattern(parser.valueEvaluator, ex, deriv);
+    }
+    CollaborativePruner.stats.totalExplore += 1;
+    if (CollaborativePruner.foundConsistentDerivation)
+      CollaborativePruner.stats.successfulExplore += 1;
+    LogInfo.end_track();
+  }
+
+  public boolean exploit() {
+    LogInfo.begin_track("Exploit");
+    CollaborativePruner.initialize(ex, CollaborativePruner.Mode.EXPLOIT);
+    Grammar miniGrammar = new MiniGrammar(CollaborativePruner.predictedRules);
+    Parser exploitParser = new FloatingParser(new Parser.Spec(miniGrammar, parser.extractor, parser.executor, parser.valueEvaluator));
+    ParserState exploitParserState = exploitParser.newParserState(params, ex, computeExpectedCounts);
+    exploitParserState.infer();
+    predDerivations.clear();
+    predDerivations.addAll(exploitParserState.predDerivations);
+    expectedCounts = exploitParserState.expectedCounts;
+    if (computeExpectedCounts) {
+      for (Derivation deriv : predDerivations)
+        CollaborativePruner.updateConsistentPattern(parser.valueEvaluator, ex, deriv);
+    }
+    boolean succeeds = CollaborativePruner.foundConsistentDerivation;
+    CollaborativePruner.stats.totalExploit += 1;
+    if (succeeds)
+      CollaborativePruner.stats.successfulExploit += 1;
+    LogInfo.end_track();
+    return succeeds;
+  }
+}
+
+// ============================================================
+// Helper classes
+// ============================================================
+
+class MiniGrammar extends Grammar {
+
+  public MiniGrammar(List<Rule> rules) {
+    this.rules.addAll(rules);
+    if (CollaborativePruner.opts.verbose >= 2) {
+      LogInfo.begin_track("MiniGrammar Rules");
+      for (Rule rule : rules)
+        LogInfo.logs("%s %s", rule, rule.isAnchored() ? "[A]" : "[F]");
+      LogInfo.end_track();
+    }
+  }
+
+}

src/edu/stanford/nlp/sempre/cprune/CPruneStats.java

@@ -0,0 +1,20 @@
+package edu.stanford.nlp.sempre.cprune;
+
+/**
+ * Stores various statistic.
+ */
+public class CPruneStats {
+  public String iter;
+  public int totalExplore = 0;
+  public int successfulExplore = 0;
+  public int totalExploit = 0;
+  public int successfulExploit = 0;
+
+  public void reset(String iter) {
+    this.iter = iter;
+    this.totalExplore = 0;
+    this.successfulExplore = 0;
+    this.totalExploit = 0;
+    this.successfulExploit = 0;
+  }
+}

src/edu/stanford/nlp/sempre/cprune/CollaborativePruner.java

@@ -0,0 +1,187 @@
+package edu.stanford.nlp.sempre.cprune;
+
+import java.io.*;
+import java.util.*;
+
+import fig.basic.*;
+import edu.stanford.nlp.sempre.*;
+
+/**
+ * Static class for collaborative pruning.
+ */
+public class CollaborativePruner {
+  public static class Options {
+    @Option(gloss = "Logging verbosity")
+    public int verbose = 0;
+    @Option(gloss = "K = Maximum number of nearest-neighbor examples to consider (-1 to use all examples so far)")
+    public int maxNumNeighbors = -1;
+    @Option(gloss = "Load cached neighbors from this file")
+    public String neighborFilePath = null;
+    @Option(gloss = "Maximum number of matching patterns (default = use all patterns)")
+    public int maxPredictedPatterns = Integer.MAX_VALUE;
+    @Option(gloss = "Maximum number of derivations per example")
+    public int maxDerivations = 5000;
+    @Option(gloss = "Maximum number of times to fall back to exploration")
+    public int maxExplorationIters = Integer.MAX_VALUE;
+  }
+
+  public static Options opts = new Options();
+
+  public enum Mode { EXPLORE, EXPLOIT, NONE }
+
+  public static Mode mode = Mode.NONE;
+  public static CPruneStats stats = new CPruneStats();
+  public static CustomGrammar customGrammar = new CustomGrammar();
+
+  // Static class; do not instantiate
+  private CollaborativePruner() { throw new RuntimeException("Cannot instantiate CollaborativePruner"); }
+
+  // Global variables
+  // Nearest neighbors
+  static Map<String, List<String>> uidToCachedNeighbors;
+  // uid => pattern
+  static Map<String, FormulaPattern> consistentPattern = new HashMap<>();
+  // patternString => customRuleString
+  static Map<String, Set<String>> customRules = new HashMap<>();
+  // set of patternStrings
+  static Set<String> allConsistentPatterns = new HashSet<>();
+
+  // Example-level variables
+  public static boolean foundConsistentDerivation = false;
+  public static Map<String, FormulaPattern> predictedPatterns;
+  public static List<Rule> predictedRules;
+
+  /**
+   * Read the cached neighbors file.
+   * Line Format: ex_id [tab] neighbor_id1,neighbor_id2,...
+   */
+  public static void loadNeighbors() {
+    if (opts.neighborFilePath == null) {
+      LogInfo.logs("neighborFilePath is null.");
+      return;
+    }
+    LogInfo.begin_track("Loading cached neighbors from %s", opts.neighborFilePath);
+    uidToCachedNeighbors = new HashMap<>();
+    try {
+      BufferedReader reader = IOUtils.openIn(opts.neighborFilePath);
+      String line;
+      while ((line = reader.readLine()) != null) {
+        String[] tokens = line.split("\t");
+        String uid = tokens[0];
+        String[] nids = tokens[1].split(",");
+        uidToCachedNeighbors.put(uid, Arrays.asList(nids));
+      }
+      reader.close();
+    } catch (IOException e) {
+      throw new RuntimeException(e);
+    }
+    LogInfo.end_track();
+  }
+
+  public static void initialize(Example ex, Mode mode) {
+    CollaborativePruner.mode = mode;
+    predictedRules = null;
+    predictedPatterns = null;
+    foundConsistentDerivation = false;
+    if (mode == Mode.EXPLOIT) {
+      preprocessExample(ex);
+    }
+  }
+
+  static void preprocessExample(Example ex) {
+    Map<String, FormulaPattern> patternFreqMap = new HashMap<>();
+    List<String> cachedNeighbors = uidToCachedNeighbors.get(ex.id);
+    int total = 0;
+
+    // Gather the neighbors
+    if (opts.maxNumNeighbors > 0) {
+      for (String nid : cachedNeighbors) {
+        // Only get examples that have been previously processed + found a consistent formula
+        if (!consistentPattern.containsKey(nid))
+          continue;
+
+        String neighborPattern = consistentPattern.get(nid).pattern;
+        if (!patternFreqMap.containsKey(neighborPattern))
+          patternFreqMap.put(neighborPattern, new FormulaPattern(neighborPattern, 0));
+        patternFreqMap.get(neighborPattern).frequency++;
+        total++;
+        if (total >= opts.maxNumNeighbors)
+          break;
+      }
+    } else {
+      for (String patternString : allConsistentPatterns) {
+        patternFreqMap.put(patternString, new FormulaPattern(patternString, 1));
+      }
+    }
+
+    // Sort by frequency (more frequent = smaller; see FormulaPattern.compareTo)
+    List<Map.Entry<String, FormulaPattern>> patternFreqEntries = new ArrayList<>(patternFreqMap.entrySet());
+    patternFreqEntries.sort(new ValueComparator<>(false));
+
+    // Gather the patterns
+    LogInfo.begin_track("Predicted patterns");
+    int rank = 0;
+    Set<String> predictedRulesStrings = new HashSet<>();
+    predictedPatterns = new HashMap<>();
+    for (Map.Entry<String, FormulaPattern> entry : patternFreqEntries) {
+      FormulaPattern newPattern = entry.getValue();
+      predictedPatterns.put(newPattern.pattern, newPattern);
+      predictedRulesStrings.addAll(customRules.get(newPattern.pattern));
+      LogInfo.logs((rank + 1) + ". " + newPattern.pattern + " (" + newPattern.frequency + ")");
+      rank++;
+      if (rank >= opts.maxPredictedPatterns)
+        break;
+    }
+    // Gather the rules
+    predictedRules = customGrammar.getRules(predictedRulesStrings);
+    LogInfo.end_track();
+  }
+
+  public static String getPatternString(Derivation deriv) {
+    if (deriv.cat.equals("$TOKEN") || deriv.cat.equals("$PHRASE")
+        || deriv.cat.equals("$LEMMA_TOKEN") || deriv.cat.equals("$LEMMA_PHRASE")) {
+      return deriv.cat;
+    } else {
+      return FormulaPattern.convertToIndexedPattern(deriv);
+    }
+  }
+
+  public static void addRules(String patternString, Derivation deriv, Example ex) {
+    if (!customRules.containsKey(patternString)) {
+      customRules.put(patternString, new HashSet<String>());
+    }
+    Set<String> parsedCustomRules = customGrammar.addCustomRule(deriv, ex);
+    customRules.get(patternString).addAll(parsedCustomRules);
+  }
+
+  /**
+   * Get called when a (consistent) formula is found.
+   * Update the consistent patterns.
+   */
+  public static void updateConsistentPattern(ValueEvaluator evaluator, Example ex, Derivation deriv) {
+    String uid = ex.id;
+    if (ex.targetValue != null)
+      deriv.compatibility = evaluator.getCompatibility(ex.targetValue, deriv.value);
+
+    if (deriv.isRootCat() && deriv.compatibility == 1) {
+      foundConsistentDerivation = true;
+      LogInfo.logs("Found consistent deriv: %s", deriv);
+
+      String patternString = getPatternString(deriv);
+      FormulaPattern newConsistentPattern = new FormulaPattern(patternString, 0);
+      newConsistentPattern.score = deriv.getScore();
+
+      FormulaPattern oldConsistentPattern = consistentPattern.get(uid);
+      if (oldConsistentPattern == null || newConsistentPattern.score > oldConsistentPattern.score) {
+        addRules(patternString, deriv, ex);
+        consistentPattern.put(uid, newConsistentPattern);
+        allConsistentPatterns.add(patternString);
+      }
+    }
+  }
+
+  public static FormulaPattern getConsistentPattern(Example ex) {
+    return consistentPattern.get(ex.id);
+  }
+
+}

src/edu/stanford/nlp/sempre/cprune/CustomGrammar.java

@@ -0,0 +1,268 @@
+package edu.stanford.nlp.sempre.cprune;
+
+import java.util.*;
+
+import edu.stanford.nlp.sempre.*;
+import fig.basic.*;
+
+public class CustomGrammar extends Grammar {
+  public static class Options {
+    @Option(gloss = "Whether to decompose the templates into multiple rules")
+    public boolean enableTemplateDecomposition = true;
+  }
+
+  public static Options opts = new Options();
+
+  public static final Set<String> baseCategories = new HashSet<String>(Arrays.asList(
+      Rule.tokenCat, Rule.phraseCat, Rule.lemmaTokenCat, Rule.lemmaPhraseCat,
+      "$Unary", "$Binary", "$Entity", "$Property"));
+
+  ArrayList<Rule> baseRules = new ArrayList<>();
+  // symbolicFormulas => symbolicFormula ID
+  Map<String, Integer> symbolicFormulas = new HashMap<>();
+  // indexedSymbolicFormula => customRuleString
+  Map<String, Set<String>> customRules = new HashMap<>();
+  // customRuleString => Binarized rules
+  Map<String, Set<Rule>> customBinarizedRules = new HashMap<>();
+
+  public void init(Grammar initGrammar) {
+    baseRules = new ArrayList<>();
+    for (Rule rule : initGrammar.getRules()) {
+      if (baseCategories.contains(rule.lhs)) {
+        baseRules.add(rule);
+      }
+    }
+    this.freshCatIndex = initGrammar.getFreshCatIndex();
+  }
+
+  public List<Rule> getRules(Collection<String> customRuleStrings) {
+    Set<Rule> ruleSet = new LinkedHashSet<>();
+    ruleSet.addAll(baseRules);
+    for (String ruleString : customRuleStrings) {
+      ruleSet.addAll(customBinarizedRules.get(ruleString));
+    }
+    return new ArrayList<Rule>(ruleSet);
+  }
+
+  public Set<String> addCustomRule(Derivation deriv, Example ex) {
+    String indexedSymbolicFormula = getIndexedSymbolicFormula(deriv);
+    if (customRules.containsKey(indexedSymbolicFormula)) {
+      return customRules.get(indexedSymbolicFormula);
+    }
+
+    CPruneDerivInfo derivInfo = aggregateSymbols(deriv);
+    Set<String> crossReferences = new HashSet<>();
+    for (Symbol symbol : derivInfo.treeSymbols.values()) {
+      if (symbol.frequency > 1) {
+        crossReferences.add(symbol.formula);
+      }
+    }
+    computeCustomRules(deriv, crossReferences);
+    customRules.put(indexedSymbolicFormula, new HashSet<String>(derivInfo.customRuleStrings));
+
+    LogInfo.begin_track("Add custom rules for formula: " + indexedSymbolicFormula);
+    for (String customRuleString : derivInfo.customRuleStrings) {
+      if (customBinarizedRules.containsKey(customRuleString)) {
+        LogInfo.log("Custom rule exists: " + customRuleString);
+        continue;
+      }
+
+      rules = new ArrayList<>();
+      LispTree tree = LispTree.proto.parseFromString(customRuleString);
+      interpretRule(tree);
+      customBinarizedRules.put(customRuleString, new HashSet<Rule>(rules));
+
+      // Debug
+      LogInfo.begin_track("Add custom rule: " + customRuleString);
+      for (Rule rule : rules) {
+        LogInfo.log(rule.toString());
+      }
+      LogInfo.end_track();
+    }
+    LogInfo.end_track();
+
+    // Debug
+    System.out.println("consistent_lf\t" + ex.id + "\t" + deriv.formula.toString());
+
+    return customRules.get(indexedSymbolicFormula);
+  }
+
+  public static String getIndexedSymbolicFormula(Derivation deriv) {
+    return getIndexedSymbolicFormula(deriv, deriv.formula.toString());
+  }
+
+  /**
+   * Replace symbols (e.g., fb:row.row.name) with placeholders (e.g., Binary#1).
+   */
+  public static String getIndexedSymbolicFormula(Derivation deriv, String formula) {
+    CPruneDerivInfo derivInfo = aggregateSymbols(deriv);
+    int index = 1;
+    List<Symbol> symbolList = new ArrayList<>(derivInfo.treeSymbols.values());
+    for (Symbol symbol : symbolList)
+      symbol.computeIndex(formula);
+    Collections.sort(symbolList);
+    for (Symbol symbol : symbolList) {
+      if (formula.equals(symbol.formula))
+        formula = symbol.category + "#" + index;
+      formula = safeReplace(formula, symbol.formula, symbol.category + "#" + index);
+      index += 1;
+    }
+    return formula;
+  }
+
+  // ============================================================
+  // Private methods
+  // ============================================================
+
+  private static String safeReplace(String formula, String target, String replacement) {
+    // (argmin 1 1 ...) and (argmax 1 1 ...) are troublesome
+    String before = formula, targetBefore = target;
+    formula = formula.replace("(argmin (number 1) (number 1)", "(ARGMIN");
+    formula = formula.replace("(argmax (number 1) (number 1)", "(ARGMAX");
+    target = target.replace("(argmin (number 1) (number 1)", "(ARGMIN");
+    target = target.replace("(argmax (number 1) (number 1)", "(ARGMAX");
+    formula = formula.replace(target + ")", replacement + ")");
+    formula = formula.replace(target + " ", replacement + " ");
+    formula = formula.replace("(ARGMIN", "(argmin (number 1) (number 1)");
+    formula = formula.replace("(ARGMAX", "(argmax (number 1) (number 1)");
+    if (CollaborativePruner.opts.verbose >= 2)
+      LogInfo.logs("REPLACE: [%s | %s] %s | %s", targetBefore, replacement, before, formula);
+    return formula;
+  }
+
+  /**
+   * Cache the symbols in deriv.tempState[cprune].treeSymbols
+   */
+  private static CPruneDerivInfo aggregateSymbols(Derivation deriv) {
+    Map<String, Object> tempState = deriv.getTempState();
+    if (tempState.containsKey("cprune")) {
+      return (CPruneDerivInfo) tempState.get("cprune");
+    }
+    CPruneDerivInfo derivInfo = new CPruneDerivInfo();
+    tempState.put("cprune", derivInfo);
+
+    Map<String, Symbol> treeSymbols = new LinkedHashMap<>();
+    derivInfo.treeSymbols = treeSymbols;
+    if (baseCategories.contains(deriv.cat)) {
+      String formula = deriv.formula.toString();
+      treeSymbols.put(formula, new Symbol(deriv.cat, formula, 1));
+    } else {
+      for (Derivation child : deriv.children) {
+        CPruneDerivInfo childInfo = aggregateSymbols(child);
+        for (Symbol symbol : childInfo.treeSymbols.values()) {
+          if (derivInfo.treeSymbols.containsKey(symbol.formula)) {
+            treeSymbols.get(symbol.formula).frequency += symbol.frequency;
+          } else {
+            treeSymbols.put(symbol.formula, symbol);
+          }
+        }
+      }
+    }
+    return derivInfo;
+  }
+
+  private CPruneDerivInfo computeCustomRules(Derivation deriv, Set<String> crossReferences) {
+    CPruneDerivInfo derivInfo = (CPruneDerivInfo) deriv.getTempState().get("cprune");
+    Map<String, Symbol> ruleSymbols = new LinkedHashMap<>();
+    derivInfo.ruleSymbols = ruleSymbols;
+    derivInfo.customRuleStrings = new ArrayList<>();
+    String formula = deriv.formula.toString();
+
+    if (baseCategories.contains(deriv.cat)) {
+      // Leaf node induces no custom rule
+      derivInfo.containsCrossReference = crossReferences.contains(formula);
+      // Propagate the symbol of this derivation to the parent
+      ruleSymbols.putAll(derivInfo.treeSymbols);
+    } else {
+      derivInfo.containsCrossReference = false;
+      for (Derivation child : deriv.children) {
+        CPruneDerivInfo childInfo = computeCustomRules(child, crossReferences);
+        derivInfo.containsCrossReference = derivInfo.containsCrossReference || childInfo.containsCrossReference;
+      }
+
+      for (Derivation child : deriv.children) {
+        CPruneDerivInfo childInfo = (CPruneDerivInfo) child.getTempState().get("cprune");
+        ruleSymbols.putAll(childInfo.ruleSymbols);
+        derivInfo.customRuleStrings.addAll(childInfo.customRuleStrings);
+      }
+
+      if (opts.enableTemplateDecomposition == false || derivInfo.containsCrossReference) {
+        // If this node contains a cross reference
+        if (deriv.isRootCat()) {
+          // If this is the root node, then generate a custom rule
+          derivInfo.customRuleStrings.add(getCustomRuleString(deriv, derivInfo));
+        }
+      } else {
+        if (!deriv.cat.startsWith("$Intermediate")) {
+          // Generate a custom rule for this node
+          derivInfo.customRuleStrings.add(getCustomRuleString(deriv, derivInfo));
+
+          // Propagate this derivation as a category to the parent
+          ruleSymbols.clear();
+          ruleSymbols.put(formula, new Symbol(hash(deriv), deriv.formula.toString(), 1));
+        }
+      }
+    }
+    return derivInfo;
+  }
+
+  private String getCustomRuleString(Derivation deriv, CPruneDerivInfo derivInfo) {
+    String formula = deriv.formula.toString();
+    List<Symbol> rhsSymbols = new ArrayList<>(derivInfo.ruleSymbols.values());
+    for (Symbol symbol : rhsSymbols)
+      symbol.computeIndex(formula);
+    Collections.sort(rhsSymbols);
+
+    String lhs = null;
+    if (derivInfo.containsCrossReference)
+      lhs = deriv.cat;
+    else
+      lhs = deriv.isRootCat() ? "$ROOT" : hash(deriv);
+
+    LinkedList<String> rhsList = new LinkedList<>();
+    int index = 1;
+    for (Symbol symbol : rhsSymbols) {
+      if (formula.equals(symbol.formula)) {
+        formula = "(IdentityFn)";
+      } else {
+        formula = safeReplace(formula, symbol.formula, "(var s" + index + ")");
+        formula = "(lambda s" + index + " " + formula + ")";
+      }
+      rhsList.addFirst(symbol.category);
+      index += 1;
+    }
+    String rhs = null;
+    if (rhsList.size() > 0) {
+      rhs = "(" + String.join(" ", rhsList) + ")";
+    } else {
+      rhs = "(nothing)";
+      formula = "(ConstantFn " + formula + ")";
+    }
+    return "(rule " + lhs + " " + rhs + " " + formula + ")";
+  }
+
+  private String hash(Derivation deriv) {
+    if (baseCategories.contains(deriv.cat))
+      return deriv.cat;
+
+    String formula = getSymbolicFormula(deriv);
+    if (!symbolicFormulas.containsKey(formula)) {
+      symbolicFormulas.put(formula, symbolicFormulas.size() + 1);
+      String hashString = "$Formula" + symbolicFormulas.get(formula);
+      LogInfo.log("Add symbolic formula: " + hashString + " = " + formula + "  (" + deriv.cat + ")");
+    }
+    return "$Formula" + symbolicFormulas.get(formula);
+  }
+
+  private static String getSymbolicFormula(Derivation deriv) {
+    CPruneDerivInfo derivInfo = aggregateSymbols(deriv);
+    String formula = deriv.formula.toString();
+    for (Symbol symbol : derivInfo.treeSymbols.values()) {
+      if (formula.equals(symbol.formula))
+        formula = symbol.category;
+      formula = safeReplace(formula, symbol.formula, symbol.category);
+    }
+    return formula;
+  }
+
+}

src/edu/stanford/nlp/sempre/cprune/FormulaPattern.java

@@ -0,0 +1,106 @@
+package edu.stanford.nlp.sempre.cprune;
+
+import java.util.regex.Pattern;
+
+import edu.stanford.nlp.sempre.Derivation;
+import fig.basic.LogInfo;
+
+public class FormulaPattern implements Comparable<FormulaPattern> {
+  public String pattern;
+  public Integer frequency;
+  public Double score;
+
+  public FormulaPattern(String pattern, Integer frequency) {
+    this.pattern = pattern;
+    this.frequency = frequency;
+  }
+
+  public Double complexity() {
+    // Roughly the number of predicates
+    return (double) (pattern.length() - pattern.replace("(@R", "***").replace("(", "").length());
+  }
+
+  @Override
+  public String toString() {
+    return "(" + pattern + ", " + frequency + ")";
+  }
+
+  @Override
+  public int compareTo(FormulaPattern that) {
+    if (this.frequency > that.frequency) {
+      return -1;
+    } else if (this.frequency < that.frequency) {
+      return 1;
+    } else {
+      return this.complexity().compareTo(that.complexity());
+    }
+  }
+
+  // ============================================================
+  // Utilities
+  // ============================================================
+
+  private static Pattern reverseRelation = Pattern.compile("!(fb:[._a-z0-9]+)");
+  private static Pattern varName = Pattern.compile("\\((lambda|var) [a-z0-9]+");
+  private static Pattern compare = Pattern.compile("(<=|>=|>|<)");
+  private static Pattern whitespace = Pattern.compile("\\s+");
+
+  public static String convertToIndexedPattern(Derivation deriv) {
+    String formula = deriv.formula.toString();
+
+    // These can interfere with (number 1)
+    formula = formula.replace("argmax (number 1) (number 1)", "argmax");
+    formula = formula.replace("argmin (number 1) (number 1)", "argmin");
+
+    formula = removePropertyPredicates(formula);
+    formula = CustomGrammar.getIndexedSymbolicFormula(deriv, formula);
+
+    formula = formula.replace("fb:type.object.type fb:type.row", "@type @row");
+    formula = reverseRelation.matcher(formula).replaceAll("(reverse $1)");
+    formula = formula.replace("fb:row.row.index", "(reverse (lambda x ((reverse @index) (var x))))");
+    formula = formula.replace("fb:row.row.next", "@next");
+    formula = varName.matcher(formula).replaceAll("($1 x");
+    formula = formula.replace("reverse", "@R");
+    formula = compare.matcher(formula).replaceAll("@compare");
+    formula = whitespace.matcher(formula).replaceAll(" ");
+
+    if (CollaborativePruner.opts.verbose >= 2)
+      LogInfo.logs("PATTERN: %s -> %s", deriv.formula, formula);
+    return formula;
+  }
+
+  private static Pattern cellProperty = Pattern.compile("!?fb:cell\\.cell\\.[_a-z0-9]+|\\(reverse fb:cell\\.cell\\.[_a-z0-9]+\\)");
+
+  /**
+   * Remove cell property relations (fb:cell.cell.*)
+   */
+  public static String removePropertyPredicates(String formula) {
+    formula = cellProperty.matcher(formula).replaceAll("@PPT");
+    while (formula.contains("@PPT")) {
+      int begin = formula.indexOf("(@PPT");
+      if (begin == -1) {
+        formula = formula.replace("@PPT", "");
+        break;
+      }
+      // Find the matching parenthesis
+      int count = 1;
+      for (int i = begin + 1; i < formula.length(); i++) {
+        if (formula.charAt(i) == '(') {
+          count++;
+        } else if (formula.charAt(i) == ')') {
+          count--;
+          if (count == 0) {
+            int end = i;
+            formula = formula.substring(0, begin) + formula.substring(begin + 6, end) + formula.substring(end + 1, formula.length());
+            break;
+          }
+        }
+        if (i == formula.length() - 1) {
+          LogInfo.fails("Unbalanced parentheses: %s", formula);
+        }
+      }
+    }
+    return formula;
+  }
+
+}

src/edu/stanford/nlp/sempre/cprune/Symbol.java

@@ -0,0 +1,31 @@
+package edu.stanford.nlp.sempre.cprune;
+
+/**
+ * Represents the leaf node of the parse tree.
+ *
+ * Any sub-derivation whose category is in CustomGrammar.baseCategories becomes a Symbol.
+ */
+public class Symbol implements Comparable<Symbol> {
+  String category;
+  String formula;
+  Integer frequency;
+  Integer index;
+
+  public Symbol(String category, String formula, int frequency) {
+    this.category = category;
+    this.formula = formula;
+    this.frequency = frequency;
+  }
+
+  public void computeIndex(String referenceString) {
+    index = referenceString.indexOf(formula);
+    if (index < 0) {
+      index = Integer.MAX_VALUE;
+    }
+  }
+
+  @Override
+  public int compareTo(Symbol that) {
+    return index.compareTo(that.index);
+  }
+}

src/edu/stanford/nlp/sempre/geo880/Geo880TypeLookup.java

@@ -0,0 +1,110 @@
+package edu.stanford.nlp.sempre.geo880;
+
+import edu.stanford.nlp.sempre.SemType;
+import edu.stanford.nlp.sempre.SemTypeHierarchy;
+import edu.stanford.nlp.sempre.TypeLookup;
+import fig.basic.IOUtils;
+import fig.basic.Option;
+import fig.basic.LogInfo;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * Type lookup for the geo880 domain, Mostly for distinguishing locations and numbers.
+ * We also use a type hierarchy provided by a file to match |location.us_state| and |location.location| etc.
+ * Created by joberant on 05/12/2016.
+ */
+public class Geo880TypeLookup implements TypeLookup{
+  public static class Options {
+    @Option(gloss = "Verbosity") public int verbose = 0;
+    @Option(gloss = "A path to a file that specified the type hierarchy.")
+    public String typeHierarchyPath;
+
+  }
+  public static Options opts = new Options();
+  public static final String LOCATION = "fb:location.location";
+  public static final String CITY = "fb:location.citytown";
+  public static final String STATE = "fb:location.us_state";
+  public static final String RIVER = "fb:location.river";
+  public static final String LAKE = "fb:location.lake";
+  public static final String MOUNTAIN = "fb:location.mountain";
+  public static final String COUNTRY = "fb:location.country";
+
+  public Geo880TypeLookup() {
+    SemTypeHierarchy semTypeHierarchy = SemTypeHierarchy.singleton;
+    if (opts.typeHierarchyPath != null) {
+      try {
+        for (String line : IOUtils.readLines(opts.typeHierarchyPath)) {
+          String[] tokens = line.split("\\s+");
+
+          // Check the file only contains relations about supertypes.
+          assert tokens[1].endsWith("included_types");
+          semTypeHierarchy.addSupertype(tokens[0], tokens[0]);
+          semTypeHierarchy.addSupertype(tokens[2], tokens[2]);
+          semTypeHierarchy.addSupertype(tokens[0], tokens[2]);
+        }
+      } catch (IOException e) {
+        e.printStackTrace();
+        throw new RuntimeException("Could not read lines from: " + opts.typeHierarchyPath);
+      }
+    }
+  }
+
+  @Override
+  public SemType getEntityType(String entity) {
+    // Entites are of the form fb:state.florida.
+    int colonIndex = entity.indexOf(':');
+    int dotIndex = entity.indexOf('.');
+    String type = entity.substring(colonIndex+1, dotIndex);
+
+    if (type.equals("place")) {
+      type = LOCATION;
+    }
+    else if (type.equals("city")) {
+      type = CITY;
+    }
+    else if (type.equals("state")) {
+      type = STATE;
+    }
+    else if (type.equals("river")) {
+      type = RIVER;
+    }
+    else if (type.equals("lake")) {
+      type = LAKE;
+    }
+    else if (type.equals("mountain")) {
+      type = MOUNTAIN;
+    }
+    else if (type.equals("country")) {
+      type = COUNTRY;
+    }
+    else {
+      throw new RuntimeException("Illegal entity: " + entity);
+    }
+    SemType result = SemType.newUnionSemType(type);
+    if (opts.verbose >= 1) {
+      LogInfo.logs("Entity=%s, Type=%s", entity, result);
+    }
+    return result;
+  }
+
+  @Override
+  public SemType getPropertyType(String property) {
+    // Properties are of the form fb:location.location.population.
+    String arg1 = property.substring(0, property.lastIndexOf('.'));
+    String suffix = property.substring(property.lastIndexOf('.') + 1);
+    String arg2 = LOCATION;
+    if (suffix.equals("density") || suffix.equals("elevation") ||
+        suffix.equals("population") || suffix.equals("size") ||
+        suffix.equals("area") || suffix.equals("length")) {
+      arg2 = "fb:type.number";
+    }
+    SemType result = SemType.newFuncSemType(arg2, arg1);
+    if (opts.verbose >= 1) {
+      LogInfo.logs("Property=%s, Type=%s", property, result);
+    }
+    return result;
+  }
+}

src/edu/stanford/nlp/sempre/geo880/Geo880ValueEvaluator.java

@@ -0,0 +1,85 @@
+package edu.stanford.nlp.sempre.geo880;
+
+import edu.stanford.nlp.sempre.*;
+import edu.stanford.nlp.sempre.tables.StringNormalizationUtils;
+import fig.basic.LogInfo;
+
+import java.util.List;
+
+/**
+ * This is only used because the data does not mention when a city is in the usa, but
+ * the kg returns usa, and we want to use exact match, so we add this logic here.
+ * Created by joberant on 03/12/2016.
+ */
+public class Geo880ValueEvaluator implements ValueEvaluator {
+
+  public double getCompatibility(Value target, Value pred) {
+    List<Value> targetList = ((ListValue) target).values;
+    if (!(pred instanceof ListValue)) return 0;
+    List<Value> predList = ((ListValue) pred).values;
+
+    // In geo880, if we return that something is contained in a state, there is no need to return fb:country.usa
+    Value toDelete = null;
+    if (predList.size() > 1 && predList.get(0) instanceof NameValue) {
+      for (Value v: predList) {
+        String id = ((NameValue) v).id;
+        if (id.equals("fb:country.usa")) {
+          toDelete = v;
+          break;
+        }
+      }
+    }
+    if (toDelete != null) {
+      predList.remove(toDelete);
+    }
+
+    if (targetList.size() != predList.size()) return 0;
+
+    for (Value targetValue : targetList) {
+      boolean found = false;
+      for (Value predValue : predList) {
+        if (getItemCompatibility(targetValue, predValue)) {
+          found = true;
+          break;
+        }
+      }
+      if (!found) return 0;
+    }
+    return 1;
+  }
+
+  // ============================================================
+  // Item Compatibility
+  // ============================================================
+
+  // Compare one element of the list.
+  protected boolean getItemCompatibility(Value target, Value pred) {
+    if (pred instanceof ErrorValue) return false;  // Never award points for error
+    if (pred == null) {
+      LogInfo.warning("Predicted value is null!");
+      return false;
+    }
+
+    if (target instanceof DescriptionValue) {
+      String targetText = ((DescriptionValue) target).value;
+      if (pred instanceof NameValue) {
+        // Just has to match the description
+        String predText = ((NameValue) pred).description;
+        if (predText == null) predText = "";
+        return targetText.equals(predText);
+      }
+    } else if (target instanceof NumberValue) {
+      NumberValue targetNumber = (NumberValue) target;
+      if (pred instanceof NumberValue) {
+        return compareNumberValues(targetNumber, (NumberValue) pred);
+      }
+    }
+
+    return target.equals(pred);
+  }
+
+  protected boolean compareNumberValues(NumberValue target, NumberValue pred) {
+    return Math.abs(target.value - pred.value) < 1e-6;
+  }
+
+}

src/edu/stanford/nlp/sempre/interactive/ApplyFn.java

@@ -0,0 +1,66 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.List;
+
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.DerivationStream;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Formula;
+import edu.stanford.nlp.sempre.Formulas;
+import edu.stanford.nlp.sempre.LambdaFormula;
+import edu.stanford.nlp.sempre.SemanticFn;
+import edu.stanford.nlp.sempre.SingleDerivationStream;
+import fig.basic.LispTree;
+import fig.basic.Option;
+
+/**
+ * Take any number of arguments and apply them to the lambda expression given in
+ * this SemanticFn TODO: type inference, some function applications
+ * 
+ * @author sidaw
+ */
+public class ApplyFn extends SemanticFn {
+  public static class Options {
+    @Option(gloss = "verbosity")
+    public int verbose = 0;
+  }
+
+  public static Options opts = new Options();
+
+  Formula formula;
+
+  @Override
+  public void init(LispTree tree) {
+    super.init(tree);
+    formula = Formulas.fromLispTree(tree.child(1));
+  }
+
+  public Formula getFormula() {
+    return formula;
+  }
+
+  public ApplyFn() {
+  }
+
+  public ApplyFn(Formula f) {
+    formula = f;
+  }
+
+  @Override
+  public DerivationStream call(final Example ex, final Callable c) {
+    return new SingleDerivationStream() {
+      @Override
+      public Derivation createDerivation() {
+        List<Derivation> args = c.getChildren();
+        Formula f = Formulas.fromLispTree(formula.toLispTree());
+        for (Derivation arg : args) {
+          if (!(f instanceof LambdaFormula))
+            throw new RuntimeException("Expected LambdaFormula, but got " + f + "; initial: " + formula);
+          f = Formulas.lambdaApply((LambdaFormula) f, arg.getFormula());
+        }
+        Derivation res = new Derivation.Builder().withCallable(c).formula(f).createDerivation();
+        return res;
+      }
+    };
+  }
+}

src/edu/stanford/nlp/sempre/interactive/BadInteractionException.java

@@ -0,0 +1,48 @@
+package edu.stanford.nlp.sempre.interactive;
+
+public class BadInteractionException extends RuntimeException {
+
+  public static BadInteractionException nonSenseDefinition(String head) {
+    String message = String.format("Definitions should make sense and useable by yourself and others"
+        + "-- using more than 10 words, " + "or more than 15 characters in a word is not allowed."
+        + "If your definition is not non-sense," + "please paste this message in our bugs channel (head: %s)", head);
+    return new BadInteractionException(message);
+  }
+
+  public static BadInteractionException headIsCore(String head) {
+    String message = String.format(
+        "Redefining the core language is not allowed, " + "please reword your command and try again (head: %s)", head);
+    return new BadInteractionException(message);
+  }
+
+  public static BadInteractionException headIsEmpty(String head) {
+    String message = String.format("Cannot define with an empty head (head: %s)", head);
+    return new BadInteractionException(message);
+  }
+
+  public BadInteractionException() {
+    // TODO Auto-generated constructor stub
+  }
+
+  public BadInteractionException(String message) {
+    super(message);
+    // TODO Auto-generated constructor stub
+  }
+
+  public BadInteractionException(Throwable cause) {
+    super(cause);
+    // TODO Auto-generated constructor stub
+  }
+
+  public BadInteractionException(String message, Throwable cause) {
+    super(message, cause);
+    // TODO Auto-generated constructor stub
+  }
+
+  public BadInteractionException(String message, Throwable cause, boolean enableSuppression,
+      boolean writableStackTrace) {
+    super(message, cause, enableSuppression, writableStackTrace);
+    // TODO Auto-generated constructor stub
+  }
+
+}

src/edu/stanford/nlp/sempre/interactive/BlockFn.java

@@ -0,0 +1,106 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.List;
+
+import org.testng.collections.Lists;
+
+import edu.stanford.nlp.sempre.ActionFormula;
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.DerivationStream;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.FeatureExtractor;
+import edu.stanford.nlp.sempre.FeatureVector;
+import edu.stanford.nlp.sempre.SemanticFn;
+import edu.stanford.nlp.sempre.SingleDerivationStream;
+import fig.basic.LispTree;
+import fig.basic.Option;
+
+/**
+ * Generates formula scoped in various modes sequential: just perform in
+ * sequence, no scoping block: basic scoping block blockr: returns selected
+ * isolate: scopes allItems instead of selected
+ * 
+ * @author sidaw
+ */
+public class BlockFn extends SemanticFn {
+  public static class Options {
+    @Option(gloss = "verbosity")
+    public int verbose = 0;
+  }
+
+  public static Options opts = new Options();
+
+  List<ActionFormula.Mode> scopingModes = Lists.newArrayList(ActionFormula.Mode.block, ActionFormula.Mode.blockr,
+      ActionFormula.Mode.isolate);
+  ActionFormula.Mode mode = ActionFormula.Mode.block;
+  boolean optional = true;
+
+  @Override
+  public void init(LispTree tree) {
+    super.init(tree);
+    if (tree.child(1).value.equals("sequential"))
+      mode = ActionFormula.Mode.sequential;
+    else if (tree.child(1).value.equals("block"))
+      mode = ActionFormula.Mode.block;
+    else if (tree.child(1).value.equals("blockr"))
+      mode = ActionFormula.Mode.blockr;
+    else if (tree.child(1).value.equals("isolate"))
+      mode = ActionFormula.Mode.isolate;
+    else
+      mode = ActionFormula.Mode.sequential;
+  }
+
+  public BlockFn(ActionFormula.Mode mode) {
+    this.mode = mode;
+  }
+
+  public BlockFn() {
+    this.mode = ActionFormula.Mode.sequential;
+  }
+
+  @Override
+  public DerivationStream call(final Example ex, final Callable c) {
+    return new SingleDerivationStream() {
+      @Override
+      public Derivation createDerivation() {
+        List<Derivation> args = c.getChildren();
+        if (args.size() == 1) {
+          Derivation onlyChild = args.get(0);
+          // LogInfo.logs("1 BlockFn %s : %s Example.size=%d, callInfo(%d,%d)",
+          // onlyChild, mode, ex.getTokens().size(), onlyChild.getStart(),
+          // onlyChild.getEnd());
+
+          if (onlyChild == null)
+            return null;
+          if (onlyChild.getStart() != 0 || onlyChild.getEnd() != ex.getTokens().size())
+            return null;
+          // do not do anything to the core language
+          if (onlyChild.allAnchored())
+            return null;
+          // if (!ILUtils.stripBlock(onlyChild).rule.isInduced()) return null;
+
+          // if already blocked explicitly, do not do anything
+          if (scopingModes.contains(((ActionFormula) onlyChild.formula).mode))
+            return null;
+
+          // do not repeat any blocks
+          if (((ActionFormula) onlyChild.formula).mode == BlockFn.this.mode)
+            return null;
+
+          FeatureVector features = new FeatureVector();
+          if (FeatureExtractor.containsDomain(":scope")) {
+            features.add(":scope",
+                BlockFn.this.mode.toString() + "::" + !InteractiveUtils.stripBlock(onlyChild).rule.isInduced());
+            features.add(":scope", BlockFn.this.mode.toString() + "::" + ex.id);
+          }
+
+          Derivation deriv = new Derivation.Builder()
+              .formula(new ActionFormula(mode, Lists.newArrayList(onlyChild.formula))).withCallable(c)
+              .localFeatureVector(features).createDerivation();
+          return deriv;
+        } else
+          return null;
+      }
+    };
+  }
+}

src/edu/stanford/nlp/sempre/interactive/CitationTracker.java

@@ -0,0 +1,166 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.io.File;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.net.URLDecoder;
+import java.net.URLEncoder;
+import java.nio.file.Paths;
+import java.time.LocalDateTime;
+import java.util.Base64;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+import com.google.common.base.Strings;
+
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Json;
+import edu.stanford.nlp.sempre.Rule;
+import fig.basic.IOUtils;
+import fig.basic.LogInfo;
+
+/**
+ * Tracks rule usage via a citation system. A rule is cited when a user makes
+ * use of that rule in a derivation.
+ * 
+ * @author sidaw
+ */
+public class CitationTracker {
+  public static final String IDPrefix = "id:";
+  public static final String HeadPrefix = "head:";
+  public static final String BodyPrefix = "body:";
+  String uid = "undefined";
+  Example ex;
+
+  public CitationTracker(String uid, Example ex) {
+    this.uid = uid;
+    this.ex = ex;
+  }
+
+  public synchronized void citeRule(Rule rule) {
+    writeLog(rule);
+    writeSummary(rule);
+  }
+
+  // The summary is ONE SINGLE line of json, has cite, self, and head
+  private synchronized void writeSummary(Rule rule) {
+    String author = getAuthor(rule);
+    String summaryPath = Paths.get(InteractiveUtils.opts.citationPath, encode(author), encode(getHead(rule)) + ".json")
+        .toString();
+    File file = new File(summaryPath);
+    file.getParentFile().mkdirs();
+
+    Map<String, Object> summary;
+    try {
+      String line = IOUtils.readLineEasy(summaryPath);
+      if (line == null)
+        summary = defaultMap(rule);
+      else
+        summary = Json.readMapHard(line);
+
+      boolean selfcite = author.equals(uid);
+      if (!selfcite) {
+        summary.put("cite", (Integer) summary.get("cite") + 1);
+        rule.source.cite++;
+      } else {
+        summary.put("self", (Integer) summary.get("self") + 1);
+        rule.source.self++;
+      }
+
+    } catch (Exception e) {
+      summary = defaultMap(rule);
+      e.printStackTrace();
+    }
+    String jsonStr = Json.writeValueAsStringHard(summary);
+    PrintWriter out = IOUtils.openOutHard(file);
+    out.println(jsonStr);
+    out.close();
+  }
+
+  private Map<String, Object> defaultMap(Rule rule) {
+    Map<String, Object> summary = new LinkedHashMap<>();
+    summary.put("cite", 0);
+    summary.put("self", 0);
+    summary.put("private", true);
+    summary.put("head", getHead(rule));
+    summary.put("body", getBody(rule));
+    return summary;
+  }
+
+  private synchronized void writeLog(Rule rule) {
+    String head = getHead(rule);
+    String author = getAuthor(rule);
+    String logPath = Paths.get(InteractiveUtils.opts.citationPath, encode(author), encode(head) + ".json.log")
+        .toString();
+    File file = new File(logPath);
+    file.getParentFile().mkdirs();
+
+    Map<String, Object> jsonMap = new LinkedHashMap<>();
+    jsonMap.put("user", this.uid);
+    // jsonMap.put("body", decode(getBody(rule)));
+    jsonMap.put("time", LocalDateTime.now().toString());
+    jsonMap.put("tokens", ex.getTokens());
+    // jsonMap.put("head", decode(headCode));
+    jsonMap.put("author", author);
+
+    String jsonStr = Json.writeValueAsStringHard(jsonMap);
+    PrintWriter out = IOUtils.openOutAppendHard(file);
+    out.println(jsonStr);
+    out.close();
+  }
+
+  public void citeAll(Derivation deriv) {
+    if (deriv.rule != null && deriv.rule.isInduced()) {
+      LogInfo.logs("CitationTracker: user %s is citing rule: %s", this.uid, deriv.rule.toString());
+      citeRule(deriv.rule);
+    }
+
+    if (deriv.children == null)
+      return;
+    for (Derivation d : deriv.children) {
+      citeAll(d);
+    }
+  }
+
+  static String getAuthor(Rule rule) {
+    try {
+      String author = rule.source.uid;
+      if (Strings.isNullOrEmpty(author))
+        return "__noname__";
+      else
+        return author;
+    } catch (Exception e) {
+      e.printStackTrace();
+      return "__noname__";
+    }
+  }
+
+  static String getHead(Rule rule) {
+    return rule.source.head;
+  }
+
+  static String getBody(Rule rule) {
+    return String.join(". ", rule.source.body);
+  }
+
+  public static String encode(String utt) {
+    try {
+      return URLEncoder.encode(utt, "UTF-8");
+    } catch (UnsupportedEncodingException e) {
+      e.printStackTrace();
+    }
+    return Base64.getUrlEncoder().encodeToString(utt.getBytes());
+    // return Base64.getUrlEncoder().encodeToString(utt.getBytes());
+  }
+
+  public static String decode(String code) {
+    try {
+      return URLDecoder.decode(code, "UTF-8");
+    } catch (UnsupportedEncodingException e) {
+      e.printStackTrace();
+    }
+    return Base64.getUrlDecoder().decode(code).toString();
+  }
+
+}

src/edu/stanford/nlp/sempre/interactive/DALAnalyzer.java

@@ -0,0 +1,121 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.Arrays;
+
+import edu.stanford.nlp.sempre.LanguageAnalyzer;
+import edu.stanford.nlp.sempre.LanguageInfo;
+
+/**
+ * LanguageAnalyzer for DAL, basically a lexer
+ *
+ * @author sidaw
+ */
+public class DALAnalyzer extends LanguageAnalyzer {
+  // Stanford tokenizer doesn't break hyphens.
+  // Replace hypens with spaces for utterances like
+  // "Spanish-speaking countries" but not for "2012-03-28".
+  public static String breakHyphens(String utterance) {
+    StringBuilder buf = new StringBuilder(utterance);
+    for (int i = 0; i < buf.length(); i++) {
+      if (buf.charAt(i) == '-' && (i + 1 < buf.length() && Character.isLetter(buf.charAt(i + 1))))
+        buf.setCharAt(i, ' ');
+    }
+    return buf.toString();
+  }
+
+  private static final String[] numbers = { "zero", "one", "two", "three", "four", "five", "six", "seven", "eight",
+      "nine", "ten" };
+
+  @Override
+  public LanguageInfo analyze(String utterance) {
+    LanguageInfo languageInfo = new LanguageInfo();
+
+    // Clear these so that analyze can hypothetically be called
+    // multiple times.
+    languageInfo.tokens.clear();
+    languageInfo.posTags.clear();
+    languageInfo.nerTags.clear();
+    languageInfo.nerValues.clear();
+    languageInfo.lemmaTokens.clear();
+
+    // Break hyphens
+    utterance = breakHyphens(utterance);
+
+    // Default analysis - create tokens crudely
+    StringBuilder buf = new StringBuilder();
+    for (int i = 0; i < utterance.length(); i++) {
+      char c = utterance.charAt(i);
+      // Put whitespace around certain characters.
+      boolean boundaryBefore = !(i - 1 >= 0) || utterance.charAt(i - 1) == ' ';
+      boolean boundaryAfter = !(i + 1 < utterance.length()) || utterance.charAt(i + 1) == ' ';
+      boolean separate = false;
+      if (c == '.') // Break off period if already space around it (to preserve
+                    // numbers like 3.5)
+        separate = boundaryBefore || boundaryAfter;
+      else if (c == '=') // separate all >, =, < except >=, <=
+        separate = !(i - 1 >= 0) || ((utterance.charAt(i - 1) != '>' && utterance.charAt(i - 1) != '<'));
+      else if (c == '>' || c == '<')
+        separate = !(i + 1 < utterance.length())
+            || ((utterance.charAt(i + 1) != '=' && utterance.charAt(i + 1) != '='));
+      else
+        separate = (",?'\"[];{}+-".indexOf(c) != -1);
+
+      if (separate)
+        buf.append(' ');
+      // Convert quotes
+      if (c == '"')
+        buf.append(boundaryBefore ? "``" : "''");
+      else if (c == '\'')
+        buf.append(boundaryBefore ? "`" : "'");
+      else if (c == '>' || c == '<') {
+        buf.append(' ');
+        buf.append(c);
+      } else if (c == '=') {
+        buf.append(c);
+        buf.append(' ');
+      } else
+        buf.append(c);
+      if (separate)
+        buf.append(' ');
+    }
+    utterance = buf.toString().trim();
+    if (!utterance.equals("")) {
+      String[] tokens = utterance.split("\\s+");
+      for (String token : tokens) {
+        String lemma = token;
+        if (token.endsWith("s") && token.length() > 1)
+          lemma = token.substring(0, token.length() - 1);
+
+        languageInfo.tokens.add(LanguageAnalyzer.opts.lowerCaseTokens ? token.toLowerCase() : token);
+        languageInfo.lemmaTokens.add(LanguageAnalyzer.opts.lowerCaseTokens ? lemma.toLowerCase() : lemma);
+
+        // Is it a written out number?
+        int x = Arrays.asList(numbers).indexOf(token);
+        if (x != -1) {
+          languageInfo.posTags.add("CD");
+          languageInfo.nerTags.add("NUMBER");
+          languageInfo.nerValues.add(x + "");
+          continue;
+        }
+
+        try {
+          Double.parseDouble(token);
+          languageInfo.posTags.add("CD");
+          languageInfo.nerTags.add("NUMBER");
+          languageInfo.nerValues.add(token);
+        } catch (NumberFormatException e) {
+          // Guess that capitalized nouns are proper
+          if (Character.isUpperCase(token.charAt(0)))
+            languageInfo.posTags.add("NNP");
+          else if (token.equals("'") || token.equals("\"") || token.equals("''") || token.equals("``"))
+            languageInfo.posTags.add("''");
+          else
+            languageInfo.posTags.add("UNK");
+          languageInfo.nerTags.add("UNK");
+          languageInfo.nerValues.add("UNK");
+        }
+      }
+    }
+    return languageInfo;
+  }
+}

src/edu/stanford/nlp/sempre/interactive/DALExecutor.java

@@ -0,0 +1,493 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.google.common.collect.Lists;
+import com.google.common.collect.ObjectArrays;
+import com.google.common.collect.Sets;
+
+import edu.stanford.nlp.sempre.*;
+import fig.basic.LogInfo;
+import fig.basic.Option;
+
+/**
+ * Handles action lambda DCS where the world has a flat structure, i.e. a list
+ * of allitems all supporting the same operations supports ActionFormula here,
+ * and does conversions of singleton sets
+ * 
+ * @author sidaw
+ */
+public class DALExecutor extends Executor {
+  public static class Options {
+    @Option(gloss = "Whether to convert NumberValue to int/double")
+    public boolean convertNumberValues = true;
+    @Option(gloss = "Whether to convert name values to string literal")
+    public boolean convertNameValues = true;
+
+    @Option(gloss = "Print stack trace on exception")
+    public boolean printStackTrace = false;
+    // the actual function will be called with the current ContextValue as its
+    // last argument if marked by contextPrefix
+    @Option(gloss = "Reduce verbosity by automatically appending, for example, edu.stanford.nlp.sempre to java calls")
+    public String classPathPrefix = "edu.stanford.nlp.sempre";
+
+    @Option(gloss = "The type of world used")
+    public String worldType = "VoxelWorld";
+
+    @Option(gloss = "the maximum number of primitive calls until we stop executing")
+    public int maxSteps = 1000;
+
+    @Option(gloss = "The maximum number of while calls")
+    public int maxWhile = 20;
+  }
+
+  public static Options opts = new Options();
+
+  @Override
+  public Response execute(Formula formula, ContextValue context) {
+    // We can do beta reduction here since macro substitution preserves the
+    // denotation (unlike for lambda DCS).
+    World world = World.fromContext(opts.worldType, context);
+    formula = Formulas.betaReduction(formula);
+    try {
+      performActions((ActionFormula) formula, world);
+      return new Response(new StringValue(world.toJSON()));
+    } catch (Exception e) {
+      // Comment this out if we expect lots of innocuous type checking failures
+      if (opts.printStackTrace) {
+        LogInfo.log("Failed to execute " + formula.toString());
+        e.printStackTrace();
+      }
+      return new Response(ErrorValue.badJava(e.toString()));
+    }
+  }
+
+  @SuppressWarnings("rawtypes")
+  private void performActions(ActionFormula f, World world) {
+    if (f.mode == ActionFormula.Mode.primitive) {
+      // use reflection to call primitive stuff
+      Value method = ((ValueFormula) f.args.get(0)).value;
+      String id = ((NameValue) method).id;
+      // all actions takes a fixed set as argument
+      invoke(id, world, f.args.subList(1, f.args.size()).stream().map(x -> processSetFormula(x, world)).toArray());
+      world.merge();
+    } else if (f.mode == ActionFormula.Mode.sequential) {
+      for (Formula child : f.args) {
+        performActions((ActionFormula) child, world);
+      }
+    } else if (f.mode == ActionFormula.Mode.repeat) {
+      Set<Object> arg = toSet(processSetFormula(f.args.get(0), world));
+      if (arg.size() > 1)
+        throw new RuntimeException("repeat has to take a single number");
+      int times;
+      if (!opts.convertNumberValues)
+        times = (int) ((NumberValue) arg.iterator().next()).value;
+      else
+        times = (int) arg.iterator().next();
+
+      for (int i = 0; i < times; i++)
+        performActions((ActionFormula) f.args.get(1), world);
+    } else if (f.mode == ActionFormula.Mode.conditional) {
+      // using the empty set to represent false
+      boolean cond = toSet(processSetFormula(f.args.get(0), world)).iterator().hasNext();
+      if (cond)
+        performActions((ActionFormula) f.args.get(1), world);
+    } else if (f.mode == ActionFormula.Mode.whileloop) {
+      // using the empty set to represent false
+      boolean cond = toSet(processSetFormula(f.args.get(0), world)).iterator().hasNext();
+      for (int i = 0; i < opts.maxWhile; i++) {
+        if (cond)
+          performActions((ActionFormula) f.args.get(1), world);
+        else
+          break;
+        cond = toSet(processSetFormula(f.args.get(0), world)).iterator().hasNext();
+      }
+    } else if (f.mode == ActionFormula.Mode.forset) {
+      // mostly deprecated
+      Set<Object> selected = toSet(processSetFormula(f.args.get(0), world));
+      Set<Item> prevSelected = world.selected;
+
+      world.selected = toItemSet(selected);
+      performActions((ActionFormula) f.args.get(1), world);
+
+      world.selected = prevSelected;
+      world.merge();
+    } else if (f.mode == ActionFormula.Mode.foreach) {
+      Set<Item> selected = toItemSet(toSet(processSetFormula(f.args.get(0), world)));
+      Set<Item> prevSelected = world.selected;
+      // CopyOnWriteArraySet<Object> fixedset =
+      // Sets.newCopyOnWriteArraySet(selected);
+      Iterator<Item> iterator = selected.iterator();
+      while (iterator.hasNext()) {
+        world.selected = (toItemSet(toSet(iterator.next())));
+        performActions((ActionFormula) f.args.get(1), world);
+      }
+      world.selected = prevSelected;
+      world.merge();
+
+    } else if (f.mode == ActionFormula.Mode.isolate) {
+      Set<Item> prevAll = world.allItems;
+      // Set<Item> prevSelected = world.selected;
+      // Set<Item> prevPrevious = world.previous;
+      if (f.args.size() > 1)
+        throw new RuntimeException("No longer supporting this isolate formula: " + f);
+
+      world.allItems = Sets.newHashSet(world.selected);
+      // world.selected = scope;
+      // world.previous = scope;
+      performActions((ActionFormula) f.args.get(0), world);
+
+      world.allItems.addAll(prevAll); // merge, overriding;
+      // world.selected = prevSelected;
+      // world.previous = prevPrevious;
+      world.merge();
+
+    } else if (f.mode == ActionFormula.Mode.block || f.mode == ActionFormula.Mode.blockr) {
+      // we should never mutate selected in actions
+      Set<Item> prevSelected = world.selected;
+      Set<Item> prevPrevious = world.previous;
+      world.previous = world.selected;
+
+      for (Formula child : f.args) {
+        performActions((ActionFormula) child, world);
+      }
+
+      // restore on default blocks
+      if (f.mode == ActionFormula.Mode.block) {
+        world.selected = prevSelected;
+        world.merge();
+      }
+      // LogInfo.logs("CBlocking prevselected=%s selected=%s all=%s",
+      // prevSelected, world.selected, world.allitems);
+      // LogInfo.logs("BlockingWorldIs %s", world.toJSON());
+      world.previous = prevPrevious;
+    }
+    // } else if (f.mode == ActionFormula.Mode.let) {
+    // // let declares a new local variable
+    // // set access and reassigns the value of some variable
+    // // block determines what is considered local scope
+    // // for now the use case is just (:blk (:let x this) (:blah) (:set this
+    // x))
+    // Set<Item> varset = toItemSet(toSet(processSetFormula(f.args.get(1),
+    // world)));
+    // Value method = ((ValueFormula)f.args.get(0)).value;
+    // String varname = ((NameValue)method).id;
+    // world.variables.put(varname, varset);
+    // } else if (f.mode == ActionFormula.Mode.set) {
+    // Set<Item> varset = toItemSet(toSet(processSetFormula(f.args.get(1),
+    // world)));
+    // Value method = ((ValueFormula)f.args.get(0)).value;
+    // String varname = ((NameValue)method).id;
+    // world.variables.get(varname).clear();
+    // world.variables.get(varname).addAll(varset);
+    // }
+
+  }
+
+  @SuppressWarnings("unchecked")
+  private Set<Object> toSet(Object maybeSet) {
+    if (maybeSet instanceof Set)
+      return (Set<Object>) maybeSet;
+    else
+      return Sets.newHashSet(maybeSet);
+  }
+
+  private Object toElement(Set<Object> set) {
+    if (set.size() == 1) {
+      return set.iterator().next();
+    }
+    return set;
+  }
+
+  private Set<Item> toItemSet(Set<Object> maybeItems) {
+    Set<Item> itemset = maybeItems.stream().map(i -> (Item) i).collect(Collectors.toSet());
+    return itemset;
+  }
+
+  static class SpecialSets {
+    static String All = "*";
+    static String EmptySet = "nothing";
+    static String This = "this"; // current scope if it exists, otherwise the
+                                 // globally marked object
+    static String Previous = "prev"; // global variable for selected
+    static String Selected = "selected"; // global variable for selected
+  };
+
+  // a subset of lambda dcs. no types, and no marks
+  // if this gets any more complicated, you should consider the
+  // LambdaDCSExecutor
+  @SuppressWarnings("unchecked")
+  private Object processSetFormula(Formula formula, final World world) {
+    if (formula instanceof ValueFormula<?>) {
+      Value v = ((ValueFormula<?>) formula).value;
+      // special unary
+      if (v instanceof NameValue) {
+        String id = ((NameValue) v).id;
+        // LogInfo.logs("%s : this %s, all: %s", id,
+        // world.selected().toString(), world.allitems.toString());
+        if (id.equals(SpecialSets.All))
+          return world.all();
+        if (id.equals(SpecialSets.This))
+          return world.selected();
+        if (id.equals(SpecialSets.Selected))
+          return world.selected();
+        if (id.equals(SpecialSets.EmptySet))
+          return world.empty();
+        if (id.equals(SpecialSets.Previous))
+          return world.previous();
+      }
+      return toObject(((ValueFormula<?>) formula).value);
+    }
+
+    if (formula instanceof JoinFormula) {
+      JoinFormula joinFormula = (JoinFormula) formula;
+      if (joinFormula.relation instanceof ValueFormula) {
+        String rel = ((ValueFormula<NameValue>) joinFormula.relation).value.id;
+        Set<Object> unary = toSet(processSetFormula(joinFormula.child, world));
+        return world.has(rel, unary);
+      } else if (joinFormula.relation instanceof ReverseFormula) {
+        ReverseFormula reverse = (ReverseFormula) joinFormula.relation;
+        String rel = ((ValueFormula<NameValue>) reverse.child).value.id;
+        Set<Object> unary = toSet(processSetFormula(joinFormula.child, world));
+        return world.get(rel, toItemSet(unary));
+      } else {
+        throw new RuntimeException("relation can either be a value, or its reverse");
+      }
+    }
+
+    if (formula instanceof MergeFormula) {
+      MergeFormula mergeFormula = (MergeFormula) formula;
+      MergeFormula.Mode mode = mergeFormula.mode;
+      Set<Object> set1 = toSet(processSetFormula(mergeFormula.child1, world));
+      Set<Object> set2 = toSet(processSetFormula(mergeFormula.child2, world));
+
+      if (mode == MergeFormula.Mode.or)
+        return Sets.union(set1, set2);
+      if (mode == MergeFormula.Mode.and)
+        return Sets.intersection(set1, set2);
+
+    }
+
+    if (formula instanceof NotFormula) {
+      NotFormula notFormula = (NotFormula) formula;
+      Set<Item> set1 = toItemSet(toSet(processSetFormula(notFormula.child, world)));
+      return Sets.difference(world.allItems, set1);
+    }
+
+    if (formula instanceof AggregateFormula) {
+      AggregateFormula aggregateFormula = (AggregateFormula) formula;
+      Set<Object> set = toSet(processSetFormula(aggregateFormula.child, world));
+      AggregateFormula.Mode mode = aggregateFormula.mode;
+      if (mode == AggregateFormula.Mode.count)
+        return Sets.newHashSet(set.size());
+      if (mode == AggregateFormula.Mode.max)
+        return Sets.newHashSet(set.stream().max((s, t) -> ((NumberValue) s).value > ((NumberValue) t).value ? 1 : -1));
+      if (mode == AggregateFormula.Mode.min)
+        return Sets.newHashSet(set.stream().max((s, t) -> ((NumberValue) s).value < ((NumberValue) t).value ? 1 : -1));
+    }
+
+    if (formula instanceof ArithmeticFormula) {
+      ArithmeticFormula arithmeticFormula = (ArithmeticFormula) formula;
+      Integer arg1 = (Integer) processSetFormula(arithmeticFormula.child1, world);
+      Integer arg2 = (Integer) processSetFormula(arithmeticFormula.child2, world);
+      ArithmeticFormula.Mode mode = arithmeticFormula.mode;
+      if (mode == ArithmeticFormula.Mode.add)
+        return arg1 + arg2;
+      if (mode == ArithmeticFormula.Mode.sub)
+        return arg1 - arg2;
+      if (mode == ArithmeticFormula.Mode.mul)
+        return arg1 * arg2;
+      if (mode == ArithmeticFormula.Mode.div)
+        return arg1 / arg2;
+    }
+
+    if (formula instanceof CallFormula) {
+      CallFormula callFormula = (CallFormula) formula;
+      @SuppressWarnings("rawtypes")
+      Value method = ((ValueFormula) callFormula.func).value;
+      String id = ((NameValue) method).id;
+      // all actions takes a fixed set as argument
+      return invoke(id, world, callFormula.args.stream().map(x -> processSetFormula(x, world)).toArray());
+    }
+    if (formula instanceof SuperlativeFormula) {
+      throw new RuntimeException("SuperlativeFormula is not implemented");
+    }
+    throw new RuntimeException("ActionExecutor does not handle this formula type: " + formula.getClass());
+  }
+
+  // Example: id = "Math.cos". similar to JavaExecutor's invoke,
+  // but matches arg by building singleton set as needed
+  private Object invoke(String id, World thisObj, Object... args) {
+    Method[] methods;
+    Class<?> cls;
+    String methodName;
+    boolean isStatic = thisObj == null;
+
+    if (isStatic) { // Static methods
+      int i = id.lastIndexOf('.');
+      if (i == -1) {
+        throw new RuntimeException("Expected <class>.<method>, but got: " + id);
+      }
+      String className = id.substring(0, i);
+      methodName = id.substring(i + 1);
+
+      try {
+        cls = Class.forName(className);
+      } catch (ClassNotFoundException e) {
+        throw new RuntimeException(e);
+      }
+      methods = cls.getMethods();
+    } else { // Instance methods
+      cls = thisObj.getClass();
+      methodName = id;
+      methods = cls.getMethods();
+    }
+
+    // Find a suitable method
+    List<Method> nameMatches = Lists.newArrayList();
+    Method bestMethod = null;
+    int bestCost = INVALID_TYPE_COST;
+    for (Method m : methods) {
+      if (!m.getName().equals(methodName))
+        continue;
+      m.setAccessible(true);
+      nameMatches.add(m);
+      if (isStatic != Modifier.isStatic(m.getModifiers()))
+        continue;
+      int cost = typeCastCost(m.getParameterTypes(), args);
+
+      // append optional selected parameter when needed:
+      if (cost == INVALID_TYPE_COST && args.length + 1 == m.getParameterCount()) {
+        args = ObjectArrays.concat(args, thisObj.selected);
+        cost = typeCastCost(m.getParameterTypes(), args);
+      }
+
+      if (cost < bestCost) {
+        bestCost = cost;
+        bestMethod = m;
+      }
+    }
+
+    if (bestMethod != null) {
+      try {
+        return bestMethod.invoke(thisObj, args);
+      } catch (InvocationTargetException e) {
+        throw new RuntimeException(e.getCause());
+      } catch (IllegalAccessException e) {
+        throw new RuntimeException(e);
+      }
+    }
+    List<String> types = Lists.newArrayList();
+    for (Object arg : args)
+      types.add(arg.getClass().toString());
+    throw new RuntimeException("Method " + methodName + " not found in class " + cls + " with arguments "
+        + Arrays.asList(args) + " having types " + types + "; candidates: " + nameMatches);
+  }
+
+  private int typeCastCost(Class[] types, Object[] args) {
+    if (types.length != args.length)
+      return INVALID_TYPE_COST;
+    int cost = 0;
+    for (int i = 0; i < types.length; i++) {
+
+      // deal with singleton sets
+      if (types[i] == Set.class)
+        args[i] = toSet(args[i]);
+      if (types[i] != Set.class && args[i].getClass() == Set.class) {
+        args[i] = toElement((Set<Object>) args[i]);
+      }
+
+      cost += typeCastCost(types[i], args[i]);
+      if (cost >= INVALID_TYPE_COST) {
+        LogInfo.dbgs("NOT COMPATIBLE: want %s, got %s with type %s", types[i], args[i], args[i].getClass());
+        break;
+      }
+    }
+    return cost;
+  }
+
+  private static Object toObject(Value value) {
+    if (value instanceof NumberValue && opts.convertNumberValues) {
+      // Unfortunately, NumberValues don't make a distinction between ints and
+      // doubles, so this is a hack.
+      double x = ((NumberValue) value).value;
+      if (x == (int) x)
+        return new Integer((int) x);
+      return new Double(x);
+    } else if (value instanceof NameValue && opts.convertNameValues) {
+      String id = ((NameValue) value).id;
+      return id;
+    } else if (value instanceof BooleanValue) {
+      return ((BooleanValue) value).value;
+    } else if (value instanceof StringValue) {
+      return ((StringValue) value).value;
+    } else if (value instanceof ListValue) {
+      List<Object> list = Lists.newArrayList();
+      for (Value elem : ((ListValue) value).values)
+        list.add(toObject(elem));
+      return list;
+    } else {
+      return value; // Preserve the Value (which can be an object)
+    }
+  }
+
+  // Return whether the object |arg| is compatible with |type|.
+  // 0: perfect match
+  // 1: don't match, but don't lose anything
+  // 2: don't match, and can lose something
+  // INVALID_TYPE_COST: impossible
+  private int typeCastCost(Class<?> type, Object arg) {
+    if (arg == null)
+      return !type.isPrimitive() ? 0 : INVALID_TYPE_COST;
+    if (type.isInstance(arg))
+      return 0;
+    if (type == Boolean.TYPE)
+      return arg instanceof Boolean ? 0 : INVALID_TYPE_COST;
+    else if (type == Integer.TYPE) {
+      if (arg instanceof Integer)
+        return 0;
+      if (arg instanceof Long)
+        return 1;
+      return INVALID_TYPE_COST;
+    }
+    if (type == Long.TYPE) {
+      if (arg instanceof Integer)
+        return 1;
+      if (arg instanceof Long)
+        return 0;
+      return INVALID_TYPE_COST;
+    }
+    if (type == Float.TYPE) {
+      if (arg instanceof Integer)
+        return 1;
+      if (arg instanceof Long)
+        return 1;
+      if (arg instanceof Float)
+        return 0;
+      if (arg instanceof Double)
+        return 2;
+      return INVALID_TYPE_COST;
+    }
+    if (type == Double.TYPE) {
+      if (arg instanceof Integer)
+        return 1;
+      if (arg instanceof Long)
+        return 1;
+      if (arg instanceof Float)
+        return 1;
+      if (arg instanceof Double)
+        return 0;
+      return INVALID_TYPE_COST;
+    }
+    return INVALID_TYPE_COST;
+  }
+
+  private static final int INVALID_TYPE_COST = 1000;
+}

src/edu/stanford/nlp/sempre/interactive/DALFeatureComputer.java

@@ -0,0 +1,166 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.FeatureComputer;
+import edu.stanford.nlp.sempre.FeatureExtractor;
+import edu.stanford.nlp.sempre.Rule;
+import fig.basic.Option;
+
+/**
+ * Feature computer for the the dependency-based action language TODOs: -
+ * control what categories to abstract out - efficiency improvement, right now
+ * use all members of the cross product
+ * 
+ * @author sidaw
+ */
+public class DALFeatureComputer implements FeatureComputer {
+  public static class Options {
+    @Option(gloss = "Verbosity")
+    public int verbose = 0;
+
+    @Option(gloss = "the N in N-gram")
+    public int ngramN = 3;
+
+    @Option(gloss = "size of the context window to consider")
+    public int windowSize = 2;
+  }
+
+  public static Options opts = new Options();
+
+  @Override
+  public void extractLocal(Example ex, Derivation deriv) {
+    addStatsFeatures(ex, deriv);
+    addWindowFeatures(ex, deriv);
+    addSocialFeatures(ex, deriv);
+    extractRuleFeatures(ex, deriv);
+    extractSpanFeatures(ex, deriv);
+  }
+
+  // function to abstract out ALL anchored stuff in the utterance.
+  private List<String> abstractAnchors(Derivation deriv, List<String> tokens, int window) {
+    if (deriv.start == -1)
+      return tokens;
+    List<String> newTokens = new ArrayList<>();
+    int startInd = Math.max(0, deriv.start - window);
+    int endInd = Math.min(tokens.size(), deriv.end + window);
+    newTokens.addAll(tokens.subList(startInd, deriv.start));
+    newTokens.add(deriv.cat);
+    newTokens.addAll(tokens.subList(deriv.end, endInd));
+    return newTokens;
+  }
+
+  private List<String> getAllNgrams(List<String> tokens, int n, Derivation deriv) {
+    List<String> ngrams = new ArrayList<>();
+    List<String> paddedTokens = new ArrayList<>();
+    if (deriv.start == -1) // floating, just add everything
+      paddedTokens.addAll(tokens);
+    else {
+      paddedTokens.addAll(tokens.subList(Math.max(0, deriv.start - n + 1), Math.min(tokens.size(), deriv.end + n - 1)));
+    }
+
+    for (int i = 0; i < paddedTokens.size() - n + 1; i++) {
+      List<String> current = new ArrayList<>(paddedTokens.subList(i, i + n));
+      ngrams.add(current.toString());
+    }
+    return ngrams;
+  }
+
+  private List<String> getAllSkipGrams(List<String> tokens, Derivation deriv) {
+    List<String> ngrams = new ArrayList<>();
+    List<String> paddedTokens = new ArrayList<>();
+    if (tokens.size() < 3)
+      return ngrams;
+
+    if (deriv.start == -1) // floating, just add everything
+      paddedTokens.addAll(tokens);
+    else
+      paddedTokens.addAll(tokens.subList(Math.max(0, deriv.start - 2), Math.min(tokens.size(), deriv.end + 2)));
+
+    for (int i = 0; i < tokens.size() - 2; i++) {
+      ngrams.add("[" + tokens.get(i).toString() + ", *, " + tokens.get(i + 2) + "]");
+    }
+    return ngrams;
+  }
+
+  private void addWindowFeatures(Example ex, Derivation deriv) {
+    if (!FeatureExtractor.containsDomain(":window"))
+      return;
+    if (deriv.rule != Rule.nullRule) {
+      deriv.addFeature(":window", abstractAnchors(deriv, ex.getTokens(), 1).toString());
+      deriv.addFeature(":window", abstractAnchors(deriv, ex.getTokens(), 2).toString());
+    }
+  }
+
+  private void addStatsFeatures(Example ex, Derivation deriv) {
+    if (!FeatureExtractor.containsDomain(":stats"))
+      return;
+    if (deriv.rule != Rule.nullRule) {
+      if (deriv.rule.isInduced())
+        deriv.addFeature(":stats", "induced");
+      else
+        deriv.addFeature(":stats", "core");
+
+      if (deriv.rule.source != null) {
+        deriv.addFeature(":stats", "cite", deriv.rule.source.cite);
+        if (deriv.rule.source.cite > 0)
+          deriv.addFeature(":stats", "has_cite");
+        else
+          deriv.addFeature(":stats", "no_cite");
+
+        if (deriv.rule.source.self > 0)
+          deriv.addFeature(":stats", "has_selfcite");
+        else
+          deriv.addFeature(":stats", "no_selfcite");
+
+        if (deriv.rule.source.align)
+          deriv.addFeature(":stats", "align");
+        else
+          deriv.addFeature(":stats", "no_align");
+
+        if (deriv.rule.getInfoTag("simple_packing") != -1.0)
+          deriv.addFeature(":stats", "simple_packing");
+        else
+          deriv.addFeature(":stats", "no_simple_packing");
+
+      }
+    }
+  }
+
+  private void addSocialFeatures(Example ex, Derivation deriv) {
+    if (!FeatureExtractor.containsDomain(":social"))
+      return;
+    if (deriv.rule != Rule.nullRule && deriv.rule.source != null) {
+      // everyone like a particular author
+      deriv.addFeature(":social", deriv.rule.source.uid);
+      // a particular user likes a particular author, perhaps himself
+      deriv.addFeature(":social", deriv.rule.source.uid + "::" + ex.id);
+      // the degree everyone likes to use their own rules
+      deriv.addFeature(":social", "isself::" + deriv.rule.source.uid.equals(ex.id));
+    }
+  }
+
+  // Add an indicator for each applied rule.
+  private void extractRuleFeatures(Example ex, Derivation deriv) {
+    if (!FeatureExtractor.containsDomain(":rule"))
+      return;
+    if (deriv.rule != Rule.nullRule) {
+      deriv.addFeature(":rule", "fire");
+      deriv.addFeature(":rule", deriv.rule.toString());
+    }
+  }
+
+  // Extract features on the linguistic information of the spanned (anchored)
+  // tokens.
+  // (Not applicable for floating rules)
+  private void extractSpanFeatures(Example ex, Derivation deriv) {
+    if (!FeatureExtractor.containsDomain(":span") || deriv.start == -1)
+      return;
+    deriv.addFeature(":span", "cat=" + deriv.cat + ":: len=" + (deriv.end - deriv.start));
+    deriv.addFeature(":span", "cat=" + deriv.cat + ":: " + ex.token(deriv.start) + "..." + ex.token(deriv.end - 1));
+  }
+
+}

src/edu/stanford/nlp/sempre/interactive/Definition.java

@@ -0,0 +1,11 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.List;
+
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Example;
+
+public class Definition {
+  public Definition(Example head, List<Derivation> chartList, List<Example> body) {
+  }
+}

src/edu/stanford/nlp/sempre/interactive/DefinitionAligner.java

@@ -0,0 +1,234 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Rule;
+import edu.stanford.nlp.sempre.interactive.GrammarInducer.ParseStatus;
+import fig.basic.LogInfo;
+import fig.basic.Option;
+
+/**
+ * Takes the definition and the head, then induce rules through alignment
+ * 
+ * @author sidaw
+ */
+
+public class DefinitionAligner {
+  public static class Options {
+    @Option(gloss = "categories that can serve as rules")
+    public Set<String> alignedCats = new HashSet<String>();
+    @Option(gloss = "phrase size")
+    public int phraseSize = 2;
+    @Option(gloss = "max length difference")
+    public int maxLengthDifference = 3;
+    @Option(gloss = "max set exclusion length")
+    public int maxSetExclusionLength = 2;
+    @Option(gloss = "max exact exclusion length")
+    public int maxExactExclusionLength = 5;
+    @Option(gloss = "window size")
+    public int windowSize = 1;
+
+    @Option(gloss = "strategies")
+    public Set<Strategies> strategies = Sets.newHashSet(Strategies.SetExclusion, Strategies.ExactExclusion);
+    @Option(gloss = "maximum matches")
+    public int maxMatches = 3;
+    @Option(gloss = "verbose")
+    public int verbose = 0;
+
+  }
+
+  public enum Strategies {
+    SetExclusion, ExactExclusion, cmdSet
+  };
+
+  public static Options opts = new Options();
+
+  public class Match {
+    @Override
+    public String toString() {
+      return "Match [deriv=" + deriv + ", start=" + start + ", end=" + end + "]";
+    }
+
+    public Match(Derivation def, int start, int end) {
+      deriv = def;
+      this.start = start;
+      this.end = end;
+      deriv.grammarInfo.start = start;
+      deriv.grammarInfo.end = end;
+    }
+
+    Derivation deriv;
+    int start;
+    int end;
+  }
+
+  List<String> headTokens;
+  List<String> defTokens;
+
+  public static List<Rule> getRules(List<String> head, List<String> def, Derivation deriv, List<Derivation> chartList) {
+    if (opts.verbose > 0)
+      LogInfo.logs("DefinitionAligner.chartList: %s", chartList);
+
+    DefinitionAligner aligner = new DefinitionAligner(head, def, deriv, chartList);
+
+    List<Rule> allAlignedRules = Lists.newArrayList();
+    if (opts.verbose > 0)
+      LogInfo.logs("DefinitionAligner.allMatches.size(): %d", aligner.allMatches.size());
+
+    for (int i = 0; i < aligner.allMatches.size() && i <= opts.maxMatches; i++) {
+      Match match = aligner.allMatches.get(i);
+
+      List<Derivation> filteredList = chartList.stream()
+          .filter(d -> d.start >= match.deriv.start && d.end <= match.deriv.end).collect(Collectors.toList());
+
+      // filter out core
+      List<Derivation> currentParses = chartList.stream().filter(d -> {
+        if (opts.verbose > 1)
+          LogInfo.logs("DefinitionAligner.chartList.d: %s", d);
+        return (d.start == match.start && d.end == match.end);
+      }).collect(Collectors.toList());
+
+      if (opts.verbose > 1)
+        LogInfo.logs("DefinitionAligner.Match: %s", match);
+      if (opts.verbose > 1)
+        LogInfo.logs("DefinitionAligner.currentParses: %s", currentParses);
+
+      if (GrammarInducer.getParseStatus(currentParses) != ParseStatus.Core) {
+        if (opts.verbose > 1)
+          LogInfo.logs("DefinitionAligner.NotCore: %s", currentParses);
+        GrammarInducer grammarInducer = new GrammarInducer(head, match.deriv, filteredList);
+        allAlignedRules.addAll(grammarInducer.getRules());
+      }
+    }
+    return allAlignedRules;
+  }
+
+  public List<Match> allMatches = new ArrayList<>();
+  private Map<String, List<Derivation>> chartMap;
+
+  public DefinitionAligner(List<String> headTokens, List<String> defTokens, Derivation def,
+      List<Derivation> chartList) {
+    this.headTokens = headTokens;
+    this.defTokens = defTokens;
+    this.chartMap = GrammarInducer.makeChartMap(chartList);
+    if (opts.verbose > 0)
+      LogInfo.logs("DefinitionAligner: head '%s' as body: '%s'", headTokens, defTokens);
+    if (Math.abs(headTokens.size() - defTokens.size()) >= 4)
+      return;
+    recursiveMatch(def);
+  }
+
+  void recursiveMatch(Derivation def) {
+    // LogInfo.logs("Considering (%d,%d): %s", def.start, def.end, def);
+    for (int start = 0; start < headTokens.size(); start++) {
+      for (int end = headTokens.size(); end > start; end--) {
+        // LogInfo.logs("Testing (%d,%d)", start, end);
+        if (end == headTokens.size() && start == 0)
+          continue;
+        if (isMatch(def, start, end)) {
+          if (opts.verbose > 0)
+            LogInfo.logs("Matched head(%d,%d)=%s with deriv(%d,%d)=%s: %s", start, end, headTokens.subList(start, end),
+                def.start, def.end, defTokens.subList(def.start, def.end), def);
+          allMatches.add(new Match(def, start, end));
+          return;
+        }
+      }
+    }
+
+    for (Derivation d : def.children) {
+      recursiveMatch(d);
+    }
+  }
+
+  boolean isMatch(Derivation def, int start, int end) {
+    if (def.start == -1 || def.end == -1)
+      return false;
+    if (chartMap.containsKey(GrammarInducer.catFormulaKey(def)))
+      return false;
+    if (opts.verbose > 0)
+      LogInfo.logs("checkingLengths (%d, %d) - (%d, %d)", start, end, def.start, def.end);
+    if (Math.abs((end - start) - (def.end - def.start)) >= opts.maxLengthDifference)
+      return false;
+    if (opts.strategies.contains(Strategies.ExactExclusion) && exactExclusion(def, start, end))
+      return true;
+    if (opts.strategies.contains(Strategies.SetExclusion) && setExclusion(def, start, end))
+      return true;
+    if (opts.strategies.contains(Strategies.cmdSet) && cmdSet(def, start, end))
+      return true;
+
+    return false;
+  }
+
+  private boolean setExclusion(Derivation def, int start, int end) {
+    // the span under consideration does not match anythign
+    if (end - start > opts.maxSetExclusionLength)
+      return false;
+    if (!headTokens.subList(start, end).stream().noneMatch(t -> defTokens.contains(t)))
+      return false;
+    if (!defTokens.subList(def.start, def.end).stream().noneMatch(t -> headTokens.contains(t)))
+      return false;
+
+    // everything before and afterwards are accounted for
+    if (!headTokens.subList(0, start).stream().allMatch(t -> defTokens.contains(t)))
+      return false;
+    if (!headTokens.subList(end, headTokens.size()).stream().allMatch(t -> defTokens.contains(t)))
+      return false;
+    return true;
+  }
+
+  private List<String> window(int lower, int upper, List<String> list) {
+    List<String> ret = new ArrayList<>();
+    for (int i = lower; i < upper; i++) {
+      if (i < 0 || i >= list.size())
+        ret.add("(*)");
+      else
+        ret.add(list.get(i));
+    }
+    return ret;
+  }
+
+  private boolean exactExclusion(Derivation def, int start, int end) {
+    if (opts.verbose > 0)
+      LogInfo.log("In exactExclusion");
+    if (end - start > opts.maxExactExclusionLength)
+      return false;
+
+    boolean prefixEq = window(start - opts.windowSize, start, headTokens)
+        .equals(window(def.start - opts.windowSize, def.start, defTokens));
+    boolean sufixEq = window(end, end + opts.windowSize, headTokens)
+        .equals(window(def.end, def.end + opts.windowSize, defTokens));
+    if (opts.verbose > 0)
+      LogInfo.logs("%b : %b", prefixEq, sufixEq);
+    if (opts.verbose > 0)
+      LogInfo.logs("(%d,%d)-head(%d,%d): %b %b %s %s", def.start, def.end, start, end, prefixEq, sufixEq,
+          window(end, end + opts.windowSize, headTokens), window(def.end, def.end + opts.windowSize, defTokens));
+    if (!prefixEq || !sufixEq)
+      return false;
+    if (headTokens.subList(start, end).equals(defTokens.subList(def.start, def.end)))
+      return false;
+
+    return true;
+  }
+
+  // exact match plus big
+  private boolean cmdSet(Derivation def, int start, int end) {
+    if (opts.verbose > 0)
+      LogInfo.log("In exactPlusBig");
+    // match only beginning and end
+    boolean cmdSet = (end == headTokens.size()) && (start > 0) && def.end == defTokens.size() && def.start > 0;
+    if (cmdSet && headTokens.subList(0, start).equals(defTokens.subList(0, start)))
+      return true;
+
+    return false;
+  }
+
+}

src/edu/stanford/nlp/sempre/interactive/GrammarInducer.java

@@ -0,0 +1,453 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.beust.jcommander.internal.Lists;
+import com.google.common.base.Function;
+
+import edu.stanford.nlp.sempre.ActionFormula;
+import edu.stanford.nlp.sempre.ConstantFn;
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Formula;
+import edu.stanford.nlp.sempre.Formulas;
+import edu.stanford.nlp.sempre.IdentityFn;
+import edu.stanford.nlp.sempre.LambdaFormula;
+import edu.stanford.nlp.sempre.Rule;
+import edu.stanford.nlp.sempre.SemanticFn;
+import edu.stanford.nlp.sempre.VariableFormula;
+import fig.basic.LispTree;
+import fig.basic.LogInfo;
+import fig.basic.Option;
+
+/**
+ * Takes two examples, and induce Rules
+ *
+ * @author sidaw
+ */
+
+public class GrammarInducer {
+  public static class Options {
+    @Option(gloss = "categories that can serve as rules")
+    public Set<String> filteredCats = new HashSet<String>();
+    @Option(gloss = "verbose")
+    public int verbose = 0;
+    @Option(gloss = "cats that never overlaps, and always save to replace")
+    public List<String> simpleCats = Lists.newArrayList("$Color", "$Number", "$Direction");
+    @Option(gloss = "use best packing")
+    public boolean useBestPacking = true;
+    @Option(gloss = "use simple packing")
+    public boolean useSimplePacking = true;
+    @Option(gloss = "maximum nonterminals in a rule")
+    public long maxNonterminals = 4;
+  }
+
+  public static Options opts = new Options();
+
+  private List<Rule> inducedRules = null;
+
+  List<String> headTokens;
+  String id;
+
+  public List<Derivation> matches;
+  Derivation def;
+
+  // induce rule is possible,
+  // otherwise set the correct status
+  public GrammarInducer(List<String> headTokens, Derivation def, List<Derivation> chartList) {
+    // grammarInfo start and end is used to indicate partial, when using aligner
+    boolean allHead = false;
+    if (def.grammarInfo.start == -1) {
+      def.grammarInfo.start = 0;
+      def.grammarInfo.end = headTokens.size();
+      allHead = true;
+    }
+
+    // dont want weird cat unary rules with strange semantics
+    if (headTokens == null || headTokens.isEmpty()) {
+      throw new RuntimeException("The head is empty, refusing to define.");
+    }
+    chartList.removeIf(d -> d.start == def.grammarInfo.start && d.end == def.grammarInfo.end);
+    this.def = def;
+
+    this.headTokens = headTokens;
+    int numTokens = headTokens.size();
+
+    this.matches = new ArrayList<>();
+    addMatches(def, makeChartMap(chartList));
+    Collections.reverse(this.matches);
+
+    inducedRules = new ArrayList<>();
+    if (allHead && opts.useSimplePacking) {
+      List<Derivation> filteredMatches = this.matches.stream().filter(d -> {
+        return opts.simpleCats.contains(d.cat) && d.allAnchored() && d.end - d.start == 1;
+      }).collect(Collectors.toList());
+
+      List<Derivation> packing = new ArrayList<>();
+      for (int i = 0; i <= headTokens.size(); i++) {
+        for (Derivation d : filteredMatches) {
+          if (d.start == i) {
+            packing.add(d);
+            break;
+          }
+        }
+      }
+
+      HashMap<String, String> formulaToCat = new HashMap<>();
+      packing.forEach(d -> formulaToCat.put(catFormulaKey(d), varName(d)));
+      buildFormula(def, formulaToCat);
+      List<Rule> simpleInduced = induceRules(packing, def);
+      for (Rule rule : simpleInduced) {
+        rule.addInfo("simple_packing", 1.0);
+        filterRule(rule);
+      }
+
+      if (opts.verbose > 1) {
+        LogInfo.logs("Simple Packing", chartList.size());
+        LogInfo.logs("chartList.size = %d", chartList.size());
+        LogInfo.log("Potential packings: ");
+        this.matches.forEach(d -> LogInfo.logs("%f: %s\t %s", d.getScore(), d.formula, d.allAnchored()));
+        LogInfo.logs("packing: %s", packing);
+        LogInfo.logs("formulaToCat: %s", formulaToCat);
+      }
+    }
+    if (opts.useBestPacking) {
+      List<Derivation> bestPacking = bestPackingDP(this.matches, numTokens);
+
+      HashMap<String, String> formulaToCat = new HashMap<>();
+      bestPacking.forEach(d -> formulaToCat.put(catFormulaKey(d), varName(d)));
+      buildFormula(def, formulaToCat);
+      for (Rule rule : induceRules(bestPacking, def)) {
+        if (rule.rhs.stream().allMatch(s -> Rule.isCat(s)))
+          continue;
+        filterRule(rule);
+      }
+
+      if (opts.verbose > 1) {
+        LogInfo.logs("chartList.size = %d", chartList.size());
+        LogInfo.log("Potential packings: ");
+        this.matches.forEach(d -> LogInfo.logs("%f: %s\t", d.getScore(), d.formula));
+        LogInfo.logs("BestPacking: %s", bestPacking);
+        LogInfo.logs("formulaToCat: %s", formulaToCat);
+      }
+    }
+
+  }
+
+  Set<String> RHSs = new HashSet<>();
+
+  private void filterRule(Rule rule) {
+    if (rule.isCatUnary()) {
+      LogInfo.logs("GrammarInducer.filterRule: not allowing CatUnary rules %s", rule.toString());
+      return;
+    }
+      
+    if (RHSs.contains(rule.rhs.toString())) {
+      LogInfo.logs("GrammarInducer.filterRule: already have %s", rule.toString());
+      return;
+    }
+    int numNT = 0;
+    for (String t : rule.rhs) {
+      if (Rule.isCat(t)) numNT++;
+    }
+    
+    if (numNT > GrammarInducer.opts.maxNonterminals ) {
+      LogInfo.logs("GrammarInducer.filterRule: too many nontermnimals (max %d) %s", GrammarInducer.opts.maxNonterminals, rule.rhs.toString());
+      return;
+    }
+    inducedRules.add(rule);
+    RHSs.add(rule.rhs.toString());
+  }
+
+  static Map<String, List<Derivation>> makeChartMap(List<Derivation> chartList) {
+    Map<String, List<Derivation>> chartMap = new HashMap<>();
+    for (Derivation d : chartList) {
+      List<Derivation> derivs = chartMap.get(catFormulaKey(d));
+      derivs = derivs != null ? derivs : new ArrayList<>();
+      derivs.add(d);
+      chartMap.put(catFormulaKey(d), derivs);
+    }
+    return chartMap;
+  }
+
+  // this is used to test for matches, same cat, same formula
+  // maybe cat needs to be more flexible
+  static String catFormulaKey(Derivation d) {
+    // return d.formula.toString();
+    return getNormalCat(d) + "::" + d.formula.toString();
+  }
+
+  private String varName(Derivation anchored) {
+    int s = def.grammarInfo.start;
+    return getNormalCat(anchored) + (anchored.start - s) + "_" + (anchored.end - s);
+  }
+
+  static private String getNormalCat(Derivation def) {
+    // return def.cat;
+    String cat = def.getCat();
+    if (cat.endsWith("s"))
+      return cat.substring(0, cat.length() - 1);
+    else
+      return cat;
+  }
+
+  // label the derivation tree with what it matches in chartList
+  private void addMatches(Derivation deriv, Map<String, List<Derivation>> chartMap) {
+    String key = catFormulaKey(deriv);
+    if (chartMap.containsKey(key)) {
+      deriv.grammarInfo.matches.addAll(chartMap.get(key));
+      deriv.grammarInfo.matched = true;
+      matches.addAll(chartMap.get(key));
+    }
+    for (Derivation d : deriv.children) {
+      addMatches(d, chartMap);
+    }
+  }
+
+  class Packing {
+    List<Derivation> packing;
+    double score;
+
+    public Packing(double score, List<Derivation> packing) {
+      this.score = score;
+      this.packing = packing;
+    }
+
+    @Override
+    public String toString() {
+      return this.score + ": " + this.packing.toString();
+    }
+  }
+
+  // the maximum starting index of every match that ends on or before end
+  private int blockingIndex(List<Derivation> matches, int end) {
+    return matches.stream().filter(d -> d.end <= end).map(d -> d.start).max((s1, s2) -> s1.compareTo(s2))
+        .orElse(Integer.MAX_VALUE / 2);
+  }
+
+  // start inclusive, end exclusive
+  private List<Derivation> bestPackingDP(List<Derivation> matches, int length) {
+    List<Packing> bestEndsAtI = new ArrayList<>(length + 1);
+    List<Packing> maximalAtI = new ArrayList<>(length + 1);
+    bestEndsAtI.add(new Packing(Double.NEGATIVE_INFINITY, new ArrayList<Derivation>()));
+    maximalAtI.add(new Packing(0.0, new ArrayList<Derivation>()));
+
+    @SuppressWarnings("unchecked")
+    List<Derivation>[] endsAtI = new ArrayList[length + 1];
+
+    for (Derivation d : matches) {
+      List<Derivation> derivs = endsAtI[d.end];
+      derivs = derivs != null ? derivs : new ArrayList<>();
+      derivs.add(d);
+      endsAtI[d.end] = derivs;
+    }
+
+    for (int i = 1; i <= length; i++) {
+      // the new maximal either uses a derivation that ends at i, plus a
+      // previous maximal
+      Packing bestOverall = new Packing(Double.NEGATIVE_INFINITY, new ArrayList<>());
+      Derivation bestDerivI = null;
+      if (endsAtI[i] != null) {
+        for (Derivation d : endsAtI[i]) {
+          double score = d.getScore() + maximalAtI.get(d.start).score;
+          if (score >= bestOverall.score) {
+            bestOverall.score = score;
+            bestDerivI = d;
+          }
+        }
+        List<Derivation> bestpacking = new ArrayList<>(maximalAtI.get(bestDerivI.start).packing);
+        bestpacking.add(bestDerivI);
+        bestOverall.packing = bestpacking;
+      }
+      bestEndsAtI.add(i, bestOverall);
+
+      // or it's a previous bestEndsAtI[j] for i-minLength+1 <= j < i
+      for (int j = blockingIndex(matches, i) + 1; j < i; j++) {
+        // LogInfo.dbgs("BlockingIndex: %d, j=%d, i=%d", blockingIndex(matches,
+        // i), j, i);
+        if (bestEndsAtI.get(j).score >= bestOverall.score)
+          bestOverall = bestEndsAtI.get(j);
+      }
+      if (opts.verbose > 1)
+        LogInfo.logs("maximalAtI[%d] = %f: %s, BlockingIndex: %d", i, bestOverall.score, bestOverall.packing,
+            blockingIndex(matches, i));
+      if (bestOverall.score > Double.NEGATIVE_INFINITY)
+        maximalAtI.add(i, bestOverall);
+      else {
+        maximalAtI.add(i, new Packing(0, new ArrayList<>()));
+      }
+    }
+    return maximalAtI.get(length).packing;
+  }
+
+  public List<Rule> getRules() {
+    return inducedRules;
+  }
+
+  private List<Rule> induceRules(List<Derivation> packings, Derivation defDeriv) {
+    List<String> RHS = getRHS(defDeriv, packings);
+    SemanticFn sem = getSemantics(defDeriv, packings);
+    String cat = getNormalCat(defDeriv);
+    Rule inducedRule = new Rule(cat, RHS, sem);
+    inducedRule.addInfo("induced", 1.0);
+    inducedRule.addInfo("anchored", 1.0);
+    List<Rule> inducedRules = new ArrayList<>();
+    if (!inducedRule.isCatUnary()) {
+      inducedRules.add(inducedRule);
+    }
+    return inducedRules;
+  }
+
+  // populate grammarInfo.formula, replacing everything that can be replaced
+  private void buildFormula(Derivation deriv, Map<String, String> replaceMap) {
+    // LogInfo.logs("BUILDING %s at (%d,%d) %s", deriv, deriv.start, deriv.end,
+    // catFormulaKey(deriv));
+    if (replaceMap.containsKey(catFormulaKey(deriv))) {
+      // LogInfo.logs("Found match %s, %s, %s", catFormulaKey(deriv),
+      // replaceMap, deriv);
+      deriv.grammarInfo.formula = new VariableFormula(replaceMap.get(catFormulaKey(deriv)));
+      return;
+    }
+    if (deriv.children.size() == 0) {
+      deriv.grammarInfo.formula = deriv.formula;
+    }
+
+    for (Derivation c : deriv.children) {
+      buildFormula(c, replaceMap);
+      // deriv.grammarInfo.start = Math.min(deriv.grammarInfo.start,
+      // c.grammarInfo.start);
+      // deriv.grammarInfo.end = Math.max(deriv.grammarInfo.end,
+      // c.grammarInfo.end);
+    }
+    Rule rule = deriv.rule;
+    List<Derivation> args = deriv.children;
+
+    // cant use the standard DerivationStream because formula is final
+    if (rule == null || rule.sem == null) {
+      deriv.grammarInfo.formula = deriv.formula;
+    } else if (rule.sem instanceof ApplyFn) {
+      Formula f = Formulas.fromLispTree(((ApplyFn) rule.sem).formula.toLispTree());
+      for (Derivation arg : args) {
+        if (!(f instanceof LambdaFormula))
+          throw new RuntimeException("Expected LambdaFormula, but got " + f);
+        Formula after = renameBoundVars(f, new HashSet<>());
+        // LogInfo.logs("renameBoundVar %s === %s", after, f);
+        f = Formulas.lambdaApply((LambdaFormula) after, arg.grammarInfo.formula);
+      }
+      deriv.grammarInfo.formula = f;
+    } else if (rule.sem instanceof IdentityFn) {
+      deriv.grammarInfo.formula = args.get(0).grammarInfo.formula;
+    } else if (rule.sem instanceof BlockFn) {
+      deriv.grammarInfo.formula = new ActionFormula(((BlockFn) rule.sem).mode,
+          args.stream().map(d -> d.grammarInfo.formula).collect(Collectors.toList()));
+    } else {
+      deriv.grammarInfo.formula = deriv.formula;
+    }
+    // LogInfo.logs("BUILT %s for %s", deriv.grammarInfo.formula,
+    // deriv.formula);
+    // LogInfo.log("built " + deriv.grammarInfo.formula);
+  }
+
+  private String newName(String s) {
+    return s.endsWith("_") ? s : s + "_";
+  }
+
+  private Formula renameBoundVars(Formula formula, Set<String> boundvars) {
+    if (formula instanceof LambdaFormula) {
+      LambdaFormula f = (LambdaFormula) formula;
+      boundvars.add(f.var);
+      return new LambdaFormula(newName(f.var), renameBoundVars(f.body, boundvars));
+    } else {
+      Formula after = formula.map(new Function<Formula, Formula>() {
+        @Override
+        public Formula apply(Formula formula) {
+          if (formula instanceof VariableFormula) { // Replace variable
+            String name = ((VariableFormula) formula).name;
+            if (boundvars.contains(name))
+              return new VariableFormula(newName(name));
+            else
+              return formula;
+          }
+          return null;
+        }
+      });
+      return after;
+    }
+  }
+
+  private SemanticFn getSemantics(final Derivation def, List<Derivation> packings) {
+    Formula baseFormula = def.grammarInfo.formula;
+    if (opts.verbose > 0)
+      LogInfo.logs("getSemantics %s", baseFormula);
+    if (packings.size() == 0) {
+      SemanticFn constantFn = new ConstantFn();
+      LispTree newTree = LispTree.proto.newList();
+      newTree.addChild("ConstantFn");
+      newTree.addChild(baseFormula.toLispTree());
+      constantFn.init(newTree);
+      return constantFn;
+    }
+
+    for (int i = packings.size() - 1; i >= 0; i--) {
+      baseFormula = new LambdaFormula(varName(packings.get(i)), Formulas.fromLispTree(baseFormula.toLispTree()));
+    }
+    SemanticFn applyFn = new ApplyFn();
+    LispTree newTree = LispTree.proto.newList();
+    newTree.addChild("interactive.ApplyFn");
+    newTree.addChild(baseFormula.toLispTree());
+    applyFn.init(newTree);
+    return applyFn;
+  }
+
+  private List<String> getRHS(Derivation def, List<Derivation> packings) {
+    List<String> rhs = new ArrayList<>(headTokens);
+    for (Derivation deriv : packings) {
+      // LogInfo.logs("got (%d,%d):%s:%s", deriv.start, deriv.end,
+      // deriv.formula, deriv.cat);
+      rhs.set(deriv.start, getNormalCat(deriv));
+      for (int i = deriv.start + 1; i < deriv.end; i++) {
+        rhs.set(i, null);
+      }
+    }
+    return rhs.subList(def.grammarInfo.start, def.grammarInfo.end).stream().filter(s -> s != null)
+        .collect(Collectors.toList());
+  }
+
+  public static enum ParseStatus {
+    Nothing, // nothing at all parses in the utterance
+    /// Float, // something parse, no longer used.
+    Induced, // redefining known utterance
+    Core;
+
+    public static ParseStatus fromString(String status) {
+      for (ParseStatus c : ParseStatus.values())
+        if (c.name().equalsIgnoreCase(status))
+          return c;
+      return null;
+    } // define known utterance in core, should reject
+  }
+
+  public static ParseStatus getParseStatus(Example ex) {
+    return getParseStatus(ex.predDerivations);
+  }
+
+  public static ParseStatus getParseStatus(List<Derivation> derivs) {
+    if (derivs.size() > 0) {
+      for (Derivation deriv : derivs) {
+        if (deriv.allAnchored()) {
+          return ParseStatus.Core;
+        }
+      }
+      return ParseStatus.Induced;
+    }
+    // could check the chart here set partial, but no need for now
+    return ParseStatus.Nothing;
+  }
+}

src/edu/stanford/nlp/sempre/interactive/InteractiveBeamParser.java

@@ -0,0 +1,577 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.LinkedHashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.beust.jcommander.internal.Lists;
+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableList;
+
+import edu.stanford.nlp.sempre.ChartParserState;
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.DerivationStream;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Formula;
+import edu.stanford.nlp.sempre.Json;
+import edu.stanford.nlp.sempre.Params;
+import edu.stanford.nlp.sempre.Parser;
+import edu.stanford.nlp.sempre.ParserState;
+import edu.stanford.nlp.sempre.Rule;
+import edu.stanford.nlp.sempre.SemanticFn;
+import edu.stanford.nlp.sempre.Trie;
+import fig.basic.Evaluation;
+import fig.basic.IOUtils;
+import fig.basic.IntRef;
+import fig.basic.LogInfo;
+import fig.basic.Option;
+import fig.basic.SetUtils;
+import fig.basic.StopWatch;
+import fig.basic.StopWatchSet;
+import fig.exec.Execution;
+
+/**
+ * A modified version of the BeamParser, with consideration for use in the interactive setting
+ * 
+ * @author Percy Liang, sidaw
+ */
+public class InteractiveBeamParser extends Parser {
+  public static class Options {
+    @Option
+    public int maxNewTreesPerSpan = Integer.MAX_VALUE;
+    @Option
+    public FloatStrategy floatStrategy = FloatStrategy.Never;
+    @Option(gloss = "track these categories")
+    public List<String> trackedCats;
+  }
+
+  public enum FloatStrategy {
+    Always, Never, NoParse
+  };
+
+  public static Options opts = new Options();
+
+  Trie trie; // For non-cat-unary rules
+  // so that duplicated rules are never added
+  Set<Rule> allRules;
+  List<Rule> interactiveCatUnaryRules;
+  public InteractiveBeamParser(Spec spec) {
+    super(spec);
+    if (opts.trackedCats != null) {
+      opts.trackedCats = opts.trackedCats.stream().map(s -> "$" + s).collect(Collectors.toList());
+      LogInfo.logs("Mapped trackedCats to: %s", opts.trackedCats);
+    }
+    interactiveCatUnaryRules = new LinkedList<>(super.catUnaryRules);
+    allRules = new LinkedHashSet<>(super.catUnaryRules);
+    // Index the non-cat-unary rules
+    trie = new Trie();
+    for (Rule rule : grammar.getRules()) {
+      addRule(rule);
+    }
+    if (Parser.opts.visualizeChartFilling)
+      this.chartFillOut = IOUtils.openOutAppendEasy(Execution.getFile("chartfill"));
+  }
+
+  @Override
+  public synchronized void addRule(Rule rule) {
+    if (allRules.contains(rule))
+      return;
+    
+    allRules.add(rule);
+
+    if (!rule.isCatUnary()) {
+      trie.add(rule);
+    } else {
+      interactiveCatUnaryRules.add(rule);
+    }
+  }
+  
+  @Override
+  public List<Rule> getCatUnaryRules() {
+    return interactiveCatUnaryRules;
+  }
+  
+  // for grammar induction, just need the formula, do not execute
+  public InteractiveBeamParserState parseWithoutExecuting(Params params, Example ex, boolean computeExpectedCounts) {
+    // Parse
+    StopWatch watch = new StopWatch();
+    watch.start();
+    InteractiveBeamParserState state = new InteractiveBeamParserState(this, params, ex);
+    state.infer();
+    watch.stop();
+    state.parseTime = watch.getCurrTimeLong();
+
+    ex.predDerivations = state.predDerivations;
+    Derivation.sortByScore(ex.predDerivations);
+    // Clean up temporary state used during parsing
+    return state;
+  }
+
+
+  @Override
+  public ParserState newParserState(Params params, Example ex, boolean computeExpectedCounts) {
+    InteractiveBeamParserState coarseState = null;
+    if (Parser.opts.coarsePrune) {
+      LogInfo.begin_track("Parser.coarsePrune");
+      coarseState = new InteractiveBeamParserState(this, params, ex, computeExpectedCounts,
+          InteractiveBeamParserState.Mode.bool, null);
+      coarseState.infer();
+      coarseState.keepTopDownReachable();
+      LogInfo.end_track();
+    }
+    return new InteractiveBeamParserState(this, params, ex, computeExpectedCounts, InteractiveBeamParserState.Mode.full,
+        coarseState);
+  }
+}
+
+/**
+ * Stores BeamFloatingParser information about parsing a particular example. The
+ * actual parsing code lives here.
+ *
+ * @author Percy Liang
+ * @author Roy Frostig
+ * @author sidaw
+ */
+class InteractiveBeamParserState extends ChartParserState {
+  public final Mode mode;
+
+  // Modes:
+  // 1) Bool: just check if cells (cat, start, end) are reachable (to prune
+  // chart)
+  // 2) Full: compute everything
+  public enum Mode {
+    bool, full
+  }
+
+  private final InteractiveBeamParser parser;
+  private final InteractiveBeamParserState coarseState; // Used to prune
+  private final boolean execute;
+
+  public List<Derivation> chartList;
+
+  public InteractiveBeamParserState(InteractiveBeamParser parser, Params params, Example ex) {
+    super(parser, params, ex, false);
+    this.parser = parser;
+    this.mode = Mode.full;
+    this.coarseState = null;
+    this.execute = false;
+  }
+
+  public InteractiveBeamParserState(InteractiveBeamParser parser, Params params, Example ex, boolean computeExpectedCounts,
+      Mode mode, InteractiveBeamParserState coarseState) {
+    super(parser, params, ex, computeExpectedCounts);
+    this.parser = parser;
+    this.mode = mode;
+    this.coarseState = coarseState;
+    this.execute = true;
+  }
+
+  @Override
+  public void infer() {
+    if (numTokens == 0)
+      return;
+
+    if (parser.verbose(2))
+      LogInfo.begin_track("ParserState.infer");
+
+    // Base case
+    for (Derivation deriv : gatherTokenAndPhraseDerivations()) {
+      featurizeAndScoreDerivation(deriv);
+      addToChart(deriv);
+    }
+
+    // Recursive case
+    for (int len = 1; len <= numTokens; len++)
+      for (int i = 0; i + len <= numTokens; i++)
+        build(i, i + len);
+
+    if (parser.verbose(2))
+      LogInfo.end_track();
+
+    // Visualize
+    if (parser.chartFillOut != null && Parser.opts.visualizeChartFilling && this.mode != Mode.bool) {
+      parser.chartFillOut.println(
+          Json.writeValueAsStringHard(new ChartFillingData(ex.id, chartFillingList, ex.utterance, ex.numTokens())));
+      parser.chartFillOut.flush();
+    }
+
+    setPredDerivations();
+
+    for (Derivation deriv : predDerivations) {
+      deriv.getAnchoredTokens();
+    }
+
+    this.chartList = this.collectChart();
+
+    boolean parseFloat = false;
+    if (InteractiveBeamParser.opts.floatStrategy == InteractiveBeamParser.FloatStrategy.Always)
+      parseFloat = true;
+    else if (InteractiveBeamParser.opts.floatStrategy == InteractiveBeamParser.FloatStrategy.NoParse)
+      parseFloat = predDerivations.size() == 0;
+    else
+      parseFloat = false;
+
+    if (mode == Mode.full) {
+      // Compute gradient with respect to the predicted derivations
+      if (this.execute)
+        ensureExecuted();
+      if (computeExpectedCounts) {
+        expectedCounts = new HashMap<>();
+        ParserState.computeExpectedCounts(predDerivations, expectedCounts);
+      }
+    }
+
+    /* If Beam Parser failed to find derivations, try a floating parser */
+    if (parseFloat) {
+      /*
+       * For every base span of the chart, add the derivations from nothing
+       * rules
+       */
+      List<Rule> nothingRules = new ArrayList<Rule>();
+      for (Rule rule : parser.grammar.getRules())
+        if (rule.isFloating() && rule.rhs.size() == 1 && rule.isRhsTerminals())
+          nothingRules.add(rule);
+      for (int i = 0; i < numTokens; i++)
+        for (Rule rule : nothingRules)
+          applyRule(i, i + 1, rule, chart[i][i + 1].get("$TOKEN"));
+
+      /* Traverse the chart bottom up */
+      for (int len = 1; len <= numTokens; len++) {
+        for (int i = 0; i + len <= numTokens; i++) {
+          buildFloating(i, i + len);
+        }
+      }
+
+      /* Add unique derivations to predDerivations */
+      List<Derivation> rootDerivs = chart[0][numTokens].get("$FROOT");
+      if (rootDerivs == null)
+        rootDerivs = new ArrayList<Derivation>(Derivation.emptyList);
+
+      List<Derivation> actionDerivs = new ArrayList<Derivation>(Derivation.emptyList);
+      if (actionDerivs != null) {
+        Set<Formula> formulas = new HashSet<Formula>();
+        for (Derivation d : rootDerivs) {
+          Formula f = d.getFormula();
+          if (!formulas.contains(f)) {
+            formulas.add(f);
+            predDerivations.add(d);
+          }
+        }
+      }
+    }
+  }
+
+  private List<Derivation> collectChart() {
+    List<Derivation> chartList = Lists.newArrayList();
+    for (int len = 1; len <= numTokens; ++len) {
+      for (int i = 0; i + len <= numTokens; ++i) {
+        for (String cat : chart[i][i + len].keySet()) {
+          if (Rule.specialCats.contains(cat))
+            continue;
+          chartList.addAll(chart[i][i + len].get(cat));
+        }
+      }
+    }
+    return chartList;
+  }
+
+  // Create all the derivations for the span [start, end).
+  protected void build(int start, int end) {
+    applyNonCatUnaryRules(start, end, start, parser.trie, new ArrayList<Derivation>(), new IntRef(0));
+
+    Set<String> cellsPruned = new HashSet<>();
+    applyCatUnaryRules(start, end, cellsPruned);
+
+    for (Map.Entry<String, List<Derivation>> entry : chart[start][end].entrySet())
+      pruneCell(cellsPruned, entry.getKey(), start, end, entry.getValue());
+  }
+
+  private static String cellString(String cat, int start, int end) {
+    return cat + ":" + start + ":" + end;
+  }
+
+  // Return number of new derivations added
+  private int applyRule(int start, int end, Rule rule, List<Derivation> children) {
+    if (Parser.opts.verbose >= 5)
+      LogInfo.logs("applyRule %s %s %s %s", start, end, rule, children);
+    try {
+      if (mode == Mode.full) {
+        StopWatchSet.begin(rule.getSemRepn());
+        DerivationStream results = rule.sem.call(ex,
+            new SemanticFn.CallInfo(rule.lhs, start, end, rule, ImmutableList.copyOf(children)));
+        StopWatchSet.end();
+        while (results.hasNext()) {
+          Derivation newDeriv = results.next();
+          featurizeAndScoreDerivation(newDeriv);
+          addToChart(newDeriv);
+        }
+        return results.estimatedSize();
+      } else if (mode == Mode.bool) {
+        Derivation deriv = new Derivation.Builder().cat(rule.lhs).start(start).end(end).rule(rule)
+            .children(ImmutableList.copyOf(children)).formula(Formula.nullFormula).createDerivation();
+        addToChart(deriv);
+        return 1;
+      } else {
+        throw new RuntimeException("Invalid mode");
+      }
+    } catch (Exception e) {
+      LogInfo.errors("Composition failed: rule = %s, children = %s", rule, children);
+      e.printStackTrace();
+      throw new RuntimeException(e);
+    }
+  }
+
+  // Don't prune the same cell more than once.
+  protected void pruneCell(Set<String> cellsPruned, String cat, int start, int end, List<Derivation> derivations) {
+    String cell = cellString(cat, start, end);
+    if (cellsPruned.contains(cell))
+      return;
+
+    cellsPruned.add(cell);
+    pruneCell(cell, derivations);
+  }
+
+  private boolean canBeRoot(int start, int end) {return start==0 && end==numTokens;};
+  // Apply all unary rules with RHS category.
+  // Before applying each unary rule (rule.lhs -> rhsCat), we can prune the cell
+  // of rhsCat
+  // because we assume acyclicity, so rhsCat's cell will never grow.
+  private void applyCatUnaryRules(int start, int end, Set<String> cellsPruned) {
+    for (Rule rule : parser.getCatUnaryRules()) {
+      if (!coarseAllows(rule.lhs, start, end))
+        continue;
+      if (rule.lhs.equals(Rule.rootCat) && !canBeRoot(start, end))
+        continue;
+      String rhsCat = rule.rhs.get(0);
+      List<Derivation> derivations = chart[start][end].get(rhsCat);
+      if (Parser.opts.verbose >= 5)
+        LogInfo.logs("applyCatUnaryRules %s %s %s %s", start, end, rule, chart[start][end]);
+      if (derivations == null)
+        continue;
+
+      // Prune before applying rules to eliminate cruft!
+      pruneCell(cellsPruned, rhsCat, start, end, derivations);
+
+      for (Derivation deriv : derivations) {
+        applyRule(start, end, rule, Collections.singletonList(deriv));
+      }
+    }
+  }
+
+  // Strategy: walk along the input on span (start:end) and traverse the trie
+  // to get the list of the rules that could apply by matching the RHS.
+  // start:end: span we're dealing with.
+  // i: current token position
+  // node: contains a link to the RHS that could apply.
+  // children: the derivations that't we're building up.
+  // numNew: Keep track of number of new derivations created
+  private void applyNonCatUnaryRules(int start, int end, int i, Trie node, ArrayList<Derivation> children,
+      IntRef numNew) {
+    if (node == null)
+      return;
+    if (!coarseAllows(node, start, end))
+      return;
+
+    if (Parser.opts.verbose >= 5) {
+      LogInfo.logs("applyNonCatUnaryRules(start=%d, end=%d, i=%d, children=[%s], %s rules)", start, end, i,
+          Joiner.on(", ").join(children), node.rules.size());
+    }
+
+    // Base case: our fencepost has walked to the end of the span, so
+    // apply the rule on all the children gathered during the walk.
+    if (i == end) {
+      Iterator<Rule> ruleIterator = node.rules.iterator();
+      while (ruleIterator.hasNext()) {
+        Rule rule = ruleIterator.next();
+        if (coarseAllows(rule.lhs, start, end)) {
+          numNew.value += applyRule(start, end, rule, children);
+        }
+      }
+      return;
+    }
+
+    // Advance terminal token
+    applyNonCatUnaryRules(start, end, i + 1, node.next(ex.token(i)), children, numNew);
+
+    // Advance non-terminal category
+    for (int j = i + 1; j <= end; j++) {
+      for (Map.Entry<String, List<Derivation>> entry : chart[i][j].entrySet()) {
+        Trie nextNode = node.next(entry.getKey());
+        for (Derivation arg : entry.getValue()) {
+          children.add(arg);
+          applyNonCatUnaryRules(start, end, j, nextNode, children, numNew);
+          children.remove(children.size() - 1);
+          if (mode != Mode.full)
+            break; // Only need one hypothesis
+          if (numNew.value >= InteractiveBeamParser.opts.maxNewTreesPerSpan)
+            return;
+        }
+      }
+    }
+  }
+
+  /* For each span, apply applicable floating rules */
+  protected void buildFloating(int start, int end) {
+    for (Rule rule : parser.grammar.getRules()) {
+      if (!rule.isFloating() || !coarseAllows(rule.lhs, start, end))
+        continue;
+
+      if (rule.rhs.size() == 1) {
+        /* Apply cat unary rules simply */
+        String rhsCat = rule.rhs.get(0);
+        List<Derivation> derivs = chart[start][end].get(rhsCat);
+
+        if (derivs == null)
+          continue;
+
+        for (Derivation deriv : derivs)
+          applyRule(start, end, rule, Collections.singletonList(deriv));
+      } else {
+        /* Apply non-cat unary rules by traversing through the subspans */
+        int derivsCreated = 0;
+        for (int i = start + 1; i < end; i++) {
+          derivsCreated += applyFloatingRule(rule, start, end, chart[start][i], chart[i][end]);
+          derivsCreated += applyFloatingRule(rule, start, end, chart[i][end], chart[start][i]);
+        }
+
+        /* If no derivs created, propagate up */
+        if (derivsCreated == 0) {
+          copyDerivs(chart[start][end - 1], chart[start][end]);
+          if (start != numTokens - 1)
+            copyDerivs(chart[start + 1][end], chart[start][end]);
+        }
+      }
+    }
+    // test prune
+    Set<String> cellsPruned = new HashSet<>();
+    for (Map.Entry<String, List<Derivation>> entry : chart[start][end].entrySet())
+      pruneCell(cellsPruned, entry.getKey(), start, end, entry.getValue());
+  }
+
+  protected int applyFloatingRule(Rule rule, int start, int end, Map<String, List<Derivation>> first,
+      Map<String, List<Derivation>> second) {
+    List<Derivation> derivs1 = first.get(rule.rhs.get(0));
+    List<Derivation> derivs2 = second.get(rule.rhs.get(1));
+
+    if (derivs1 == null || derivs2 == null)
+      return 0;
+
+    int derivsCreated = 0;
+
+    for (Derivation deriv1 : derivs1) {
+      for (Derivation deriv2 : derivs2) {
+        List<Derivation> children = new ArrayList<Derivation>();
+        children.add(deriv1);
+        children.add(deriv2);
+        derivsCreated += applyRule(start, end, rule, children);
+      }
+    }
+
+    return derivsCreated;
+  }
+
+  protected void copyDerivs(Map<String, List<Derivation>> source, Map<String, List<Derivation>> dest) {
+    if (source == null || dest == null)
+      return;
+
+    for (String cat : source.keySet()) {
+      List<Derivation> derivations = dest.get(cat);
+      if (derivations == null)
+        dest.put(cat, derivations = new ArrayList<>());
+
+      /* add only if the formula not already present to ensure no duplicates */
+      Set<Formula> formulas = new HashSet<Formula>();
+      for (Derivation deriv : derivations)
+        formulas.add(deriv.formula);
+
+      for (Derivation deriv : source.get(cat)) {
+        if (!formulas.contains(deriv.formula)) {
+          derivations.add(deriv);
+          formulas.add(deriv.formula);
+        }
+      }
+    }
+  }
+
+  protected void addDerivs(List<Derivation> source, List<Derivation> dest) {
+    if (dest == null || source == null)
+      return;
+    dest.addAll(source);
+  }
+
+  // -- Coarse state pruning --
+
+  // Remove any (cat, start, end) which isn't reachable from the
+  // (Rule.rootCat, 0, numTokens)
+  public void keepTopDownReachable() {
+    if (numTokens == 0)
+      return;
+
+    Set<String> reachable = new HashSet<>();
+    collectReachable(reachable, Rule.rootCat, 0, numTokens);
+
+    // Remove all derivations associated with (cat, start, end) that aren't
+    // reachable.
+    for (int start = 0; start < numTokens; start++) {
+      for (int end = start + 1; end <= numTokens; end++) {
+        List<String> toRemoveCats = new LinkedList<>();
+        for (String cat : chart[start][end].keySet()) {
+          String key = catStartEndKey(cat, start, end);
+          if (!reachable.contains(key)) {
+            toRemoveCats.add(cat);
+          }
+        }
+        Collections.sort(toRemoveCats);
+        for (String cat : toRemoveCats) {
+          if (parser.verbose(4)) {
+            LogInfo.logs("Pruning chart %s(%s,%s)", cat, start, end);
+          }
+          chart[start][end].remove(cat);
+        }
+      }
+    }
+  }
+
+  private void collectReachable(Set<String> reachable, String cat, int start, int end) {
+    String key = catStartEndKey(cat, start, end);
+    if (reachable.contains(key))
+      return;
+
+    if (!chart[start][end].containsKey(cat)) {
+      // This should only happen for the root when there are no parses.
+      return;
+    }
+
+    reachable.add(key);
+    for (Derivation deriv : chart[start][end].get(cat)) {
+      for (Derivation subderiv : deriv.children) {
+        collectReachable(reachable, subderiv.cat, subderiv.start, subderiv.end);
+      }
+    }
+  }
+
+  private String catStartEndKey(String cat, int start, int end) {
+    return cat + ":" + start + ":" + end;
+  }
+
+  // For pruning with the coarse state
+  protected boolean coarseAllows(Trie node, int start, int end) {
+    if (coarseState == null)
+      return true;
+    return SetUtils.intersects(node.cats, coarseState.chart[start][end].keySet());
+  }
+
+  protected boolean coarseAllows(String cat, int start, int end) {
+    if (coarseState == null)
+      return true;
+    return coarseState.chart[start][end].containsKey(cat);
+  }
+}

src/edu/stanford/nlp/sempre/interactive/InteractiveMaster.java

@@ -0,0 +1,343 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.io.PrintWriter;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import com.google.common.collect.Lists;
+
+import edu.stanford.nlp.sempre.Builder;
+import edu.stanford.nlp.sempre.ContextValue;
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Formula;
+import edu.stanford.nlp.sempre.Formulas;
+import edu.stanford.nlp.sempre.Master;
+import edu.stanford.nlp.sempre.Params;
+import edu.stanford.nlp.sempre.Parser;
+import edu.stanford.nlp.sempre.ParserState;
+import edu.stanford.nlp.sempre.Rule;
+import edu.stanford.nlp.sempre.RuleSource;
+import edu.stanford.nlp.sempre.Session;
+import fig.basic.IOUtils;
+import fig.basic.LispTree;
+import fig.basic.LogInfo;
+import fig.basic.Option;
+import fig.basic.Ref;
+
+/**
+ * An InteractiveMaster supports interactive commands, and grammar induction
+ * methods.
+ */
+public class InteractiveMaster extends Master {
+  public static class Options {
+    @Option(gloss = "Write out new grammar rules")
+    public String intOutputPath;
+    @Option(gloss = "each session gets a different model with its own parameters")
+    public boolean independentSessions = false;
+    @Option(gloss = "number of utterances to return for autocomplete")
+    public int autocompleteCount = 5;
+    @Option(gloss = "only allow interactive commands")
+    public boolean onlyInteractive = false;
+
+    @Option(gloss = "try partial matches")
+    public boolean useAligner = true;
+
+    @Option(gloss = "use the best formula when no match or not provided")
+    public int maxSequence = 20;
+    @Option(gloss = "path to the citations")
+    public int maxChars = 200;
+    
+    @Option(gloss = "allow regular commands specified in Master")
+    public boolean allowRegularCommands = false;
+  }
+
+  public static Options opts = new Options();
+
+  public InteractiveMaster(Builder builder) {
+    super(builder);
+  }
+
+  @Override
+  protected void printHelp() {
+    // interactive commands
+    LogInfo.log("Interactive commands");
+    LogInfo.log(
+        "  (:def head [[body1,bodyformula1],[body2,bodyformula2]]): provide a definition for the original utterance");
+    LogInfo.log("  (:q |utterance|): provide a definition for the original utterance");
+    LogInfo.log("  (:accept |formula1| |formula2|): accept any derivation with those corresponding formula");
+    LogInfo.log("  (:reject |formula1| |formula2|): reject any derivations with those corresponding formula");
+    LogInfo.log("Main commands:");
+    super.printHelp();
+  }
+
+  @Override
+  public void runServer() {
+    InteractiveServer server = new InteractiveServer(this);
+    server.run();
+  }
+
+  @Override
+  public Response processQuery(Session session, String line) {
+    LogInfo.begin_track("InteractiveMaster.handleQuery");
+    LogInfo.logs("session %s", session.id);
+    LogInfo.logs("query %s", line);
+    line = line.trim();
+    Response response = new Response();
+    if (line.startsWith("(:"))
+      handleCommand(session, line, response);
+    else if (line.startsWith("(") && opts.allowRegularCommands || session.id.equals("stdin"))
+      super.processQuery(session, line);
+    else
+      handleCommand(session, String.format("(:q \"%s\")", line), response);
+    LogInfo.end_track();
+    return response;
+  }
+
+  void handleCommand(Session session, String line, Response response) {
+    LispTree tree = LispTree.proto.parseFromString(line);
+    tree = builder.grammar.applyMacros(tree);
+
+    String command = tree.child(0).value;
+    QueryStats stats = new QueryStats(response, command);
+    // Start of interactive commands
+    if (command.equals(":q")) {
+      // Create example
+      String utt = tree.children.get(1).value;
+      Example ex = exampleFromUtterance(utt, session);
+
+      if (!utteranceAllowed(ex, response)) {
+        stats.error("utterance_too_expensive");
+        // returns with size and error message
+        return;
+      }
+
+      builder.parser.parse(builder.params, ex, false);
+
+      stats.size(ex.predDerivations != null ? ex.predDerivations.size() : 0);
+      stats.status(InteractiveUtils.getParseStatus(ex));
+
+      LogInfo.logs("parse stats: %s", response.stats);
+      response.ex = ex;
+    } else if (command.equals(":qdbg")) {
+      // Create example
+      String utt = tree.children.get(1).value;
+      Example ex = exampleFromUtterance(utt, session);
+
+      builder.parser.parse(builder.params, ex, false);
+
+      Derivation.opts.showCat = true;
+      Derivation.opts.showRules = true;
+      for (Derivation d : ex.predDerivations) {
+        response.lines.add(d.toLispTree().toString());
+      }
+      Derivation.opts.showCat = false;
+      Derivation.opts.showRules = false;
+      response.ex = ex;
+    } else if (command.equals(":reject")) {
+      stats.put("rejectSize", tree.children.size());
+    } else if (command.equals(":accept")) {
+      String utt = tree.children.get(1).value;
+      List<Formula> targetFormulas = new ArrayList<>();
+      try {
+        targetFormulas = tree.children.subList(2, tree.children.size()).stream()
+            .map(t -> Formulas.fromLispTree(LispTree.proto.parseFromString(t.value))).collect(Collectors.toList());
+      } catch (Exception e) {
+        e.printStackTrace();
+        response.lines.add("cannot accept formula: ");
+      }
+
+      Example ex = exampleFromUtterance(utt, session);
+      response.ex = ex;
+
+      // Parse!
+      ((InteractiveBeamParser)builder.parser).parseWithoutExecuting(builder.params, ex, false);
+
+      int rank = -1;
+      Derivation match = null;
+      for (int i = 0; i < ex.predDerivations.size(); i++) {
+        Derivation derivi = ex.predDerivations.get(i);
+        if (targetFormulas.contains(derivi.formula)) {
+          rank = i;
+          match = derivi;
+          break;
+        }
+      }
+      if (rank == -1) {
+        stats.error("unable to match on accept");
+      }
+      stats.rank(rank);
+      stats.status(InteractiveUtils.getParseStatus(ex));
+      stats.size(ex.predDerivations.size());
+
+      stats.put("formulas.size", targetFormulas.size());
+      stats.put("len_formula", targetFormulas.get(0).toLispTree().toString().length());
+      stats.put("len_utterance", ex.utterance.length());
+
+      if (match != null) {
+        if (session.isWritingCitation()) {
+          InteractiveUtils.cite(match, ex);
+        }
+        // ex.setTargetValue(match.value); // this is just for logging, not
+        // actually used for learning
+        if (session.isLearning()) {
+          LogInfo.begin_track("Updating parameters");
+          learner.onlineLearnExampleByFormula(ex, targetFormulas);
+          LogInfo.end_track();
+        }
+      }
+    } else if (command.startsWith(":def")) {
+      stats.put("type", "def"); // startsWith
+      if (tree.children.size() == 3) {
+        String head = tree.children.get(1).value;
+        String jsonDef = tree.children.get(2).value;
+
+        List<Rule> inducedRules = new ArrayList<>();
+        stats.put("head_len", head.length());
+        stats.put("json_len", jsonDef.length());
+        try {
+          inducedRules.addAll(induceRulesHelper(command, head, jsonDef, builder.parser, builder.params, session,
+              new Ref<Response>(response)));
+          stats.put("num_rules", inducedRules.size());
+        } catch (BadInteractionException e) {
+          stats.put("num_rules", 0);
+          stats.error(e.getMessage());
+          response.lines.add(e.getMessage());
+          return;
+        }
+        if (inducedRules.size() > 0) {
+          if (session.isLearning()) {
+            for (Rule rule : inducedRules) {
+              InteractiveUtils.addRuleInteractive(rule, builder.parser);
+            }
+            stats.put("total_rules", ((InteractiveBeamParser)builder.parser).allRules.size());
+            stats.put("total_unicat", ((InteractiveBeamParser)builder.parser).interactiveCatUnaryRules.size());
+          }
+          // TODO : should not have to parse again, I guess just set the formula
+          // or something
+          // builder.parser.parse(builder.params, refExHead.value, false);
+          // write out the grammar
+          if (session.isWritingGrammar()) {
+            PrintWriter out = IOUtils
+                .openOutAppendHard(Paths.get(InteractiveMaster.opts.intOutputPath, "grammar.log.json").toString());
+            for (Rule rule : inducedRules) {
+              out.println(rule.toJson());
+            }
+            out.close();
+          }
+        } else {
+          LogInfo.logs("No rule induced for head %s", head);
+        }
+      } else {
+        LogInfo.logs("Invalid format for def");
+      }
+    } else if (command.equals(":printInfo")) {
+      LogInfo.logs("Printing and overriding grammar and parameters...");
+      builder.params.write(Paths.get(InteractiveMaster.opts.intOutputPath, "params.params").toString());
+      PrintWriter out = IOUtils
+          .openOutAppendHard(Paths.get(InteractiveMaster.opts.intOutputPath + "grammar.final.json").toString());
+      for (Rule rule : builder.grammar.getRules()) {
+        out.println(rule.toJson());
+      }
+      out.close();
+      LogInfo.logs("Done printing and overriding grammar and parameters...");
+    } else if (command.equals(":context")) {
+      if (tree.children.size() == 1) {
+        LogInfo.logs("%s", session.context);
+      } else {
+        session.context = ContextValue
+            .fromString(String.format("(context (graph NaiveKnowledgeGraph ((string \"%s\") (name b) (name c))))",
+                tree.children.get(1).toString()));
+        response.stats.put("context_length", tree.children.get(1).toString().length());
+      }
+    } else {
+      LogInfo.log("Invalid command: " + tree);
+    }
+  }
+
+  private static Example exampleFromUtterance(String utt, Session session) {
+    Example.Builder b = new Example.Builder();
+    b.setId(session.id);
+    b.setUtterance(utt);
+    b.setContext(session.context);
+    Example ex = b.createExample();
+    ex.preprocess();
+    return ex;
+  }
+
+  public static List<Rule> induceRulesHelper(String command, String head, String jsonDef, Parser parser, Params params,
+      Session session, Ref<Response> refResponse) throws BadInteractionException {
+    Example exHead = exampleFromUtterance(head, session);
+    LogInfo.logs("head: %s", exHead.getTokens());
+
+    if (exHead.getTokens() == null || exHead.getTokens().size() == 0)
+      throw BadInteractionException.headIsEmpty(head);
+    if (isNonsense(exHead))
+      throw BadInteractionException.nonSenseDefinition(head);
+    
+    InteractiveBeamParserState state = ((InteractiveBeamParser)parser).parseWithoutExecuting(params, exHead, false);
+    
+    if (GrammarInducer.getParseStatus(exHead) == GrammarInducer.ParseStatus.Core)
+      throw BadInteractionException.headIsCore(head);
+
+    LogInfo.logs("num anchored: %d", state.chartList.size());
+    List<String> bodyList = InteractiveUtils.utterancefromJson(jsonDef, false);
+    LogInfo.logs("bodyutterances:\n %s", String.join("\t", bodyList));
+
+    Derivation bodyDeriv = InteractiveUtils
+        .combine(InteractiveUtils.derivsfromJson(jsonDef, parser, params, refResponse));
+    if (refResponse != null) {
+      refResponse.value.ex = exHead;
+    }
+
+    List<Rule> inducedRules = new ArrayList<>();
+    GrammarInducer grammarInducer = new GrammarInducer(exHead.getTokens(), bodyDeriv, state.chartList);
+    inducedRules.addAll(grammarInducer.getRules());
+
+    for (Rule rule : inducedRules) {
+      rule.source = new RuleSource(session.id, head, bodyList);
+    }
+
+    if (opts.useAligner && bodyList.size() == 1) {
+      List<Rule> alignedRules = DefinitionAligner.getRules(exHead.getTokens(),
+          InteractiveUtils.utterancefromJson(jsonDef, true), bodyDeriv, state.chartList);
+      for (Rule rule : alignedRules) {
+        rule.source = new RuleSource(session.id, head, bodyList);
+        rule.source.align = true;
+      }
+      inducedRules.addAll(alignedRules);
+    }
+
+    exHead.predDerivations = Lists.newArrayList(bodyDeriv);
+    return inducedRules;
+  }
+
+  private static boolean isNonsense(Example exHead) {
+    List<String> tokens = exHead.getTokens();
+    if (tokens.size() > 10)
+      return true;
+    if (tokens.size() == 0)
+      return true;
+    return tokens.stream().anyMatch(s -> s.length() > 15);
+  }
+
+  private boolean utteranceAllowed(Example ex, Response response) {
+    if (ex.utterance.length() > opts.maxChars) {
+      response.lines.add(String.format("refused to execute: too many characters in one command (current: %d, max: %d)",
+          ex.utterance.length(), opts.maxChars));
+      return false;
+    }
+    long approxSeq = ex.getLemmaTokens().stream().filter(s -> s.contains(";")).count();
+    if (approxSeq >= opts.maxSequence) {
+      response.lines.add(String.format(
+          "refused to execute: too many steps in one command -- "
+              + "consider defining some of steps as one single step.  (current: %d, max: %d)",
+          approxSeq, opts.maxSequence));
+      return false;
+    }
+    return true;
+  }
+
+}

src/edu/stanford/nlp/sempre/interactive/InteractiveServer.java

@@ -0,0 +1,358 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.PrintWriter;
+import java.io.UnsupportedEncodingException;
+import java.net.HttpCookie;
+import java.net.InetSocketAddress;
+import java.net.URI;
+import java.net.URLDecoder;
+import java.net.URLEncoder;
+import java.time.LocalDateTime;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.base.Strings;
+import com.sun.net.httpserver.Headers;
+import com.sun.net.httpserver.HttpExchange;
+import com.sun.net.httpserver.HttpHandler;
+import com.sun.net.httpserver.HttpServer;
+
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.ErrorValue;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Json;
+import edu.stanford.nlp.sempre.Master;
+import edu.stanford.nlp.sempre.Session;
+import edu.stanford.nlp.sempre.StringValue;
+import edu.stanford.nlp.sempre.Value;
+import fig.basic.IOUtils;
+import fig.basic.LogInfo;
+import fig.basic.MapUtils;
+import fig.basic.Option;
+
+/**
+ * JsonServer, interactive learning queries run through this. All the logs are
+ * handled here.
+ *
+ * @author sidaw
+ */
+public class InteractiveServer {
+  public static class Options {
+    @Option
+    public int port = 8400;
+    @Option
+    public int numThreads = 4;
+    @Option
+    public int verbose = 1;
+    @Option
+    public int maxCandidates = Integer.MAX_VALUE;
+    @Option
+    public String queryLogPath = "./int-output/query.log";
+    @Option
+    public String responseLogPath = "./int-output/response.log";
+    @Option
+    public String fullResponseLogPath;
+    @Option
+    public int maxExecutionTime = 10; // in seconds
+  }
+
+  public static Options opts = new Options();
+  private static Object queryLogLock = new Object();
+  private static Object responseLogLock = new Object();
+  private static AtomicLong queryCounter = new AtomicLong();
+  Master master;
+
+  class Handler implements HttpHandler {
+    @Override
+    public void handle(HttpExchange exchange) {
+      try {
+        new ExchangeState(exchange);
+      } catch (Exception e) {
+        e.printStackTrace();
+      }
+    }
+  }
+
+  class ExchangeState {
+    // Input
+    HttpExchange exchange;
+    Map<String, String> reqParams = new HashMap<>();
+    String remoteHost;
+
+    // For header
+    HttpCookie cookie;
+    boolean isNewSession;
+
+    // For writing main content
+
+    public ExchangeState(HttpExchange exchange) throws IOException {
+      this.exchange = exchange;
+
+      URI uri = exchange.getRequestURI();
+      this.remoteHost = exchange.getRemoteAddress().getHostName();
+
+      // Don't use uri.getQuery: it can't distinguish between '+' and '-'
+      String[] tokens = uri.toString().split("\\?");
+      if (tokens.length == 2) {
+        for (String s : tokens[1].split("&")) {
+          String[] kv = s.split("=", 2);
+          try {
+            String key = URLDecoder.decode(kv[0], "UTF-8");
+            String value = URLDecoder.decode(kv[1], "UTF-8");
+            // logs("%s => %s", key, value);
+            reqParams.put(key, value);
+          } catch (UnsupportedEncodingException e) {
+            throw new RuntimeException(e);
+          }
+        }
+      }
+      // do not decode sessionId, keep it filename and lisptree friendly
+      String sessionId = URLEncoder.encode(MapUtils.get(reqParams, "sessionId", ""), "UTF-8");
+      if (sessionId != null) {
+        isNewSession = false;
+      } else {
+        isNewSession = true;
+      }
+
+      if (opts.verbose >= 2)
+        logs("GET %s from %s (%ssessionId=%s)", uri, remoteHost, isNewSession ? "new " : "", sessionId);
+
+      String uriPath = uri.getPath();
+      if (uriPath.equals("/"))
+        uriPath += "index.html";
+      if (uriPath.equals("/sempre")) {
+        handleQuery(sessionId);
+      } else {
+        // getFile(opts.basePath + uriPath); security
+      }
+      exchange.close();
+    }
+
+    String getMimeType(String path) {
+      String[] tokens = path.split("\\.");
+      String ext = tokens[tokens.length - 1];
+      if (ext.equals("html"))
+        return "text/html";
+      if (ext.equals("css"))
+        return "text/css";
+      if (ext.equals("jpeg"))
+        return "image/jpeg";
+      if (ext.equals("gif"))
+        return "image/gif";
+      return "text/plain";
+    }
+
+    void setHeaders(String mimeType) throws IOException {
+      Headers headers = exchange.getResponseHeaders();
+      headers.set("Content-Type", mimeType);
+      headers.set("Access-Control-Allow-Origin", "*");
+      if (isNewSession && cookie != null)
+        headers.set("Set-Cookie", cookie.toString());
+      exchange.sendResponseHeaders(200, 0);
+    }
+
+    Map<String, Object> makeJson(Master.Response response) {
+      Map<String, Object> json = new HashMap<String, Object>();
+      json.put("stats", response.stats);
+
+      if (response.lines != null) {
+        json.put("lines", response.lines);
+      }
+      if (response.getExample() != null) {
+        List<Object> items = new ArrayList<Object>();
+        json.put("candidates", items);
+        List<Derivation> allCandidates = response.getExample().getPredDerivations();
+        Derivation.sortByScore(allCandidates);
+        if (allCandidates != null) {
+          if (allCandidates.size() >= InteractiveServer.opts.maxCandidates) {
+            response.lines.add(String.format("Exceeded max options: (current: %d / max: %d) ", allCandidates.size(),
+                InteractiveServer.opts.maxCandidates));
+            allCandidates = allCandidates.subList(0, InteractiveServer.opts.maxCandidates);
+          }
+
+          for (Derivation deriv : allCandidates) {
+            Map<String, Object> item = new HashMap<String, Object>();
+            Value value = deriv.getValue();
+            if (value instanceof StringValue)
+              item.put("value", ((StringValue) value).value);
+            else if (value instanceof ErrorValue)
+              item.put("value", ((ErrorValue) value).sortString());
+            else if (value != null)
+              item.put("value", value.sortString());
+            else
+              item.put("value", "[[]]");
+            item.put("score", deriv.getScore());
+            item.put("prob", deriv.getProb());
+            item.put("anchored", deriv.allAnchored); // used only anchored rules
+            item.put("formula", deriv.formula.toLispTree().toString());
+            items.add(item);
+          }
+        }
+      }
+
+      return json;
+    }
+
+    // This should be concurrent
+    Master.Response processQuery(Session session, String query) {
+      String message = null;
+      Master.Response response = master.new Response();
+      ExecutorService executor = Executors.newSingleThreadExecutor();
+      Future<Master.Response> future = executor.submit(() -> master.processQuery(session, query));
+      long startTime = System.nanoTime();
+      try {
+        // most exceptions should be handled in InteractiveMaster
+        // so the response can be more specific
+        response = future.get(opts.maxExecutionTime, TimeUnit.SECONDS);
+      } catch (Throwable e) {
+        e.printStackTrace();
+        message = e.toString();
+        response.lines.add(String.format("Exceeded the maximum allowed time: %ss", opts.maxExecutionTime));
+        response.stats.put("uncaught_error", message);
+        LogInfo.flush();
+        LogInfo.resetInfos();
+      } finally {
+        future.cancel(true);
+        executor.shutdown();
+        long endTime = System.nanoTime();
+        response.stats.put("walltime", (endTime - startTime) / 1.0e9);
+        response.stats.put("count", queryCounter.get());
+      }
+      return response;
+    }
+
+    void handleQuery(String sessionId) throws IOException {
+      String query = reqParams.get("q");
+      long queryNumber = queryCounter.incrementAndGet();
+      Session session = master.getSession(sessionId);
+      session.reqParams = reqParams;
+      session.remoteHost = remoteHost;
+      session.format = "json";
+
+      LocalDateTime queryTime = LocalDateTime.now();
+      synchronized (queryLogLock) { // write the query log
+        Map<String, Object> jsonMap = new LinkedHashMap<>();
+        jsonMap.put("count", queryNumber);
+        jsonMap.put("q", query);
+        // jsonMap.put("remote", remoteHost);
+        // jsonMap.put("time", queryTime.toString());
+        jsonMap.put("sessionId", sessionId);
+        reqParams.remove("q");
+        jsonMap.putAll(reqParams);
+        if (session.isLogging()) {
+          logLine(opts.queryLogPath, Json.writeValueAsStringHard(jsonMap));
+        } else {
+          logLine(opts.queryLogPath + ".sandbox", Json.writeValueAsStringHard(jsonMap));
+        }
+      }
+
+      // If JSON, don't store cookies.
+
+      if (query == null)
+        query = "null";
+      logs("Server.handleQuery %s: %s", session.id, query);
+
+      // Print header
+      setHeaders("application/json");
+
+      Master.Response masterResponse = null;
+      if (query != null) {
+        masterResponse = processQuery(session, query);
+      }
+
+      Map<String, Object> responseMap = null;
+      {
+        PrintWriter out = new PrintWriter(new OutputStreamWriter(exchange.getResponseBody()));
+        if (masterResponse != null) {
+          // Render answer
+          Example ex = masterResponse.getExample();
+          responseMap = makeJson(masterResponse);
+          out.println(Json.writeValueAsStringHard(responseMap));
+        }
+        out.close();
+      }
+
+      synchronized (responseLogLock) { // write the response log log
+        Map<String, Object> jsonMap = new LinkedHashMap<>();
+        LocalDateTime responseTime = LocalDateTime.now();
+        // jsonMap.put("responseTime", responseTime.toString());
+        jsonMap.put("time", queryTime.toString());
+        jsonMap.put("ms", String.format("%.3f", java.time.Duration.between(queryTime, responseTime).toNanos() / 1.0e6));
+        jsonMap.put("sessionId", sessionId);
+        jsonMap.put("q", query); // backwards compatibility...
+        jsonMap.put("lines", responseMap.get("lines"));
+        if (session.isLogging()) {
+          logLine(opts.responseLogPath, Json.writeValueAsStringHard(jsonMap));
+          if (!Strings.isNullOrEmpty(opts.fullResponseLogPath)) {
+            jsonMap.put("candidates", responseMap.get("candidates"));
+            logLine(opts.fullResponseLogPath, Json.writeValueAsStringHard(jsonMap));
+          }
+        } else {
+          logLine(opts.responseLogPath + ".sandbox", Json.writeValueAsStringHard(jsonMap));
+          if (!Strings.isNullOrEmpty(opts.fullResponseLogPath)) {
+            jsonMap.put("candidates", responseMap.get("candidates"));
+            logLine(opts.fullResponseLogPath + ".sandbox", Json.writeValueAsStringHard(jsonMap));
+          }
+          // LogInfo.log(Json.writeValueAsStringHard(jsonMap));
+        }
+      }
+    }
+
+    void logLine(String path, String line) {
+      PrintWriter out;
+      try {
+        out = IOUtils.openOutAppend(path);
+        out.println(line);
+        out.close();
+      } catch (IOException e) {
+        // TODO Auto-generated catch block
+        e.printStackTrace();
+      }
+    }
+  }
+
+  private void logs(String s, Object... args) {
+  };
+
+  public InteractiveServer(Master master) {
+    this.master = master;
+  }
+
+  public void run() {
+    try {
+      String hostname = fig.basic.SysInfoUtils.getHostName();
+      HttpServer server = HttpServer.create(new InetSocketAddress(opts.port), 10);
+      // generous timeout here
+      ExecutorService pool = new ThreadPoolExecutor(opts.numThreads, opts.numThreads, 120, TimeUnit.SECONDS,
+          new LinkedBlockingQueue<Runnable>());
+      // Executors.newFixedThreadPool(opts.numThreads);
+      server.createContext("/", new Handler());
+      server.setExecutor(pool);
+      server.start();
+      LogInfo.logs("JSON Server (%d threads) started at http://%s:%s/sempre", opts.numThreads, hostname, opts.port);
+      LogInfo.log("Press Ctrl-D to terminate.");
+      LogInfo.begin_threads();
+      while (LogInfo.stdin.readLine() != null) {
+      }
+      LogInfo.log("Shutting down server...");
+      server.stop(0);
+      LogInfo.log("Shutting down executor pool...");
+      pool.shutdown();
+      LogInfo.end_threads();
+    } catch (IOException e) {
+      throw new RuntimeException(e);
+    }
+  }
+}

src/edu/stanford/nlp/sempre/interactive/InteractiveUtils.java

@@ -0,0 +1,199 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import org.testng.collections.Lists;
+
+import com.google.common.collect.ImmutableList;
+
+import edu.stanford.nlp.sempre.ActionFormula;
+import edu.stanford.nlp.sempre.BeamParser;
+import edu.stanford.nlp.sempre.Derivation;
+import edu.stanford.nlp.sempre.Example;
+import edu.stanford.nlp.sempre.Formula;
+import edu.stanford.nlp.sempre.Formulas;
+import edu.stanford.nlp.sempre.IdentityFn;
+import edu.stanford.nlp.sempre.Json;
+import edu.stanford.nlp.sempre.Master;
+import edu.stanford.nlp.sempre.Params;
+import edu.stanford.nlp.sempre.Parser;
+import edu.stanford.nlp.sempre.Rule;
+import edu.stanford.nlp.sempre.SemanticFn;
+import fig.basic.LispTree;
+import fig.basic.LogInfo;
+import fig.basic.Option;
+import fig.basic.Ref;
+
+/**
+ * Utilities for interactive learning
+ *
+ * @author sidaw
+ */
+public final class InteractiveUtils {
+  public static class Options {
+    @Option(gloss = "use the best formula when no match or not provided")
+    public boolean useBestFormula = false;
+
+    @Option(gloss = "path to the citations")
+    public String citationPath;
+
+    @Option(gloss = "verbose")
+    public int verbose = 0;
+  }
+
+  public static Options opts = new Options();
+
+  private InteractiveUtils() {
+  }
+
+  // dont spam my log when reading things in the beginning...
+  public static boolean fakeLog = false;
+
+  public static Derivation stripDerivation(Derivation deriv) {
+    while (deriv.rule.sem instanceof IdentityFn) {
+      deriv = deriv.child(0);
+    }
+    return deriv;
+  }
+
+  public static Derivation stripBlock(Derivation deriv) {
+    if (opts.verbose > 0)
+      LogInfo.logs("StripBlock %s %s %s", deriv, deriv.rule, deriv.cat);
+    while ((deriv.rule.sem instanceof BlockFn || deriv.rule.sem instanceof IdentityFn) && deriv.children.size() == 1) {
+      deriv = deriv.child(0);
+    }
+    return deriv;
+  }
+
+  public static List<Derivation> derivsfromJson(String jsonDef, Parser parser, Params params,
+      Ref<Master.Response> refResponse) {
+    @SuppressWarnings("unchecked")
+    List<Object> body = Json.readValueHard(jsonDef, List.class);
+    // string together the body definition
+    List<Derivation> allDerivs = new ArrayList<>();
+    int numFailed = 0;
+    for (Object obj : body) {
+      @SuppressWarnings("unchecked")
+      List<String> pair = (List<String>) obj;
+      String utt = pair.get(0);
+      String formula = pair.get(1);
+
+      if (formula.equals("()")) {
+        LogInfo.logs("Error: Got empty formula");
+        continue;
+      }
+
+      Example.Builder b = new Example.Builder();
+      // b.setId("session:" + sessionId);
+      b.setUtterance(utt);
+      Example ex = b.createExample();
+      ex.preprocess();
+
+      LogInfo.logs("Parsing body: %s", ex.utterance);
+      ((InteractiveBeamParser)parser).parseWithoutExecuting(params, ex, false);
+
+      boolean found = false;
+      Formula targetFormula = Formulas.fromLispTree(LispTree.proto.parseFromString(formula));
+      for (Derivation d : ex.predDerivations) {
+        // LogInfo.logs("considering: %s", d.formula.toString());
+        if (d.formula.equals(targetFormula)) {
+          found = true;
+          allDerivs.add(stripDerivation(d));
+          break;
+        }
+      }
+      if (!found && !formula.equals("?")) {
+        LogInfo.errors("matching formula not found: %s :: %s", utt, formula);
+        numFailed++;
+      }
+      // just making testing easier, use top derivation when we formula is not
+      // given
+      if (!found && ex.predDerivations.size() > 0 && (formula.equals("?") || formula == null || opts.useBestFormula))
+        allDerivs.add(stripDerivation(ex.predDerivations.get(0)));
+      else if (!found) {
+        Derivation res = new Derivation.Builder().formula(targetFormula)
+            // setting start to -1 is important,
+            // which grammarInducer interprets to mean we do not want partial
+            // rules
+            .withCallable(new SemanticFn.CallInfo("$Action", -1, -1, null, new ArrayList<>())).createDerivation();
+        allDerivs.add(res);
+      }
+    }
+    if (refResponse != null) {
+      refResponse.value.stats.put("num_failed", numFailed);
+      refResponse.value.stats.put("num_body", body.size());
+    }
+    // LogInfo.logs("returning deriv list %s, \n %s", allDerivs.toString(),
+    // jsonDef);
+    return allDerivs;
+  }
+
+  public static List<String> utterancefromJson(String jsonDef, boolean tokenize) {
+    @SuppressWarnings("unchecked")
+    List<Object> body = Json.readValueHard(jsonDef, List.class);
+    // string together the body definition
+    List<String> utts = new ArrayList<>();
+    for (int i = 0; i < body.size(); i++) {
+      Object obj = body.get(i);
+      @SuppressWarnings("unchecked")
+      List<String> pair = (List<String>) obj;
+      String utt = pair.get(0);
+
+      Example.Builder b = new Example.Builder();
+      // b.setId("session:" + sessionId);
+      b.setUtterance(utt);
+      Example ex = b.createExample();
+      ex.preprocess();
+
+      if (tokenize) {
+        utts.addAll(ex.getTokens());
+        if (i != body.size() - 1 && !utts.get(utts.size() - 1).equals(";"))
+          utts.add(";");
+      } else {
+        utts.add(String.join(" ", ex.getTokens()));
+      }
+
+    }
+    return utts;
+  }
+
+  public static synchronized void addRuleInteractive(Rule rule, Parser parser) {
+    LogInfo.logs("addRuleInteractive: %s", rule);
+    if (parser instanceof InteractiveBeamParser) {
+      parser.addRule(rule);
+    } else {
+      throw new RuntimeException("interactively adding rule not supported for paser " + parser.getClass().toString());
+    }
+  }
+
+  static Rule blockRule(ActionFormula.Mode mode) {
+    BlockFn b = new BlockFn(mode);
+    b.init(LispTree.proto.parseFromString("(BlockFn sequential)"));
+    return new Rule("$Action", Lists.newArrayList("$Action", "$Action"), b);
+  }
+
+  public static Derivation combine(List<Derivation> children) {
+    ActionFormula.Mode mode = ActionFormula.Mode.sequential;
+    if (children.size() == 1) {
+      return children.get(0);
+    }
+    Formula f = new ActionFormula(mode, children.stream().map(d -> d.formula).collect(Collectors.toList()));
+    Derivation res = new Derivation.Builder().formula(f)
+        // setting start to -1 is important,
+        // which grammarInducer interprets to mean we do not want partial rules
+        .withCallable(new SemanticFn.CallInfo("$Action", -1, -1, blockRule(mode), ImmutableList.copyOf(children)))
+        .createDerivation();
+    return res;
+  }
+
+  public static String getParseStatus(Example ex) {
+    return GrammarInducer.getParseStatus(ex).toString();
+  }
+
+  public static void cite(Derivation match, Example ex) {
+    CitationTracker tracker = new CitationTracker(ex.id, ex);
+    tracker.citeAll(match);
+  }
+}

src/edu/stanford/nlp/sempre/interactive/Item.java

@@ -0,0 +1,16 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import java.util.Set;
+
+// Individual items with some properties
+public abstract class Item {
+  public Set<String> names;
+
+  public abstract boolean selected(); // explicit global selection
+
+  public abstract void select(boolean sel);
+
+  public abstract void update(String rel, Object value);
+
+  public abstract Object get(String rel);
+}

src/edu/stanford/nlp/sempre/interactive/QueryStats.java

@@ -0,0 +1,41 @@
+package edu.stanford.nlp.sempre.interactive;
+
+import edu.stanford.nlp.sempre.Master;
+
+public class QueryStats {
+  Master.Response response;
+  QueryType type;
+
+  public enum QueryType {
+    q, def, accept, reject, other
+  };
+
+  public QueryStats(Master.Response response) {
+    this.response = response;
+  }
+
+  public QueryStats(Master.Response response, String command) {
+    this.response = response;
+    put("type", command.substring(1));
+  }
+
+  public void put(String k, Object v) {
+    response.stats.put(k, v);
+  }
+
+  public void size(int num) {
+    put("size", num);
+  }
+
+  public void status(String status) {
+    put("status", status);
+  }
+
+  public void rank(int r) {
+    put("rank", r);
+  }
+
+  public void error(String msg) {
+    put("error", msg);
+  }
+}