From 656ab584657feee562978cc574f403046ca67436 Mon Sep 17 00:00:00 2001 From: Ziang Xie Date: Mon, 20 Jan 2014 12:35:25 -0800 Subject: [PATCH] minor fixes to quickstart and readme plus gitignore --- .gitignore | 9 +++++++++ QUICKSTART.md | 11 +++++++++++ TUTORIAL.md | 25 ++++++++++--------------- 3 files changed, 30 insertions(+), 15 deletions(-) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..a51a034 --- /dev/null +++ b/.gitignore @@ -0,0 +1,9 @@ +lib +state +test-output +*.cache + +virtuoso-opensource + +data/free917*.json +data/emnlp*.grammar diff --git a/QUICKSTART.md b/QUICKSTART.md index 41540a9..a8a9a47 100644 --- a/QUICKSTART.md +++ b/QUICKSTART.md @@ -8,6 +8,9 @@ These commands will download necessary resources: ./download-dependencies emnlp2013 ./download-dependencies fullfreebase_vdb +See `README.md` for other requirements SEMPRE depends on +(JDK 7 and Ruby). + # Install the Database Freebase is stored in a database called virtuoso. These commands will @@ -15,6 +18,9 @@ install a copy of it. Make sure to execute the `git checkout tags/v7.0.0` to ensure you have a compatible version of virtuoso (instead of the most recent version). + # For Ubuntu, make sure these dependencies are installed + sudo apt-get install -y automake gawk gperf libtool bison flex libssl-dev + # Clone into sempre folder git clone https://github.com/openlink/virtuoso-opensource cd virtuoso-opensource git checkout tags/v7.0.0 @@ -32,6 +38,8 @@ This will start the virtuoso database on `localhost:3093` and import freebase: # Running the System on New Questions +First make sure you have compiled sempre by running `make`. + Create a file called `testinput` that has your test questions in this format: (example (utterance "what states make up the midwest us?") (targetValues (description ""))) @@ -71,3 +79,6 @@ a little over three days to complete. @sparqlserver=localhost:3093 \ @domain=webquestions \ @cacheserver=local + +Alternatively, you can sanity check the system on the Free917 dataset by +setting `@domain=free917`. This should take a few hours to complete. diff --git a/TUTORIAL.md b/TUTORIAL.md index 8492ba3..dba5dfb 100644 --- a/TUTORIAL.md +++ b/TUTORIAL.md @@ -55,7 +55,7 @@ these logical forms automatically from natural language. We can execute these logical forms by typing into the interactive prompt: (execute (call + (number 3) (number 4))) - + This should print out `(number 7)`, which we refer to as the *denotation* of the logical form. Try to execute the other logical forms and see what you get. @@ -81,7 +81,7 @@ here, there is no difference. This concludes the section on logical forms and denotations. We have presented one system of logical forms, which are executed using the `JavaExecutor`. The system is general in that it supports other types of logical forms, for -example, which encode SPARQL database queries. +example, those which encode SPARQL database queries. ## Parsing @@ -266,7 +266,7 @@ two possibilities: Now type in three and four - + There should be two derivations each with probability 0.5: (derivation (formula (((lambda y (lambda x (call + (var x) (var y)))) (number 4.0)) (number 3.0))) (value (number 7.0)) (type fb:type.number)) [score=0, prob=0.500] @@ -312,7 +312,7 @@ forms to denotations by executing Java code. A major application of semantic parsing is where the logical forms are database queries. In this section, we will look at querying graph databases. -A graph database (e.g., Freebase) stores information about entities +A graph database (e.g., Freebase) stores information about entities and their properties; concretely, it is just a set of triples $(s, p, o)$, where $s$ and $o$ are entities and $p$ is a property. For example: @@ -325,12 +325,7 @@ See `data/tutorial.ttl` for an example of a tiny subset of the Freebase graph pertaining to geography about California. We will use Virtuoso to provide the backend for querying this data. Make sure -you have Virtuoso downloaded and compiled: - - git clone https://github.com/openlink/virtuoso-opensource - # For Ubuntu, make sure these dependencies are installed - sudo apt-get install -y automake gawk gperf libtool bison flex libssl-dev - cd virtuoso-opensource && ./autogen.sh && ./configure --prefix=$PWD/install && make && make install +you have Virtuoso downloaded and compiled (see QUICKSTART.md for instructions). Start the server: @@ -342,12 +337,12 @@ Add the small graph to the database: Aside: if you want to work with a larger database: - ./download-dependencies geofreebase_ttl # Subset of Freebase for geography in .ttl format + ./download-dependencies geofreebase_ttl # Subset of Freebase for geography in .ttl format ./download-dependencies geofreebase_vdb # Virtuoso index for subset of Freebase for geography ./download-dependencies fullfreebase_ttl # All of Freebase in .ttl format (BIG FILE) ./download-dependencies fullfreebase_vdb # Virtuoso index for all of Freebase (BIG FILE) -Then, just replace `tutorial.vdb` with the appropriate `lib/fb_data/??.exec/vdb` path. +Then, just replace `tutorial.vdb` with the appropriate `lib/freebase/??.exec/vdb` path. Now we are ready to make some queries. Start up the interactive prompt, now with the `SparqlExecutor`: @@ -361,7 +356,7 @@ semantics. The simplest formula is a single entity: fb:en.california To execute this query, simply type the following into the interactive prompt: - + (execute fb:en.california) This should return: @@ -404,13 +399,13 @@ Here are the following types of logical forms: a binary $b$ and $y$ is in the set denoted by unary $u$. 1. Count `(count |u|)`: denotes the set containing the cardinality of the set denoted by `u`. 1. Superlative `(argmax |rank| |count| |u| |b|)`: sort the elements of `z` by decreasing `b` - and return `count` elements starting at offset `rank` (0-based). + and return `count` elements starting at offset `rank` (1-based). 1. Mu abstraction `(mark (var |v|) |u|)`: same as the unary |u| denoting entities |x|, with the exception that |x| must be equal to all occurrences of the variable |v| in |u|. 2. Lambda abstraction `(lambda (var |v|) |u|)`: produces a binary (x,y) where `x` is in the set denoted by `u` and `y` is the value taken on by variable - `v`z. + `v`. See `SparqlExecutorTest.java` for examples.