Merge branch 'cassandra-3.X' into trunk

CHANGES.txt

@@ -28,6 +28,8 @@
  * cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946)
  * Add support for arithmetic operators (CASSANDRA-11935)
  * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
+Merged from 3.0:
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 
 
 3.10

NEWS.txt

@@ -44,6 +44,9 @@ Upgrading
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
+   - Only Java and JavaScript are now supported UDF languages.
+     The sandbox in 3.0 already prevented the use of script languages except Java
+     and JavaScript.
 
 3.10
 ====

doc/cql3/CQL.textile

@@ -2078,7 +2078,7 @@ SELECT AVG(players) FROM plays;
 
 h2(#udfs). User-Defined Functions
 
-User-defined functions allow execution of user-provided code in Cassandra. By default, Cassandra supports defining functions in _Java_ and _JavaScript_. Support for other JSR 223 compliant scripting languages (such as Python, Ruby, and Scala) can be added by adding a JAR to the classpath.
+User-defined functions allow execution of user-provided code in Cassandra. By default, Cassandra supports defining functions in _Java_ and _JavaScript_. Support for other JSR 223 compliant scripting languages (such as Python, Ruby, and Scala) has been removed in 3.0.11.
 
 UDFs are part of the Cassandra schema.  As such, they are automatically propagated to all nodes in the cluster.
 

lib/jsr223/clojure/README.txt

@@ -1,8 +0,0 @@
-Apache Cassandra User-Defined-Functions JSR 223 scripting
-=========================================================
-
-Unfortunately the JSR-223 support provided by the project https://github.com/ato/clojure-jsr223
-and the related ones do not provide compileable script support.
-
-The JSR-223 javax.script.Compilable implementation takes source file names or readers but not script sources
-as all other JSR-223 implementations do.

lib/jsr223/groovy/README.txt

@@ -1,35 +0,0 @@
-Apache Cassandra User-Defined-Functions JSR 223 scripting
-=========================================================
-
-Using JSR-223 capable Groovy
-
-Tested with version 2.3.6
-
-Installation
-------------
-
-1. Download Groovy binary release
-2. Unpack the downloaded archive into a temporary directory
-3. Copy the jar groovy-all-2.3.6-indy.jar from the Groovy embeddable directory to $CASSANDRA_HOME/lib/jsr223/groovy
-   "indy" means "invokedynamic" and is a JVM instruction for scripting languages new to Java 7.
-4. Restart your Cassandra daemon if it's already running
-
-Cassandra log should contain a line like this:
-  INFO  10:49:45 Found scripting engine Groovy Scripting Engine 2.0 - Groovy 2.3.6 - language names: [groovy, Groovy]
-Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
-
-Smoke Test
-----------
-
-To test Groovy functionality, open cqlsh and execute the following command:
-  CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE groovy AS 'return "foo";' ;
-
-If you get the error
-  code=2200 [Invalid query] message="Invalid language groovy for 'foobar'"
-Groovy for Apache Cassandra has not been installed correctly.
-
-Notes / Java7 invokedynamic
----------------------------
-
-Groovy provides jars that support invokedynamic bytecode instruction. These jars are whose ending with
-"-indy.jar".

lib/jsr223/jaskell/README.txt

@@ -1,5 +0,0 @@
-Apache Cassandra User-Defined-Functions JSR 223 scripting
-=========================================================
-
-Unfortunately Jaskell JSR-223 support is quite old and the Jaskell engine seems to be quite
-unsupported. If you find a solution, please open a ticket at Apache Cassandra JIRA.

lib/jsr223/jruby/README.txt

@@ -1,54 +0,0 @@
-Apache Cassandra User-Defined-Functions JSR 223 scripting
-=========================================================
-
-Using JSR-223 capable JRuby
-
-Tested with version 1.7.15
-
-Installation
-------------
-
-1. Download JRuby binary release
-2. Unpack the downloaded archive into a temporary directory
-3. Copy everything from the JRuby lib directory to $CASSANDRA_HOME/lib/jsr223/jruby
-4. Restart your Cassandra daemon if it's already running
-
-Cassandra log should contain a line like this:
-  INFO  10:29:03 Found scripting engine JSR 223 JRuby Engine 1.7.15 - ruby jruby 1.7.15 - language names: [ruby, jruby]
-Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
-
-
-Smoke Test
-----------
-
-To test JRuby functionality, open cqlsh and execute the following command:
-  CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE ruby AS 'return "foo";' ;
-
-If you get the error
-  code=2200 [Invalid query] message="Invalid language ruby for 'foobar'"
-JRuby for Apache Cassandra has not been installed correctly.
-
-
-Ruby require/include
---------------------
-
-You can use Ruby require and include in your scripts as in the following example:
-
-
-CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE ruby AS '
-require "bigdecimal"
-require "bigdecimal/math"
-
-include BigMath
-
-a = BigDecimal((PI(100)/2).to_s)
-
-return "foo " + a.to_s;
-' ;
-
-
-Notes / Java7 invokedynamic
----------------------------
-
-See JRuby wiki pages https://github.com/jruby/jruby/wiki/ConfiguringJRuby and
-https://github.com/jruby/jruby/wiki/PerformanceTuning for more information and optimization tips.

lib/jsr223/jython/README.txt

@@ -1,33 +0,0 @@
-Apache Cassandra User-Defined-Functions JSR 223 scripting
-=========================================================
-
-Using JSR-223 capable Jython
-
-Tested with version 2.3.5
-
-Installation
-------------
-
-1. Download Jython binary release
-2. Unpack the downloaded archive into a temporary directory
-3. Copy the jar jython.jar from the Jython directory to $CASSANDRA_HOME/lib/jsr223/jython
-4. Restart your Cassandra daemon if it's already running
-
-Cassandra log should contain a line like this:
-  INFO  10:58:18 Found scripting engine jython 2.5.3 - python 2.5 - language names: [python, jython]
-Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
-
-Smoke Test
-----------
-
-To test Jython functionality, open cqlsh and execute the following command:
-  CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE python AS '''foo''' ;
-
-If you get the error
-  code=2200 [Invalid query] message="Invalid language python for 'foobar'"
-Jython for Apache Cassandra has not been installed correctly.
-
-Notes / Java7 invokedynamic
----------------------------
-
-Jython currently targets Java6 only. They want to switch to Java7 + invokedynamic in Jython 3.

lib/jsr223/scala/README.txt

@@ -1,37 +0,0 @@
-Apache Cassandra User-Defined-Functions JSR 223 scripting
-=========================================================
-
-Using JSR-223 capable Scala
-
-Tested with version 2.11.2
-
-Installation
-------------
-
-1. Download Scala binary release
-2. Unpack the downloaded archive into a temporary directory
-3. Copy the following jars from the Scala lib directory to $CASSANDRA_HOME/lib/jsr223/scala
-   scala-compiler.jar
-   scala-library.jar
-   scala-reflect.jar
-4. Restart your Cassandra daemon if it's already running
-
-Cassandra log should contain a line like this:
-  INFO  11:42:35 Found scripting engine Scala Interpreter 1.0 - Scala version 2.11.2 - language names: [scala]
-Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
-
-Smoke Test
-----------
-
-To test Scala functionality, open cqlsh and execute the following command:
-  CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE scala AS 'return "foo";' ;
-
-If you get the error
-  code=2200 [Invalid query] message="Invalid language scala for 'foobar'"
-Scala for Apache Cassandra has not been installed correctly.
-
-Notes / Java7 invokedynamic
----------------------------
-
-Scala 2.10 has Java6 support only. 2.11 has experimental invokedynamic support (use at your own risk!).
-2.12 introduces an upgrade directly to Java8 - see https://stackoverflow.com/questions/14285894/advantages-of-scala-emitting-bytecode-for-the-jvm-1-7

src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDFunction.java

@@ -28,6 +28,9 @@ import java.util.concurrent.ExecutorService;
 import javax.script.*;
 
 import jdk.nashorn.api.scripting.AbstractJSObject;
+import jdk.nashorn.api.scripting.ClassFilter;
+import jdk.nashorn.api.scripting.NashornScriptEngine;
+import jdk.nashorn.api.scripting.NashornScriptEngineFactory;
 import org.apache.cassandra.concurrent.NamedThreadFactory;
 import org.apache.cassandra.cql3.ColumnIdentifier;
 import org.apache.cassandra.db.marshal.AbstractType;
@@ -36,8 +39,6 @@ import org.apache.cassandra.transport.ProtocolVersion;
 
 final class ScriptBasedUDFunction extends UDFunction
 {
-    static final Map<String, Compilable> scriptEngines = new HashMap<>();
-
     private static final ProtectionDomain protectionDomain;
     private static final AccessControlContext accessControlContext;
 
@@ -93,23 +94,17 @@ final class ScriptBasedUDFunction extends UDFunction
                                                                               UDFunction::initializeThread)),
                                "userscripts");
 
+    private static final ClassFilter classFilter = clsName -> secureResource(clsName.replace('.', '/') + ".class");
+
+    private static final NashornScriptEngine scriptEngine;
+
+
     static
     {
         ScriptEngineManager scriptEngineManager = new ScriptEngineManager();
-        for (ScriptEngineFactory scriptEngineFactory : scriptEngineManager.getEngineFactories())
+        ScriptEngine engine = scriptEngineManager.getEngineByName("nashorn");
-        {
+        NashornScriptEngineFactory factory = engine != null ? (NashornScriptEngineFactory) engine.getFactory() : null;
-            ScriptEngine scriptEngine = scriptEngineFactory.getScriptEngine();
+        scriptEngine = factory != null ? (NashornScriptEngine) factory.getScriptEngine(new String[]{}, udfClassLoader, classFilter) : null;
-            boolean compilable = scriptEngine instanceof Compilable;
-            if (compilable)
-            {
-                logger.info("Found scripting engine {} {} - {} {} - language names: {}",
-                            scriptEngineFactory.getEngineName(), scriptEngineFactory.getEngineVersion(),
-                            scriptEngineFactory.getLanguageName(), scriptEngineFactory.getLanguageVersion(),
-                            scriptEngineFactory.getNames());
-                for (String name : scriptEngineFactory.getNames())
-                    scriptEngines.put(name, (Compilable) scriptEngine);
-            }
-        }
 
         try
         {
@@ -141,8 +136,7 @@ final class ScriptBasedUDFunction extends UDFunction
     {
         super(name, argNames, argTypes, returnType, calledOnNullInput, language, body);
 
-        Compilable scriptEngine = scriptEngines.get(language);
+        if (!"JavaScript".equalsIgnoreCase(language) || scriptEngine == null)
-        if (scriptEngine == null)
             throw new InvalidRequestException(String.format("Invalid language '%s' for function '%s'", language, name));
 
         // execute compilation with no-permissions to prevent evil code e.g. via "static code blocks" / "class initialization"
@@ -161,10 +155,7 @@ final class ScriptBasedUDFunction extends UDFunction
 
         // It's not always possible to simply pass a plain Java object as a binding to Nashorn and
         // let the script execute methods on it.
-        udfContextBinding =
+        udfContextBinding = new UDFContextWrapper();
-            ("Oracle Nashorn".equals(((ScriptEngine) scriptEngine).getFactory().getEngineName()))
-                ? new UDFContextWrapper()
-                : udfContext;
     }
 
     protected ExecutorService executor()

test/unit/org/apache/cassandra/cql3/validation/entities/UFScriptTest.java

@@ -392,8 +392,7 @@ public class UFScriptTest extends CQLTester
         DatabaseDescriptor.enableScriptedUserDefinedFunctions(false);
         try
         {
-            assertInvalid("double",
+            assertInvalid("CREATE OR REPLACE FUNCTION " + KEYSPACE + ".assertNotEnabled(val double) " +
-                          "CREATE OR REPLACE FUNCTION " + KEYSPACE + ".assertNotEnabled(val double) " +
                           "RETURNS NULL ON NULL INPUT " +
                           "RETURNS double " +
                           "LANGUAGE javascript\n" +

test/unit/org/apache/cassandra/cql3/validation/entities/UFSecurityTest.java

@@ -149,7 +149,6 @@ public class UFSecurityTest extends CQLTester
         "new java.net.ServerSocket().bind(null); 0;",
         "var thread = new java.lang.Thread(); thread.start(); 0;",
         "java.lang.System.getProperty(\"foo.bar.baz\"); 0;",
-        "java.lang.Class.forName(\"java.lang.System\"); 0;",
         "java.lang.Runtime.getRuntime().exec(\"/tmp/foo\"); 0;",
         "java.lang.Runtime.getRuntime().loadLibrary(\"foobar\"); 0;",
         "java.lang.Runtime.getRuntime().loadLibrary(\"foobar\"); 0;",
@@ -177,6 +176,17 @@ public class UFSecurityTest extends CQLTester
                 assertAccessControlException(script, e);
             }
         }
+
+        String script = "java.lang.Class.forName(\"java.lang.System\"); 0;";
+        String fName = createFunction(KEYSPACE_PER_TEST, "double",
+                                      "CREATE OR REPLACE FUNCTION %s(val double) " +
+                                      "RETURNS NULL ON NULL INPUT " +
+                                      "RETURNS double " +
+                                      "LANGUAGE javascript\n" +
+                                      "AS '" + script + "';");
+        assertInvalidThrowMessage("Java reflection not supported when class filter is present",
+                                  FunctionExecutionException.class,
+                                  "SELECT " + fName + "(dval) FROM %s WHERE key=1");
     }
 
     private static void assertAccessControlException(String script, FunctionExecutionException e)