natas_hw
/
cassandra
mirror of https://github.com/apache/cassandra
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Restrict script UDFs to Nashorn

This commit is contained in:
Robert Stupp 2016-12-09 20:10:16 +01:00
parent ad8c23691a
commit 0e9e0a4a83
12 changed files with 287 additions and 197 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGES.txt
View File

@ -176,6 +176,7 @@ Merged from 3.0:
* Correct log message for statistics of offheap memtable flush (CASSANDRA-12776)
* Explicitly set locale for string validation (CASSANDRA-12541,CASSANDRA-12542,CASSANDRA-12543,CASSANDRA-12545)
Merged from 2.2:
* Remove support for non-JavaScript UDFs (CASSANDRA-12883)
* Fix DynamicEndpointSnitch noop in multi-datacenter situations (CASSANDRA-13074)
* cqlsh copy-from: encode column names to avoid primary key parsing errors (CASSANDRA-12909)
* Temporarily fix bug that creates commit log when running offline tools (CASSANDRA-8616)

3
NEWS.txt
View File

@ -21,6 +21,9 @@ Upgrading
- Specifying the default_time_to_live option when creating or altering a
materialized view was erroneously accepted (and ignored). It is now
properly rejected.
- Only Java and JavaScript are now supported UDF languages.
The sandbox in 3.0 already prevented the use of script languages except Java
and JavaScript.
3.10
====

2
doc/cql3/CQL.textile
View File

@ -2078,7 +2078,7 @@ SELECT AVG(players) FROM plays;
h2(#udfs). User-Defined Functions
User-defined functions allow execution of user-provided code in Cassandra. By default, Cassandra supports defining functions in _Java_ and _JavaScript_. Support for other JSR 223 compliant scripting languages (such as Python, Ruby, and Scala) can be added by adding a JAR to the classpath.
User-defined functions allow execution of user-provided code in Cassandra. By default, Cassandra supports defining functions in _Java_ and _JavaScript_. Support for other JSR 223 compliant scripting languages (such as Python, Ruby, and Scala) has been removed in 3.0.11.
UDFs are part of the Cassandra schema. As such, they are automatically propagated to all nodes in the cluster.

8
lib/jsr223/clojure/README.txt
View File

@ -1,8 +0,0 @@
Apache Cassandra User-Defined-Functions JSR 223 scripting
=========================================================
Unfortunately the JSR-223 support provided by the project https://github.com/ato/clojure-jsr223
and the related ones do not provide compileable script support.
The JSR-223 javax.script.Compilable implementation takes source file names or readers but not script sources
as all other JSR-223 implementations do.

35
lib/jsr223/groovy/README.txt
View File

@ -1,35 +0,0 @@
Apache Cassandra User-Defined-Functions JSR 223 scripting
=========================================================
Using JSR-223 capable Groovy
Tested with version 2.3.6
Installation
------------
1. Download Groovy binary release
2. Unpack the downloaded archive into a temporary directory
3. Copy the jar groovy-all-2.3.6-indy.jar from the Groovy embeddable directory to $CASSANDRA_HOME/lib/jsr223/groovy
"indy" means "invokedynamic" and is a JVM instruction for scripting languages new to Java 7.
4. Restart your Cassandra daemon if it's already running
Cassandra log should contain a line like this:
INFO 10:49:45 Found scripting engine Groovy Scripting Engine 2.0 - Groovy 2.3.6 - language names: [groovy, Groovy]
Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
Smoke Test
----------
To test Groovy functionality, open cqlsh and execute the following command:
CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE groovy AS 'return "foo";' ;
If you get the error
code=2200 [Invalid query] message="Invalid language groovy for 'foobar'"
Groovy for Apache Cassandra has not been installed correctly.
Notes / Java7 invokedynamic
---------------------------
Groovy provides jars that support invokedynamic bytecode instruction. These jars are whose ending with
"-indy.jar".

5
lib/jsr223/jaskell/README.txt
View File

@ -1,5 +0,0 @@
Apache Cassandra User-Defined-Functions JSR 223 scripting
=========================================================
Unfortunately Jaskell JSR-223 support is quite old and the Jaskell engine seems to be quite
unsupported. If you find a solution, please open a ticket at Apache Cassandra JIRA.

54
lib/jsr223/jruby/README.txt
View File

@ -1,54 +0,0 @@
Apache Cassandra User-Defined-Functions JSR 223 scripting
=========================================================
Using JSR-223 capable JRuby
Tested with version 1.7.15
Installation
------------
1. Download JRuby binary release
2. Unpack the downloaded archive into a temporary directory
3. Copy everything from the JRuby lib directory to $CASSANDRA_HOME/lib/jsr223/jruby
4. Restart your Cassandra daemon if it's already running
Cassandra log should contain a line like this:
INFO 10:29:03 Found scripting engine JSR 223 JRuby Engine 1.7.15 - ruby jruby 1.7.15 - language names: [ruby, jruby]
Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
Smoke Test
----------
To test JRuby functionality, open cqlsh and execute the following command:
CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE ruby AS 'return "foo";' ;
If you get the error
code=2200 [Invalid query] message="Invalid language ruby for 'foobar'"
JRuby for Apache Cassandra has not been installed correctly.
Ruby require/include
--------------------
You can use Ruby require and include in your scripts as in the following example:
CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE ruby AS '
require "bigdecimal"
require "bigdecimal/math"
include BigMath
a = BigDecimal((PI(100)/2).to_s)
return "foo " + a.to_s;
' ;
Notes / Java7 invokedynamic
---------------------------
See JRuby wiki pages https://github.com/jruby/jruby/wiki/ConfiguringJRuby and
https://github.com/jruby/jruby/wiki/PerformanceTuning for more information and optimization tips.

33
lib/jsr223/jython/README.txt
View File

@ -1,33 +0,0 @@
Apache Cassandra User-Defined-Functions JSR 223 scripting
=========================================================
Using JSR-223 capable Jython
Tested with version 2.3.5
Installation
------------
1. Download Jython binary release
2. Unpack the downloaded archive into a temporary directory
3. Copy the jar jython.jar from the Jython directory to $CASSANDRA_HOME/lib/jsr223/jython
4. Restart your Cassandra daemon if it's already running
Cassandra log should contain a line like this:
INFO 10:58:18 Found scripting engine jython 2.5.3 - python 2.5 - language names: [python, jython]
Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
Smoke Test
----------
To test Jython functionality, open cqlsh and execute the following command:
CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE python AS '''foo''' ;
If you get the error
code=2200 [Invalid query] message="Invalid language python for 'foobar'"
Jython for Apache Cassandra has not been installed correctly.
Notes / Java7 invokedynamic
---------------------------
Jython currently targets Java6 only. They want to switch to Java7 + invokedynamic in Jython 3.

37
lib/jsr223/scala/README.txt
View File

@ -1,37 +0,0 @@
Apache Cassandra User-Defined-Functions JSR 223 scripting
=========================================================
Using JSR-223 capable Scala
Tested with version 2.11.2
Installation
------------
1. Download Scala binary release
2. Unpack the downloaded archive into a temporary directory
3. Copy the following jars from the Scala lib directory to $CASSANDRA_HOME/lib/jsr223/scala
scala-compiler.jar
scala-library.jar
scala-reflect.jar
4. Restart your Cassandra daemon if it's already running
Cassandra log should contain a line like this:
INFO 11:42:35 Found scripting engine Scala Interpreter 1.0 - Scala version 2.11.2 - language names: [scala]
Such a line appears when you already have scripted UDFs in your system or add a scripted UDF for the first time (see below).
Smoke Test
----------
To test Scala functionality, open cqlsh and execute the following command:
CREATE OR REPLACE FUNCTION foobar ( input text ) RETURNS text LANGUAGE scala AS 'return "foo";' ;
If you get the error
code=2200 [Invalid query] message="Invalid language scala for 'foobar'"
Scala for Apache Cassandra has not been installed correctly.
Notes / Java7 invokedynamic
---------------------------
Scala 2.10 has Java6 support only. 2.11 has experimental invokedynamic support (use at your own risk!).
2.12 introduces an upgrade directly to Java8 - see https://stackoverflow.com/questions/14285894/advantages-of-scala-emitting-bytecode-for-the-jvm-1-7

35
src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDFunction.java
View File

@ -28,6 +28,9 @@ import java.util.concurrent.ExecutorService;
import javax.script.*;
import jdk.nashorn.api.scripting.AbstractJSObject;
import jdk.nashorn.api.scripting.ClassFilter;
import jdk.nashorn.api.scripting.NashornScriptEngine;
import jdk.nashorn.api.scripting.NashornScriptEngineFactory;
import org.apache.cassandra.concurrent.NamedThreadFactory;
import org.apache.cassandra.cql3.ColumnIdentifier;
import org.apache.cassandra.db.marshal.AbstractType;
@ -36,8 +39,6 @@ import org.apache.cassandra.transport.ProtocolVersion;
final class ScriptBasedUDFunction extends UDFunction
{
static final Map<String, Compilable> scriptEngines = new HashMap<>();
private static final ProtectionDomain protectionDomain;
private static final AccessControlContext accessControlContext;
@ -93,23 +94,17 @@ final class ScriptBasedUDFunction extends UDFunction
UDFunction::initializeThread)),
"userscripts");
private static final ClassFilter classFilter = clsName -> secureResource(clsName.replace('.', '/') + ".class");
private static final NashornScriptEngine scriptEngine;
static
{
ScriptEngineManager scriptEngineManager = new ScriptEngineManager();
for (ScriptEngineFactory scriptEngineFactory : scriptEngineManager.getEngineFactories())
{
ScriptEngine scriptEngine = scriptEngineFactory.getScriptEngine();
boolean compilable = scriptEngine instanceof Compilable;
if (compilable)
{
logger.info("Found scripting engine {} {} - {} {} - language names: {}",
scriptEngineFactory.getEngineName(), scriptEngineFactory.getEngineVersion(),
scriptEngineFactory.getLanguageName(), scriptEngineFactory.getLanguageVersion(),
scriptEngineFactory.getNames());
for (String name : scriptEngineFactory.getNames())
scriptEngines.put(name, (Compilable) scriptEngine);
}
}
ScriptEngine engine = scriptEngineManager.getEngineByName("nashorn");
NashornScriptEngineFactory factory = engine != null ? (NashornScriptEngineFactory) engine.getFactory() : null;
scriptEngine = factory != null ? (NashornScriptEngine) factory.getScriptEngine(new String[]{}, udfClassLoader, classFilter) : null;
try
{
@ -141,8 +136,7 @@ final class ScriptBasedUDFunction extends UDFunction
{
super(name, argNames, argTypes, returnType, calledOnNullInput, language, body);
Compilable scriptEngine = scriptEngines.get(language);
if (scriptEngine == null)
if (!"JavaScript".equalsIgnoreCase(language) || scriptEngine == null)
throw new InvalidRequestException(String.format("Invalid language '%s' for function '%s'", language, name));
// execute compilation with no-permissions to prevent evil code e.g. via "static code blocks" / "class initialization"
@ -161,10 +155,7 @@ final class ScriptBasedUDFunction extends UDFunction
// It's not always possible to simply pass a plain Java object as a binding to Nashorn and
// let the script execute methods on it.
udfContextBinding =
("Oracle Nashorn".equals(((ScriptEngine) scriptEngine).getFactory().getEngineName()))
? new UDFContextWrapper()
: udfContext;
udfContextBinding = new UDFContextWrapper();
}
protected ExecutorService executor()

3
test/unit/org/apache/cassandra/cql3/validation/entities/UFScriptTest.java
View File

@ -392,8 +392,7 @@ public class UFScriptTest extends CQLTester
DatabaseDescriptor.enableScriptedUserDefinedFunctions(false);
try
{
assertInvalid("double",
"CREATE OR REPLACE FUNCTION " + KEYSPACE + ".assertNotEnabled(val double) " +
assertInvalid("CREATE OR REPLACE FUNCTION " + KEYSPACE + ".assertNotEnabled(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE javascript\n" +

268
test/unit/org/apache/cassandra/cql3/validation/entities/UFSecurityTest.java Normal file
View File

@ -0,0 +1,268 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.cassandra.cql3.validation.entities;
import java.security.AccessControlException;
import java.util.List;
import org.junit.Assert;
import org.junit.Test;
import org.apache.cassandra.config.Config;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.cql3.CQLTester;
import org.apache.cassandra.cql3.functions.UDHelper;
import org.apache.cassandra.exceptions.FunctionExecutionException;
import org.apache.cassandra.service.ClientWarn;
public class UFSecurityTest extends CQLTester
{
@Test
public void testSecurityPermissions() throws Throwable
{
createTable("CREATE TABLE %s (key int primary key, dval double)");
execute("INSERT INTO %s (key, dval) VALUES (?, ?)", 1, 1d);
// Java UDFs
try
{
String fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE JAVA\n" +
"AS 'System.getProperty(\"foo.bar.baz\"); return 0d;';");
execute("SELECT " + fName + "(dval) FROM %s WHERE key=1");
Assert.fail();
}
catch (FunctionExecutionException e)
{
assertAccessControlException("System.getProperty(\"foo.bar.baz\"); return 0d;", e);
}
String[][] typesAndSources =
{
{"", "try { Class.forName(\"" + UDHelper.class.getName() + "\"); } catch (Exception e) { throw new RuntimeException(e); } return 0d;"},
{"sun.misc.Unsafe", "sun.misc.Unsafe.getUnsafe(); return 0d;"},
{"", "try { Class.forName(\"sun.misc.Unsafe\"); } catch (Exception e) { throw new RuntimeException(e); } return 0d;"},
{"java.nio.file.FileSystems", "try {" +
" java.nio.file.FileSystems.getDefault(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.nio.channels.FileChannel", "try {" +
" java.nio.channels.FileChannel.open(java.nio.file.FileSystems.getDefault().getPath(\"/etc/passwd\")).close(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.nio.channels.SocketChannel", "try {" +
" java.nio.channels.SocketChannel.open().close(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.io.FileInputStream", "try {" +
" new java.io.FileInputStream(\"./foobar\").close(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.lang.Runtime", "try {" +
" java.lang.Runtime.getRuntime(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"org.apache.cassandra.service.StorageService",
"try {" +
" org.apache.cassandra.service.StorageService v = org.apache.cassandra.service.StorageService.instance; v.isShutdown(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.net.ServerSocket", "try {" +
" new java.net.ServerSocket().bind(); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.io.FileOutputStream","try {" +
" new java.io.FileOutputStream(\".foo\"); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'},
{"java.lang.Runtime", "try {" +
" java.lang.Runtime.getRuntime().exec(\"/tmp/foo\"); return 0d;" +
"} catch (Exception t) {" +
" throw new RuntimeException(t);" +
'}'}
};
for (String[] typeAndSource : typesAndSources)
{
assertInvalidMessage(typeAndSource[0] + " cannot be resolved",
"CREATE OR REPLACE FUNCTION " + KEYSPACE + ".invalid_class_access(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE JAVA\n" +
"AS '" + typeAndSource[1] + "';");
}
// JavaScript UDFs
try
{
String fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE javascript\n" +
"AS 'org.apache.cassandra.service.StorageService.instance.isShutdown(); 0;';");
execute("SELECT " + fName + "(dval) FROM %s WHERE key=1");
Assert.fail("Javascript security check failed");
}
catch (FunctionExecutionException e)
{
assertAccessControlException("", e);
}
String[] javascript =
{
"java.lang.management.ManagmentFactory.getThreadMXBean(); 0;",
"new java.io.FileInputStream(\"/tmp/foo\"); 0;",
"new java.io.FileOutputStream(\"/tmp/foo\"); 0;",
"java.nio.file.FileSystems.getDefault().createFileExclusively(\"./foo_bar_baz\"); 0;",
"java.nio.channels.FileChannel.open(java.nio.file.FileSystems.getDefault().getPath(\"/etc/passwd\")); 0;",
"java.nio.channels.SocketChannel.open(); 0;",
"new java.net.ServerSocket().bind(null); 0;",
"var thread = new java.lang.Thread(); thread.start(); 0;",
"java.lang.System.getProperty(\"foo.bar.baz\"); 0;",
"java.lang.Runtime.getRuntime().exec(\"/tmp/foo\"); 0;",
"java.lang.Runtime.getRuntime().loadLibrary(\"foobar\"); 0;",
"java.lang.Runtime.getRuntime().loadLibrary(\"foobar\"); 0;",
// TODO these (ugly) calls are still possible - these can consume CPU (as one could do with an evil loop, too)
// "java.lang.Runtime.getRuntime().traceMethodCalls(true); 0;",
// "java.lang.Runtime.getRuntime().gc(); 0;",
// "java.lang.Runtime.getRuntime(); 0;",
};
for (String script : javascript)
{
try
{
String fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE javascript\n" +
"AS '" + script + "';");
execute("SELECT " + fName + "(dval) FROM %s WHERE key=1");
Assert.fail("Javascript security check failed: " + script);
}
catch (FunctionExecutionException e)
{
assertAccessControlException(script, e);
}
}
String script = "java.lang.Class.forName(\"java.lang.System\"); 0;";
String fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE javascript\n" +
"AS '" + script + "';");
assertInvalidThrowMessage("Java reflection not supported when class filter is present",
FunctionExecutionException.class,
"SELECT " + fName + "(dval) FROM %s WHERE key=1");
}
private static void assertAccessControlException(String script, FunctionExecutionException e)
{
for (Throwable t = e; t != null && t != t.getCause(); t = t.getCause())
if (t instanceof AccessControlException)
return;
Assert.fail("no AccessControlException for " + script + " (got " + e + ')');
}
@Test
public void testAmokUDF() throws Throwable
{
createTable("CREATE TABLE %s (key int primary key, dval double)");
execute("INSERT INTO %s (key, dval) VALUES (?, ?)", 1, 1d);
long udfWarnTimeout = DatabaseDescriptor.getUserDefinedFunctionWarnTimeout();
long udfFailTimeout = DatabaseDescriptor.getUserDefinedFunctionFailTimeout();
int maxTries = 5;
for (int i = 1; i <= maxTries; i++)
{
try
{
// short timeout
DatabaseDescriptor.setUserDefinedFunctionWarnTimeout(10);
DatabaseDescriptor.setUserDefinedFunctionFailTimeout(250);
// don't kill the unit test... - default policy is "die"
DatabaseDescriptor.setUserFunctionTimeoutPolicy(Config.UserFunctionTimeoutPolicy.ignore);
ClientWarn.instance.captureWarnings();
String fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE JAVA\n" +
"AS 'long t=System.currentTimeMillis()+110; while (t>System.currentTimeMillis()) { }; return 0d;'");
execute("SELECT " + fName + "(dval) FROM %s WHERE key=1");
List<String> warnings = ClientWarn.instance.getWarnings();
Assert.assertNotNull(warnings);
Assert.assertFalse(warnings.isEmpty());
ClientWarn.instance.resetWarnings();
// Java UDF
fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE JAVA\n" +
"AS 'long t=System.currentTimeMillis()+500; while (t>System.currentTimeMillis()) { }; return 0d;';");
assertInvalidMessage("ran longer than 250ms", "SELECT " + fName + "(dval) FROM %s WHERE key=1");
// Javascript UDF
fName = createFunction(KEYSPACE_PER_TEST, "double",
"CREATE OR REPLACE FUNCTION %s(val double) " +
"RETURNS NULL ON NULL INPUT " +
"RETURNS double " +
"LANGUAGE JAVASCRIPT\n" +
"AS 'var t=java.lang.System.currentTimeMillis()+500; while (t>java.lang.System.currentTimeMillis()) { }; 0;';");
assertInvalidMessage("ran longer than 250ms", "SELECT " + fName + "(dval) FROM %s WHERE key=1");
return;
}
catch (Error | RuntimeException e)
{
if (i == maxTries)
throw e;
}
finally
{
// reset to defaults
DatabaseDescriptor.setUserDefinedFunctionWarnTimeout(udfWarnTimeout);
DatabaseDescriptor.setUserDefinedFunctionFailTimeout(udfFailTimeout);
}
}
}
}