natas_hw
/
cassandra
mirror of https://github.com/apache/cassandra
0
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove COMPACT STORAGE internals 

Patch by Ekaterina Dimitrova, reviewed by Sylvain Lebresne for
CASSANDRA-13994
This commit is contained in:
Ekaterina Dimitrova 2020-04-16 08:31:34 -04:00 committed by Sylvain Lebresne
parent efce6b39fb
commit cba0c27ce9
55 changed files with 435 additions and 1105 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGES.txt
View File

@ -1,4 +1,5 @@
4.0-beta2
* Remove COMPACT STORAGE internals (CASSANDRA-13994)
* Make TimestampSerializer accept fractional seconds of varying precision (CASSANDRA-15976)
* Improve cassandra-stress logging when using a profile file that doesn't exist (CASSANDRA-14425)
* Improve logging for socket connection/disconnection (CASSANDRA-15980)

35
doc/source/cql/ddl.rst
View File

@ -244,8 +244,7 @@ Creating a new table uses the ``CREATE TABLE`` statement:
partition_key: `column_name`
: | '(' `column_name` ( ',' `column_name` )* ')'
clustering_columns: `column_name` ( ',' `column_name` )*
table_options: COMPACT STORAGE [ AND `table_options` ]
: | CLUSTERING ORDER BY '(' `clustering_order` ')' [ AND `table_options` ]
table_options: CLUSTERING ORDER BY '(' `clustering_order` ')' [ AND `options` ]
: | `options`
clustering_order: `column_name` (ASC | DESC) ( ',' `column_name` (ASC | DESC) )*
@ -330,7 +329,6 @@ that example being ``pk``, both rows are in that same partition): the 2nd insert
The use of static columns as the following restrictions:
- tables with the ``COMPACT STORAGE`` option (see below) cannot use them.
- a table without clustering columns cannot have static columns (in a table without clustering columns, every partition
has only one row, and so every column is inherently static).
- only non ``PRIMARY KEY`` columns can be static.
@ -452,32 +450,11 @@ Table options
~~~~~~~~~~~~~
A CQL table has a number of options that can be set at creation (and, for most of them, :ref:`altered
<alter-table-statement>` later). These options are specified after the ``WITH`` keyword.
<alter-table-statement>` later). These options are specified after the ``WITH`` keyword and are described
in the following sections.
Amongst those options, two important ones cannot be changed after creation and influence which queries can be done
against the table: the ``COMPACT STORAGE`` option and the ``CLUSTERING ORDER`` option. Those, as well as the other
options of a table are described in the following sections.
.. _compact-tables:
Compact tables
``````````````
.. warning:: Since Cassandra 3.0, compact tables have the exact same layout internally than non compact ones (for the
same schema obviously), and declaring a table compact **only** creates artificial limitations on the table definition
and usage. It only exists for historical reason and is preserved for backward compatibility And as ``COMPACT
STORAGE`` cannot, as of Cassandra |version|, be removed, it is strongly discouraged to create new table with the
``COMPACT STORAGE`` option.
A *compact* table is one defined with the ``COMPACT STORAGE`` option. This option is only maintained for backward
compatibility for definitions created before CQL version 3 and shouldn't be used for new tables. Declaring a
table with this option creates limitations for the table which are largely arbitrary (and exists for historical
reasons). Amongst those limitation:
- a compact table cannot use collections nor static columns.
- if a compact table has at least one clustering column, then it must have *exactly* one column outside of the primary
key ones. This imply you cannot add or remove columns after creation in particular.
- a compact table is limited in the indexes it can create, and no materialized view can be created on it.
But please bear in mind that the ``CLUSTERING ORDER`` option cannot be changed after table creation and
has influences on the performance of some queries.
.. _clustering-order:
@ -807,7 +784,7 @@ The ``ALTER TABLE`` statement can:
below. Due to lazy removal, the altering itself is a constant (in the amount of data removed or contained in the
cluster) time operation.
- Change some of the table options (through the ``WITH`` instruction). The :ref:`supported options
<create-table-options>` are the same that when creating a table (outside of ``COMPACT STORAGE`` and ``CLUSTERING
<create-table-options>` are the same that when creating a table (outside of ``CLUSTERING
ORDER`` that cannot be changed after creation). Note that setting any ``compaction`` sub-options has the effect of
erasing all previous ``compaction`` options, so you need to re-specify all the sub-options if you want to keep them.
The same note applies to the set of ``compression`` sub-options.

87
src/antlr/Parser.g
View File

@ -136,9 +136,9 @@ options {
return res;
}
public void addRawUpdate(List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations, ColumnMetadata.Raw key, Operation.RawUpdate update)
public void addRawUpdate(List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations, ColumnIdentifier key, Operation.RawUpdate update)
{
for (Pair<ColumnMetadata.Raw, Operation.RawUpdate> p : operations)
for (Pair<ColumnIdentifier, Operation.RawUpdate> p : operations)
{
if (p.left.equals(key) && !p.right.isCompatibleWith(update))
addRecognitionError("Multiple incompatible setting of column " + key);
@ -266,8 +266,8 @@ selectStatement returns [SelectStatement.RawStatement expr]
@init {
Term.Raw limit = null;
Term.Raw perPartitionLimit = null;
Map<ColumnMetadata.Raw, Boolean> orderings = new LinkedHashMap<>();
List<ColumnMetadata.Raw> groups = new ArrayList<>();
Map<ColumnIdentifier, Boolean> orderings = new LinkedHashMap<>();
List<ColumnIdentifier> groups = new ArrayList<>();
boolean allowFiltering = false;
boolean isJson = false;
}
@ -415,8 +415,8 @@ simpleUnaliasedSelector returns [Selectable.Raw s]
selectionFunction returns [Selectable.Raw s]
: K_COUNT '(' '\*' ')' { $s = Selectable.WithFunction.Raw.newCountRowsFunction(); }
| K_WRITETIME '(' c=cident ')' { $s = new Selectable.WritetimeOrTTL.Raw(c, true); }
| K_TTL '(' c=cident ')' { $s = new Selectable.WritetimeOrTTL.Raw(c, false); }
| K_WRITETIME '(' c=sident ')' { $s = new Selectable.WritetimeOrTTL.Raw(c, true); }
| K_TTL '(' c=sident ')' { $s = new Selectable.WritetimeOrTTL.Raw(c, false); }
| K_CAST '(' sn=unaliasedSelector K_AS t=native_type ')' {$s = new Selectable.WithCast.Raw(sn, t);}
| f=functionName args=selectionFunctionArgs { $s = new Selectable.WithFunction.Raw(f, args); }
;
@ -435,7 +435,7 @@ selectionFunctionArgs returns [List<Selectable.Raw> a]
')'
;
sident returns [Selectable.Raw id]
sident returns [Selectable.RawIdentifier id]
: t=IDENT { $id = Selectable.RawIdentifier.forUnquoted($t.text); }
| t=QUOTED_NAME { $id = Selectable.RawIdentifier.forQuoted($t.text); }
| k=unreserved_keyword { $id = Selectable.RawIdentifier.forUnquoted(k); }
@ -456,14 +456,14 @@ customIndexExpression [WhereClause.Builder clause]
: 'expr(' idxName[name] ',' t=term ')' { clause.add(new CustomIndexExpression(name, t));}
;
orderByClause[Map<ColumnMetadata.Raw, Boolean> orderings]
orderByClause[Map<ColumnIdentifier, Boolean> orderings]
@init{
boolean reversed = false;
}
: c=cident (K_ASC | K_DESC { reversed = true; })? { orderings.put(c, reversed); }
;
groupByClause[List<ColumnMetadata.Raw> groups]
groupByClause[List<ColumnIdentifier> groups]
: c=cident { groups.add(c); }
;
@ -482,7 +482,7 @@ insertStatement returns [ModificationStatement.Parsed expr]
normalInsertStatement [QualifiedName qn] returns [UpdateStatement.ParsedInsert expr]
@init {
Attributes.Raw attrs = new Attributes.Raw();
List<ColumnMetadata.Raw> columnNames = new ArrayList<>();
List<ColumnIdentifier> columnNames = new ArrayList<>();
List<Term.Raw> values = new ArrayList<>();
boolean ifNotExists = false;
}
@ -536,7 +536,7 @@ usingClauseObjective[Attributes.Raw attrs]
updateStatement returns [UpdateStatement.ParsedUpdate expr]
@init {
Attributes.Raw attrs = new Attributes.Raw();
List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations = new ArrayList<>();
List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations = new ArrayList<>();
boolean ifExists = false;
}
: K_UPDATE cf=columnFamilyName
@ -549,13 +549,13 @@ updateStatement returns [UpdateStatement.ParsedUpdate expr]
attrs,
operations,
wclause.build(),
conditions == null ? Collections.<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>>emptyList() : conditions,
conditions == null ? Collections.<Pair<ColumnIdentifier, ColumnCondition.Raw>>emptyList() : conditions,
ifExists);
}
;
updateConditions returns [List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> conditions]
@init { conditions = new ArrayList<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>>(); }
updateConditions returns [List<Pair<ColumnIdentifier, ColumnCondition.Raw>> conditions]
@init { conditions = new ArrayList<Pair<ColumnIdentifier, ColumnCondition.Raw>>(); }
: columnCondition[conditions] ( K_AND columnCondition[conditions] )*
;
@ -583,7 +583,7 @@ deleteStatement returns [DeleteStatement.Parsed expr]
attrs,
columnDeletions,
wclause.build(),
conditions == null ? Collections.<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>>emptyList() : conditions,
conditions == null ? Collections.<Pair<ColumnIdentifier, ColumnCondition.Raw>>emptyList() : conditions,
ifExists);
}
;
@ -935,17 +935,17 @@ alterTableStatement returns [AlterTableStatement.Raw stmt]
(
K_ALTER id=cident K_TYPE v=comparatorType { $stmt.alter(id, v); }
| K_ADD ( id=schema_cident v=comparatorType b=isStaticColumn { $stmt.add(id, v, b); }
| ('(' id1=schema_cident v1=comparatorType b1=isStaticColumn { $stmt.add(id1, v1, b1); }
( ',' idn=schema_cident vn=comparatorType bn=isStaticColumn { $stmt.add(idn, vn, bn); } )* ')') )
| K_ADD ( id=ident v=comparatorType b=isStaticColumn { $stmt.add(id, v, b); }
| ('(' id1=ident v1=comparatorType b1=isStaticColumn { $stmt.add(id1, v1, b1); }
( ',' idn=ident vn=comparatorType bn=isStaticColumn { $stmt.add(idn, vn, bn); } )* ')') )
| K_DROP ( id=schema_cident { $stmt.drop(id); }
| ('(' id1=schema_cident { $stmt.drop(id1); }
( ',' idn=schema_cident { $stmt.drop(idn); } )* ')') )
| K_DROP ( id=ident { $stmt.drop(id); }
| ('(' id1=ident { $stmt.drop(id1); }
( ',' idn=ident { $stmt.drop(idn); } )* ')') )
( K_USING K_TIMESTAMP t=INTEGER { $stmt.timestamp(Long.parseLong(Constants.Literal.integer($t.text).getText())); } )?
| K_RENAME id1=schema_cident K_TO toId1=schema_cident { $stmt.rename(id1, toId1); }
( K_AND idn=schema_cident K_TO toIdn=schema_cident { $stmt.rename(idn, toIdn); } )*
| K_RENAME id1=ident K_TO toId1=ident { $stmt.rename(id1, toId1); }
( K_AND idn=ident K_TO toIdn=ident { $stmt.rename(idn, toIdn); } )*
| K_WITH properties[$stmt.attrs] { $stmt.attrs(); }
)
@ -1343,26 +1343,13 @@ describeStatement returns [DescribeStatement stmt]
/** DEFINITIONS **/
// Column Identifiers. These need to be treated differently from other
// identifiers because the underlying comparator is not necessarily text. See
// CASSANDRA-8178 for details.
// Also, we need to support the internal of the super column map (for backward
// compatibility) which is empty (we only want to allow this is in data manipulation
// queries, not in schema defition etc).
cident returns [ColumnMetadata.Raw id]
: EMPTY_QUOTED_NAME { $id = ColumnMetadata.Raw.forQuoted(""); }
| t=IDENT { $id = ColumnMetadata.Raw.forUnquoted($t.text); }
| t=QUOTED_NAME { $id = ColumnMetadata.Raw.forQuoted($t.text); }
| k=unreserved_keyword { $id = ColumnMetadata.Raw.forUnquoted(k); }
// Like ident, but for case where we take a column name that can be the legacy super column empty name. Importantly,
// this should not be used in DDL statements, as we don't want to let users create such column.
cident returns [ColumnIdentifier id]
: EMPTY_QUOTED_NAME { $id = ColumnIdentifier.getInterned("", true); }
| t=ident { $id = t; }
;
schema_cident returns [ColumnMetadata.Raw id]
: t=IDENT { $id = ColumnMetadata.Raw.forUnquoted($t.text); }
| t=QUOTED_NAME { $id = ColumnMetadata.Raw.forQuoted($t.text); }
| k=unreserved_keyword { $id = ColumnMetadata.Raw.forUnquoted(k); }
;
// Column identifiers where the comparator is known to be text
ident returns [ColumnIdentifier id]
: t=IDENT { $id = ColumnIdentifier.getInterned($t.text, false); }
| t=QUOTED_NAME { $id = ColumnIdentifier.getInterned($t.text, true); }
@ -1570,18 +1557,18 @@ simpleTerm returns [Term.Raw term]
| '(' c=comparatorType ')' t=simpleTerm { $term = new TypeCast(c, t); }
;
columnOperation[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations]
columnOperation[List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations]
: key=cident columnOperationDifferentiator[operations, key]
;
columnOperationDifferentiator[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations, ColumnMetadata.Raw key]
columnOperationDifferentiator[List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations, ColumnIdentifier key]
: '=' normalColumnOperation[operations, key]
| shorthandColumnOperation[operations, key]
| '[' k=term ']' collectionColumnOperation[operations, key, k]
| '.' field=fident udtColumnOperation[operations, key, field]
;
normalColumnOperation[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations, ColumnMetadata.Raw key]
normalColumnOperation[List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations, ColumnIdentifier key]
: t=term ('+' c=cident )?
{
if (c == null)
@ -1611,28 +1598,28 @@ normalColumnOperation[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operat
}
;
shorthandColumnOperation[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations, ColumnMetadata.Raw key]
shorthandColumnOperation[List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations, ColumnIdentifier key]
: sig=('+=' | '-=') t=term
{
addRawUpdate(operations, key, $sig.text.equals("+=") ? new Operation.Addition(t) : new Operation.Substraction(t));
}
;
collectionColumnOperation[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations, ColumnMetadata.Raw key, Term.Raw k]
collectionColumnOperation[List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations, ColumnIdentifier key, Term.Raw k]
: '=' t=term
{
addRawUpdate(operations, key, new Operation.SetElement(k, t));
}
;
udtColumnOperation[List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> operations, ColumnMetadata.Raw key, FieldIdentifier field]
udtColumnOperation[List<Pair<ColumnIdentifier, Operation.RawUpdate>> operations, ColumnIdentifier key, FieldIdentifier field]
: '=' t=term
{
addRawUpdate(operations, key, new Operation.SetField(field, t));
}
;
columnCondition[List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> conditions]
columnCondition[List<Pair<ColumnIdentifier, ColumnCondition.Raw>> conditions]
// Note: we'll reject duplicates later
: key=cident
( op=relationType t=term { conditions.add(Pair.create(key, ColumnCondition.Raw.simpleCondition(t, op))); }
@ -1724,8 +1711,8 @@ inMarker returns [AbstractMarker.INRaw marker]
| ':' name=noncol_ident { $marker = newINBindVariables(name); }
;
tupleOfIdentifiers returns [List<ColumnMetadata.Raw> ids]
@init { $ids = new ArrayList<ColumnMetadata.Raw>(); }
tupleOfIdentifiers returns [List<ColumnIdentifier> ids]
@init { $ids = new ArrayList<ColumnIdentifier>(); }
: '(' n1=cident { $ids.add(n1); } (',' ni=cident { $ids.add(ni); })* ')'
;

3
src/java/org/apache/cassandra/cql3/CQL3Type.java
View File

@ -138,6 +138,7 @@ public interface CQL3Type
return type;
}
@Override
public String toCQLLiteral(ByteBuffer buffer, ProtocolVersion version)
{
// *always* use the 'blob' syntax to express custom types in CQL
@ -186,6 +187,7 @@ public interface CQL3Type
return true;
}
@Override
public String toCQLLiteral(ByteBuffer buffer, ProtocolVersion version)
{
if (buffer == null)
@ -318,6 +320,7 @@ public interface CQL3Type
return type;
}
@Override
public String toCQLLiteral(ByteBuffer buffer, ProtocolVersion version)
{
if (buffer == null)

1
src/java/org/apache/cassandra/cql3/Maps.java
View File

@ -240,6 +240,7 @@ public abstract class Maps
}
}
@Override
public ByteBuffer get(ProtocolVersion protocolVersion)
{
List<ByteBuffer> buffers = new ArrayList<>(2 * map.size());

21
src/java/org/apache/cassandra/cql3/MultiColumnRelation.java
View File

@ -47,7 +47,7 @@ import static org.apache.cassandra.cql3.statements.RequestValidations.invalidReq
*/
public class MultiColumnRelation extends Relation
{
private final List<ColumnMetadata.Raw> entities;
private final List<ColumnIdentifier> entities;
/** A Tuples.Literal or Tuples.Raw marker */
private final Term.MultiColumnRaw valuesOrMarker;
@ -57,7 +57,7 @@ public class MultiColumnRelation extends Relation
private final Tuples.INRaw inMarker;
private MultiColumnRelation(List<ColumnMetadata.Raw> entities, Operator relationType, Term.MultiColumnRaw valuesOrMarker, List<? extends Term.MultiColumnRaw> inValues, Tuples.INRaw inMarker)
private MultiColumnRelation(List<ColumnIdentifier> entities, Operator relationType, Term.MultiColumnRaw valuesOrMarker, List<? extends Term.MultiColumnRaw> inValues, Tuples.INRaw inMarker)
{
this.entities = entities;
this.relationType = relationType;
@ -77,7 +77,7 @@ public class MultiColumnRelation extends Relation
* @param valuesOrMarker a Tuples.Literal instance or a Tuples.Raw marker
* @return a new <code>MultiColumnRelation</code> instance
*/
public static MultiColumnRelation createNonInRelation(List<ColumnMetadata.Raw> entities, Operator relationType, Term.MultiColumnRaw valuesOrMarker)
public static MultiColumnRelation createNonInRelation(List<ColumnIdentifier> entities, Operator relationType, Term.MultiColumnRaw valuesOrMarker)
{
assert relationType != Operator.IN;
return new MultiColumnRelation(entities, relationType, valuesOrMarker, null, null);
@ -90,7 +90,7 @@ public class MultiColumnRelation extends Relation
* @param inValues a list of Tuples.Literal instances or a Tuples.Raw markers
* @return a new <code>MultiColumnRelation</code> instance
*/
public static MultiColumnRelation createInRelation(List<ColumnMetadata.Raw> entities, List<? extends Term.MultiColumnRaw> inValues)
public static MultiColumnRelation createInRelation(List<ColumnIdentifier> entities, List<? extends Term.MultiColumnRaw> inValues)
{
return new MultiColumnRelation(entities, Operator.IN, null, inValues, null);
}
@ -102,12 +102,12 @@ public class MultiColumnRelation extends Relation
* @param inMarker a single IN marker
* @return a new <code>MultiColumnRelation</code> instance
*/
public static MultiColumnRelation createSingleMarkerInRelation(List<ColumnMetadata.Raw> entities, Tuples.INRaw inMarker)
public static MultiColumnRelation createSingleMarkerInRelation(List<ColumnIdentifier> entities, Tuples.INRaw inMarker)
{
return new MultiColumnRelation(entities, Operator.IN, null, null, inMarker);
}
public List<ColumnMetadata.Raw> getEntities()
public List<ColumnIdentifier> getEntities()
{
return entities;
}
@ -200,9 +200,9 @@ public class MultiColumnRelation extends Relation
{
List<ColumnMetadata> names = new ArrayList<>(getEntities().size());
int previousPosition = -1;
for (ColumnMetadata.Raw raw : getEntities())
for (ColumnIdentifier id : getEntities())
{
ColumnMetadata def = raw.prepare(table);
ColumnMetadata def = table.getExistingColumn(id);
checkTrue(def.isClusteringColumn(), "Multi-column relations can only be applied to clustering columns but was applied to: %s", def.name);
checkFalse(names.contains(def), "Column \"%s\" appeared twice in a relation: %s", def.name, this);
@ -216,12 +216,13 @@ public class MultiColumnRelation extends Relation
return names;
}
public Relation renameIdentifier(ColumnMetadata.Raw from, ColumnMetadata.Raw to)
@Override
public Relation renameIdentifier(ColumnIdentifier from, ColumnIdentifier to)
{
if (!entities.contains(from))
return this;
List<ColumnMetadata.Raw> newEntities = entities.stream().map(e -> e.equals(from) ? to : e).collect(Collectors.toList());
List<ColumnIdentifier> newEntities = entities.stream().map(e -> e.equals(from) ? to : e).collect(Collectors.toList());
return new MultiColumnRelation(newEntities, operator(), valuesOrMarker, inValues, inMarker);
}

20
src/java/org/apache/cassandra/cql3/Operation.java
View File

@ -135,7 +135,7 @@ public abstract class Operation
/**
* The name of the column affected by this delete operation.
*/
public ColumnMetadata.Raw affectedColumn();
public ColumnIdentifier affectedColumn();
/**
* This method validates the operation (i.e. validate it is well typed)
@ -436,14 +436,14 @@ public abstract class Operation
public static class ColumnDeletion implements RawDeletion
{
private final ColumnMetadata.Raw id;
private final ColumnIdentifier id;
public ColumnDeletion(ColumnMetadata.Raw id)
public ColumnDeletion(ColumnIdentifier id)
{
this.id = id;
}
public ColumnMetadata.Raw affectedColumn()
public ColumnIdentifier affectedColumn()
{
return id;
}
@ -457,16 +457,16 @@ public abstract class Operation
public static class ElementDeletion implements RawDeletion
{
private final ColumnMetadata.Raw id;
private final ColumnIdentifier id;
private final Term.Raw element;
public ElementDeletion(ColumnMetadata.Raw id, Term.Raw element)
public ElementDeletion(ColumnIdentifier id, Term.Raw element)
{
this.id = id;
this.element = element;
}
public ColumnMetadata.Raw affectedColumn()
public ColumnIdentifier affectedColumn()
{
return id;
}
@ -496,16 +496,16 @@ public abstract class Operation
public static class FieldDeletion implements RawDeletion
{
private final ColumnMetadata.Raw id;
private final ColumnIdentifier id;
private final FieldIdentifier field;
public FieldDeletion(ColumnMetadata.Raw id, FieldIdentifier field)
public FieldDeletion(ColumnIdentifier id, FieldIdentifier field)
{
this.id = id;
this.field = field;
}
public ColumnMetadata.Raw affectedColumn()
public ColumnIdentifier affectedColumn()
{
return id;
}

3
src/java/org/apache/cassandra/cql3/Relation.java
View File

@ -21,7 +21,6 @@ import java.util.ArrayList;
import java.util.List;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.cql3.restrictions.Restriction;
import org.apache.cassandra.cql3.statements.Bound;
import org.apache.cassandra.exceptions.InvalidRequestException;
@ -257,5 +256,5 @@ public abstract class Relation
* @return this object, if the old identifier is not in the set of entities that this relation covers; otherwise
* a new Relation with "from" replaced by "to" is returned.
*/
public abstract Relation renameIdentifier(ColumnMetadata.Raw from, ColumnMetadata.Raw to);
public abstract Relation renameIdentifier(ColumnIdentifier from, ColumnIdentifier to);
}

26
src/java/org/apache/cassandra/cql3/SingleColumnRelation.java
View File

@ -44,12 +44,12 @@ import static org.apache.cassandra.cql3.statements.RequestValidations.invalidReq
*/
public final class SingleColumnRelation extends Relation
{
private final ColumnMetadata.Raw entity;
private final ColumnIdentifier entity;
private final Term.Raw mapKey;
private final Term.Raw value;
private final List<Term.Raw> inValues;
private SingleColumnRelation(ColumnMetadata.Raw entity, Term.Raw mapKey, Operator type, Term.Raw value, List<Term.Raw> inValues)
private SingleColumnRelation(ColumnIdentifier entity, Term.Raw mapKey, Operator type, Term.Raw value, List<Term.Raw> inValues)
{
this.entity = entity;
this.mapKey = mapKey;
@ -69,7 +69,7 @@ public final class SingleColumnRelation extends Relation
* @param type the type that describes how this entity relates to the value.
* @param value the value being compared.
*/
public SingleColumnRelation(ColumnMetadata.Raw entity, Term.Raw mapKey, Operator type, Term.Raw value)
public SingleColumnRelation(ColumnIdentifier entity, Term.Raw mapKey, Operator type, Term.Raw value)
{
this(entity, mapKey, type, value, null);
}
@ -81,7 +81,7 @@ public final class SingleColumnRelation extends Relation
* @param type the type that describes how this entity relates to the value.
* @param value the value being compared.
*/
public SingleColumnRelation(ColumnMetadata.Raw entity, Operator type, Term.Raw value)
public SingleColumnRelation(ColumnIdentifier entity, Operator type, Term.Raw value)
{
this(entity, null, type, value);
}
@ -96,12 +96,12 @@ public final class SingleColumnRelation extends Relation
return inValues;
}
public static SingleColumnRelation createInRelation(ColumnMetadata.Raw entity, List<Term.Raw> inValues)
public static SingleColumnRelation createInRelation(ColumnIdentifier entity, List<Term.Raw> inValues)
{
return new SingleColumnRelation(entity, null, Operator.IN, null, inValues);
}
public ColumnMetadata.Raw getEntity()
public ColumnIdentifier getEntity()
{
return entity;
}
@ -135,7 +135,7 @@ public final class SingleColumnRelation extends Relation
}
}
public Relation renameIdentifier(ColumnMetadata.Raw from, ColumnMetadata.Raw to)
public Relation renameIdentifier(ColumnIdentifier from, ColumnIdentifier to)
{
return entity.equals(from)
? new SingleColumnRelation(to, mapKey, operator(), value, inValues)
@ -181,7 +181,7 @@ public final class SingleColumnRelation extends Relation
@Override
protected Restriction newEQRestriction(TableMetadata table, VariableSpecifications boundNames)
{
ColumnMetadata columnDef = entity.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(entity);
if (mapKey == null)
{
Term term = toTerm(toReceivers(columnDef), value, table.keyspace, boundNames);
@ -196,7 +196,7 @@ public final class SingleColumnRelation extends Relation
@Override
protected Restriction newINRestriction(TableMetadata table, VariableSpecifications boundNames)
{
ColumnMetadata columnDef = entity.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(entity);
List<? extends ColumnSpecification> receivers = toReceivers(columnDef);
List<Term> terms = toTerms(receivers, inValues, table.keyspace, boundNames);
if (terms == null)
@ -218,7 +218,7 @@ public final class SingleColumnRelation extends Relation
Bound bound,
boolean inclusive)
{
ColumnMetadata columnDef = entity.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(entity);
if (columnDef.type.referencesDuration())
{
@ -237,7 +237,7 @@ public final class SingleColumnRelation extends Relation
VariableSpecifications boundNames,
boolean isKey) throws InvalidRequestException
{
ColumnMetadata columnDef = entity.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(entity);
Term term = toTerm(toReceivers(columnDef), value, table.keyspace, boundNames);
return new SingleColumnRestriction.ContainsRestriction(columnDef, term, isKey);
}
@ -246,7 +246,7 @@ public final class SingleColumnRelation extends Relation
protected Restriction newIsNotRestriction(TableMetadata table,
VariableSpecifications boundNames) throws InvalidRequestException
{
ColumnMetadata columnDef = entity.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(entity);
// currently enforced by the grammar
assert value == Constants.NULL_LITERAL : "Expected null literal for IS NOT relation: " + this.toString();
return new SingleColumnRestriction.IsNotNullRestriction(columnDef);
@ -258,7 +258,7 @@ public final class SingleColumnRelation extends Relation
if (mapKey != null)
throw invalidRequest("%s can't be used with collections.", operator());
ColumnMetadata columnDef = entity.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(entity);
Term term = toTerm(toReceivers(columnDef), value, table.keyspace, boundNames);
return new SingleColumnRestriction.LikeRestriction(columnDef, operator, term);

13
src/java/org/apache/cassandra/cql3/TokenRelation.java
View File

@ -48,11 +48,11 @@ import static org.apache.cassandra.cql3.statements.RequestValidations.invalidReq
*/
public final class TokenRelation extends Relation
{
private final List<ColumnMetadata.Raw> entities;
private final List<ColumnIdentifier> entities;
private final Term.Raw value;
public TokenRelation(List<ColumnMetadata.Raw> entities, Operator type, Term.Raw value)
public TokenRelation(List<ColumnIdentifier> entities, Operator type, Term.Raw value)
{
this.entities = entities;
this.relationType = type;
@ -129,12 +129,13 @@ public final class TokenRelation extends Relation
return term;
}
public Relation renameIdentifier(ColumnMetadata.Raw from, ColumnMetadata.Raw to)
@Override
public Relation renameIdentifier(ColumnIdentifier from, ColumnIdentifier to)
{
if (!entities.contains(from))
return this;
List<ColumnMetadata.Raw> newEntities = entities.stream().map(e -> e.equals(from) ? to : e).collect(Collectors.toList());
List<ColumnIdentifier> newEntities = entities.stream().map(e -> e.equals(from) ? to : e).collect(Collectors.toList());
return new TokenRelation(newEntities, operator(), value);
}
@ -173,8 +174,8 @@ public final class TokenRelation extends Relation
private List<ColumnMetadata> getColumnDefinitions(TableMetadata table)
{
List<ColumnMetadata> columnDefs = new ArrayList<>(entities.size());
for ( ColumnMetadata.Raw raw : entities)
columnDefs.add(raw.prepare(table));
for (ColumnIdentifier id : entities)
columnDefs.add(table.getExistingColumn(id));
return columnDefs;
}

18
src/java/org/apache/cassandra/cql3/UpdateParameters.java
View File

@ -82,16 +82,6 @@ public class UpdateParameters
public void newRow(Clustering clustering) throws InvalidRequestException
{
if (metadata.isDense() && !metadata.isCompound())
{
// If it's a COMPACT STORAGE table with a single clustering column and for backward compatibility we
// don't want to allow that to be empty (even though this would be fine for the storage engine).
assert clustering.size() == 1;
ByteBuffer value = clustering.get(0);
if (value == null || !value.hasRemaining())
throw new InvalidRequestException("Invalid empty or null value for column " + metadata.clusteringColumns().get(0).name);
}
if (clustering == Clustering.STATIC_CLUSTERING)
{
if (staticBuilder == null)
@ -120,13 +110,7 @@ public class UpdateParameters
public void addRowDeletion()
{
// For compact tables, at the exclusion of the static row (of static compact tables), each row ever has a single column,
// the "compact" one. As such, deleting the row or deleting that single cell is equivalent. We favor the later
// for backward compatibility (thought it doesn't truly matter anymore).
if (metadata.isCompactTable() && builder.clustering() != Clustering.STATIC_CLUSTERING)
addTombstone(metadata.compactValueColumn);
else
builder.addRowDeletion(Row.Deletion.regular(deletionTime));
builder.addRowDeletion(Row.Deletion.regular(deletionTime));
}
public void addTombstone(ColumnMetadata column) throws InvalidRequestException

3
src/java/org/apache/cassandra/cql3/WhereClause.java
View File

@ -23,7 +23,6 @@ import java.util.Objects;
import com.google.common.collect.ImmutableList;
import org.antlr.runtime.RecognitionException;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.cql3.restrictions.CustomIndexExpression;
import static java.lang.String.join;
@ -60,7 +59,7 @@ public final class WhereClause
* @param to the new identifier
* @return a new WhereClause with with "from" replaced by "to" in all relations
*/
public WhereClause renameIdentifier(ColumnMetadata.Raw from, ColumnMetadata.Raw to)
public WhereClause renameIdentifier(ColumnIdentifier from, ColumnIdentifier to)
{
WhereClause.Builder builder = new WhereClause.Builder();

15
src/java/org/apache/cassandra/cql3/restrictions/StatementRestrictions.java
View File

@ -35,7 +35,6 @@ import org.apache.cassandra.index.Index;
import org.apache.cassandra.index.IndexRegistry;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.utils.btree.BTreeSet;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
@ -510,8 +509,7 @@ public final class StatementRestrictions
checkFalse(!type.allowClusteringColumnSlices() && clusteringColumnsRestrictions.hasSlice(),
"Slice restrictions are not supported on the clustering columns in %s statements", type);
if (!type.allowClusteringColumnSlices()
&& (!table.isCompactTable() || (table.isCompactTable() && !hasClusteringColumnsRestrictions())))
if (!type.allowClusteringColumnSlices())
{
if (!selectsOnlyStaticColumns && hasUnrestrictedClusteringColumns())
throw invalidRequest("Some clustering keys are missing: %s",
@ -747,12 +745,6 @@ public final class StatementRestrictions
*/
public NavigableSet<Clustering> getClusteringColumns(QueryOptions options)
{
// If this is a names command and the table is a static compact one, then as far as CQL is concerned we have
// only a single row which internally correspond to the static parts. In which case we want to return an empty
// set (since that's what ClusteringIndexNamesFilter expects).
if (table.isStaticCompactTable())
return BTreeSet.empty(table.comparator);
return clusteringColumnsRestrictions.valuesAsClustering(options);
}
@ -775,10 +767,7 @@ public final class StatementRestrictions
*/
public boolean isColumnRange()
{
// For static compact tables we want to ignore the fake clustering column (note that if we weren't special casing,
// this would mean a 'SELECT *' on a static compact table would query whole partitions, even though we'll only return
// the static part as far as CQL is concerned. This is thus mostly an optimization to use the query-by-name path).
int numberOfClusteringColumns = table.isStaticCompactTable() ? 0 : table.clusteringColumns().size();
int numberOfClusteringColumns = table.clusteringColumns().size();
// it is a range query if it has at least one the column alias for which no relation is defined or is not EQ or IN.
return clusteringColumnsRestrictions.size() < numberOfClusteringColumns
|| !clusteringColumnsRestrictions.hasOnlyEqualityRestrictions();

52
src/java/org/apache/cassandra/cql3/selection/Selectable.java
View File

@ -114,9 +114,9 @@ public interface Selectable extends AssignmentTestable
}
}
public static abstract class Raw
public interface Raw
{
public abstract Selectable prepare(TableMetadata table);
public Selectable prepare(TableMetadata table);
}
public static class WithTerm implements Selectable
@ -204,7 +204,7 @@ public interface Selectable extends AssignmentTestable
return rawTerm.getText();
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final Term.Raw term;
@ -265,17 +265,18 @@ public interface Selectable extends AssignmentTestable
return predicate.test(column);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final ColumnMetadata.Raw id;
private final Selectable.RawIdentifier id;
private final boolean isWritetime;
public Raw(ColumnMetadata.Raw id, boolean isWritetime)
public Raw(Selectable.RawIdentifier id, boolean isWritetime)
{
this.id = id;
this.isWritetime = isWritetime;
}
@Override
public WritetimeOrTTL prepare(TableMetadata table)
{
return new WritetimeOrTTL(id.prepare(table), isWritetime);
@ -317,7 +318,7 @@ public interface Selectable extends AssignmentTestable
return function.returnType();
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final FunctionName functionName;
private final List<Selectable.Raw> args;
@ -346,6 +347,7 @@ public interface Selectable extends AssignmentTestable
Collections.singletonList(arg));
}
@Override
public Selectable prepare(TableMetadata table)
{
List<Selectable> preparedArgs = new ArrayList<>(args.size());
@ -475,7 +477,7 @@ public interface Selectable extends AssignmentTestable
return arg.selectColumns(predicate);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final CQL3Type type;
private final Selectable.Raw arg;
@ -565,7 +567,7 @@ public interface Selectable extends AssignmentTestable
return selected.selectColumns(predicate);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final Selectable.Raw selected;
private final FieldIdentifier field;
@ -689,7 +691,7 @@ public interface Selectable extends AssignmentTestable
return Tuples.tupleToString(selectables);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final List<Selectable.Raw> raws;
@ -698,6 +700,7 @@ public interface Selectable extends AssignmentTestable
this.raws = raws;
}
@Override
public Selectable prepare(TableMetadata cfm)
{
return new BetweenParenthesesOrWithTuple(raws.stream().map(p -> p.prepare(cfm)).collect(Collectors.toList()));
@ -773,7 +776,7 @@ public interface Selectable extends AssignmentTestable
return Lists.listToString(selectables);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final List<Selectable.Raw> raws;
@ -782,6 +785,7 @@ public interface Selectable extends AssignmentTestable
this.raws = raws;
}
@Override
public Selectable prepare(TableMetadata cfm)
{
return new WithList(raws.stream().map(p -> p.prepare(cfm)).collect(Collectors.toList()));
@ -865,7 +869,7 @@ public interface Selectable extends AssignmentTestable
return Sets.setToString(selectables);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final List<Selectable.Raw> raws;
@ -874,6 +878,7 @@ public interface Selectable extends AssignmentTestable
this.raws = raws;
}
@Override
public Selectable prepare(TableMetadata cfm)
{
return new WithSet(raws.stream().map(p -> p.prepare(cfm)).collect(Collectors.toList()));
@ -884,7 +889,7 @@ public interface Selectable extends AssignmentTestable
/**
* {@code Selectable} for literal Maps or UDTs.
* <p>The parser cannot differentiate between a Map or a UDT in the selection cause because a
* {@code ColumnMetadata} is equivalent to a {@code FieldIdentifier} from a syntax point of view.
* {@code ColumnIdentifier} is equivalent to a {@code FieldIdentifier} from a syntax point of view.
* By consequence, we are forced to wait until the type is known to be able to differentiate them.</p>
*/
public static class WithMapOrUdt implements Selectable
@ -1044,7 +1049,7 @@ public interface Selectable extends AssignmentTestable
return fields;
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final List<Pair<Selectable.Raw, Selectable.Raw>> raws;
@ -1053,6 +1058,7 @@ public interface Selectable extends AssignmentTestable
this.raws = raws;
}
@Override
public Selectable prepare(TableMetadata cfm)
{
return new WithMapOrUdt(cfm, raws);
@ -1149,7 +1155,7 @@ public interface Selectable extends AssignmentTestable
return String.format("(%s)%s", typeName, selectable);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final CQL3Type.Raw typeRaw;
@ -1177,7 +1183,7 @@ public interface Selectable extends AssignmentTestable
* identifier have the same syntax. By consequence, we need to wait until the type is known to create the proper
* Object: {@code ColumnMetadata} or {@code FieldIdentifier}.
*/
public static final class RawIdentifier extends Selectable.Raw
public static final class RawIdentifier implements Selectable.Raw
{
private final String text;
@ -1186,7 +1192,7 @@ public interface Selectable extends AssignmentTestable
/**
* Creates a {@code RawIdentifier} from an unquoted identifier string.
*/
public static Raw forUnquoted(String text)
public static RawIdentifier forUnquoted(String text)
{
return new RawIdentifier(text, false);
}
@ -1194,7 +1200,7 @@ public interface Selectable extends AssignmentTestable
/**
* Creates a {@code RawIdentifier} from a quoted identifier string.
*/
public static Raw forQuoted(String text)
public static RawIdentifier forQuoted(String text)
{
return new RawIdentifier(text, true);
}
@ -1206,11 +1212,9 @@ public interface Selectable extends AssignmentTestable
}
@Override
public Selectable prepare(TableMetadata cfm)
public ColumnMetadata prepare(TableMetadata cfm)
{
ColumnMetadata.Raw raw = quoted ? ColumnMetadata.Raw.forQuoted(text)
: ColumnMetadata.Raw.forUnquoted(text);
return raw.prepare(cfm);
return cfm.getExistingColumn(ColumnIdentifier.getInterned(text, quoted));
}
public FieldIdentifier toFieldIdentifier()
@ -1281,7 +1285,7 @@ public interface Selectable extends AssignmentTestable
return selected.selectColumns(predicate);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final Selectable.Raw selected;
private final Term.Raw element;
@ -1367,7 +1371,7 @@ public interface Selectable extends AssignmentTestable
return selected.selectColumns(predicate);
}
public static class Raw extends Selectable.Raw
public static class Raw implements Selectable.Raw
{
private final Selectable.Raw selected;
// Both from and to can be null if they haven't been provided

2
src/java/org/apache/cassandra/cql3/selection/Selection.java
View File

@ -86,7 +86,7 @@ public abstract class Selection
*/
public boolean containsStaticColumns()
{
if (table.isStaticCompactTable() || !table.hasStaticColumns())
if (!table.hasStaticColumns())
return false;
if (isWildcard())

4
src/java/org/apache/cassandra/cql3/statements/DeleteStatement.java
View File

@ -130,7 +130,7 @@ public class DeleteStatement extends ModificationStatement
Attributes.Raw attrs,
List<Operation.RawDeletion> deletions,
WhereClause whereClause,
List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> conditions,
List<Pair<ColumnIdentifier, ColumnCondition.Raw>> conditions,
boolean ifExists)
{
super(name, StatementType.DELETE, attrs, conditions, false, ifExists);
@ -151,7 +151,7 @@ public class DeleteStatement extends ModificationStatement
for (Operation.RawDeletion deletion : deletions)
{
ColumnMetadata def = getColumnDefinition(metadata, deletion.affectedColumn());
ColumnMetadata def = metadata.getExistingColumn(deletion.affectedColumn());
// For compact, we only have one value except the key, so the only form of DELETE that make sense is without a column
// list. However, we support having the value name for coherence with the static/sparse case

39
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
View File

@ -134,15 +134,7 @@ public abstract class ModificationStatement implements CQLStatement
}
}
RegularAndStaticColumns modifiedColumns = updatedColumnsBuilder.build();
// Compact tables have not row marker. So if we don't actually update any particular column,
// this means that we're only updating the PK, which we allow if only those were declared in
// the definition. In that case however, we do went to write the compactValueColumn (since again
// we can't use a "row marker") so add it automatically.
if (metadata.isCompactTable() && modifiedColumns.isEmpty() && updatesRegularRows())
modifiedColumns = metadata.regularAndStaticColumns();
this.updatedColumns = modifiedColumns;
this.updatedColumns = updatedColumnsBuilder.build();
this.conditionColumns = conditionColumnsBuilder.build();
this.requiresRead = requiresReadBuilder.build();
}
@ -851,14 +843,14 @@ public abstract class ModificationStatement implements CQLStatement
{
protected final StatementType type;
private final Attributes.Raw attrs;
private final List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> conditions;
private final List<Pair<ColumnIdentifier, ColumnCondition.Raw>> conditions;
private final boolean ifNotExists;
private final boolean ifExists;
protected Parsed(QualifiedName name,
StatementType type,
Attributes.Raw attrs,
List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> conditions,
List<Pair<ColumnIdentifier, ColumnCondition.Raw>> conditions,
boolean ifNotExists,
boolean ifExists)
{
@ -931,9 +923,9 @@ public abstract class ModificationStatement implements CQLStatement
ColumnConditions.Builder builder = ColumnConditions.newBuilder();
for (Pair<ColumnMetadata.Raw, ColumnCondition.Raw> entry : conditions)
for (Pair<ColumnIdentifier, ColumnCondition.Raw> entry : conditions)
{
ColumnMetadata def = entry.left.prepare(metadata);
ColumnMetadata def = metadata.getExistingColumn(entry.left);
ColumnCondition condition = entry.right.prepare(keyspace(), def, metadata);
condition.collectMarkerSpecification(bindVariables);
@ -971,26 +963,9 @@ public abstract class ModificationStatement implements CQLStatement
return new StatementRestrictions(type, metadata, where, boundNames, applyOnlyToStaticColumns, false, false);
}
/**
* Retrieves the <code>ColumnMetadata</code> corresponding to the specified raw <code>ColumnIdentifier</code>.
*
* @param metadata the column family meta data
* @param rawId the raw <code>ColumnIdentifier</code>
* @return the <code>ColumnMetadata</code> corresponding to the specified raw <code>ColumnIdentifier</code>
*/
protected static ColumnMetadata getColumnDefinition(TableMetadata metadata, ColumnMetadata.Raw rawId)
public List<Pair<ColumnIdentifier, ColumnCondition.Raw>> getConditions()
{
return rawId.prepare(metadata);
}
public List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> getConditions()
{
ImmutableList.Builder<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> builder = ImmutableList.builderWithExpectedSize(conditions.size());
for (Pair<ColumnMetadata.Raw, ColumnCondition.Raw> condition : conditions)
builder.add(Pair.create(condition.left, condition.right));
return builder.build();
return conditions;
}
}
}

22
src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
View File

@ -811,9 +811,7 @@ public class SelectStatement implements CQLStatement
// The general rational is that if some rows are specifically selected by the query (have clustering or
// regular columns restrictions), we ignore partitions that are empty outside of static content, but if it's a full partition
// query, then we include that content.
// We make an exception for "static compact" table are from a CQL standpoint we always want to show their static
// content for backward compatiblity.
return queriesFullPartitions() || table.isStaticCompactTable();
return queriesFullPartitions();
}
// Used by ModificationStatement for CAS operations
@ -1043,7 +1041,7 @@ public class SelectStatement implements CQLStatement
*/
private boolean selectOnlyStaticColumns(TableMetadata table, List<Selectable> selectables)
{
if (table.isStaticCompactTable() || !table.hasStaticColumns() || selectables.isEmpty())
if (!table.hasStaticColumns() || selectables.isEmpty())
return false;
return Selectable.selectColumns(selectables, (column) -> column.isStatic())
@ -1060,9 +1058,9 @@ public class SelectStatement implements CQLStatement
return Collections.emptyMap();
Map<ColumnMetadata, Boolean> orderingColumns = new LinkedHashMap<>();
for (Map.Entry<ColumnMetadata.Raw, Boolean> entry : parameters.orderings.entrySet())
for (Map.Entry<ColumnIdentifier, Boolean> entry : parameters.orderings.entrySet())
{
orderingColumns.put(entry.getKey().prepare(table), entry.getValue());
orderingColumns.put(table.getExistingColumn(entry.getKey()), entry.getValue());
}
return orderingColumns;
}
@ -1154,9 +1152,9 @@ public class SelectStatement implements CQLStatement
int clusteringPrefixSize = 0;
Iterator<ColumnMetadata> pkColumns = metadata.primaryKeyColumns().iterator();
for (ColumnMetadata.Raw raw : parameters.groups)
for (ColumnIdentifier id : parameters.groups)
{
ColumnMetadata def = raw.prepare(metadata);
ColumnMetadata def = metadata.getExistingColumn(id);
checkTrue(def.isPartitionKey() || def.isClusteringColumn(),
"Group by is currently only supported on the columns of the PRIMARY KEY, got %s", def.name);
@ -1288,14 +1286,14 @@ public class SelectStatement implements CQLStatement
public static class Parameters
{
// Public because CASSANDRA-9858
public final Map<ColumnMetadata.Raw, Boolean> orderings;
public final List<ColumnMetadata.Raw> groups;
public final Map<ColumnIdentifier, Boolean> orderings;
public final List<ColumnIdentifier> groups;
public final boolean isDistinct;
public final boolean allowFiltering;
public final boolean isJson;
public Parameters(Map<ColumnMetadata.Raw, Boolean> orderings,
List<ColumnMetadata.Raw> groups,
public Parameters(Map<ColumnIdentifier, Boolean> orderings,
List<ColumnIdentifier> groups,
boolean isDistinct,
boolean allowFiltering,
boolean isJson)

41
src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
View File

@ -18,7 +18,6 @@
package org.apache.cassandra.cql3.statements;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import org.apache.cassandra.audit.AuditLogContext;
@ -28,7 +27,6 @@ import org.apache.cassandra.cql3.conditions.ColumnCondition;
import org.apache.cassandra.cql3.conditions.Conditions;
import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
import org.apache.cassandra.db.Clustering;
import org.apache.cassandra.db.CompactTables;
import org.apache.cassandra.db.Slice;
import org.apache.cassandra.db.partitions.PartitionUpdate;
import org.apache.cassandra.schema.ColumnMetadata;
@ -40,7 +38,6 @@ import org.apache.commons.lang3.builder.ToStringStyle;
import static org.apache.cassandra.cql3.statements.RequestValidations.checkContainsNoDuplicates;
import static org.apache.cassandra.cql3.statements.RequestValidations.checkFalse;
import static org.apache.cassandra.cql3.statements.RequestValidations.checkTrue;
/**
* An <code>UPDATE</code> statement parsed from a CQL query statement.
@ -48,8 +45,6 @@ import static org.apache.cassandra.cql3.statements.RequestValidations.checkTrue;
*/
public class UpdateStatement extends ModificationStatement
{
private static final Constants.Value EMPTY = new Constants.Value(ByteBufferUtil.EMPTY_BYTE_BUFFER);
private UpdateStatement(StatementType type,
VariableSpecifications bindVariables,
TableMetadata metadata,
@ -68,26 +63,12 @@ public class UpdateStatement extends ModificationStatement
{
params.newRow(clustering);
// We update the row timestamp (ex-row marker) only on INSERT (#6782)
// Further, COMPACT tables semantic differs from "CQL3" ones in that a row exists only if it has
// a non-null column, so we don't want to set the row timestamp for them.
if (type.isInsert() && metadata().isCQLTable())
// We update the row timestamp only on INSERT (#6782)
if (type.isInsert())
params.addPrimaryKeyLivenessInfo();
List<Operation> updates = getRegularOperations();
// For compact table, we don't accept an insert/update that only sets the PK unless the is no
// declared non-PK columns (which we recognize because in that case
// the compact value is of type "EmptyType").
if (metadata().isCompactTable() && updates.isEmpty())
{
checkTrue(CompactTables.hasEmptyCompactValue(metadata),
"Column %s is mandatory for this COMPACT STORAGE table",
metadata().compactValueColumn.name);
updates = Collections.<Operation>singletonList(new Constants.Setter(metadata().compactValueColumn, EMPTY));
}
for (Operation op : updates)
op.execute(updateBuilder.partitionKey(), params);
@ -111,7 +92,7 @@ public class UpdateStatement extends ModificationStatement
public static class ParsedInsert extends ModificationStatement.Parsed
{
private final List<ColumnMetadata.Raw> columnNames;
private final List<ColumnIdentifier> columnNames;
private final List<Term.Raw> columnValues;
/**
@ -125,7 +106,7 @@ public class UpdateStatement extends ModificationStatement
*/
public ParsedInsert(QualifiedName name,
Attributes.Raw attrs,
List<ColumnMetadata.Raw> columnNames,
List<ColumnIdentifier> columnNames,
List<Term.Raw> columnValues,
boolean ifNotExists)
{
@ -155,7 +136,7 @@ public class UpdateStatement extends ModificationStatement
for (int i = 0; i < columnNames.size(); i++)
{
ColumnMetadata def = getColumnDefinition(metadata, columnNames.get(i));
ColumnMetadata def = metadata.getExistingColumn(columnNames.get(i));
if (def.isClusteringColumn())
hasClusteringColumnsSet = true;
@ -232,7 +213,7 @@ public class UpdateStatement extends ModificationStatement
Term.Raw raw = prepared.getRawTermForColumn(def, defaultUnset);
if (def.isPrimaryKeyColumn())
{
whereClause.add(new SingleColumnRelation(ColumnMetadata.Raw.forColumn(def), Operator.EQ, raw));
whereClause.add(new SingleColumnRelation(def.name, Operator.EQ, raw));
}
else
{
@ -265,7 +246,7 @@ public class UpdateStatement extends ModificationStatement
public static class ParsedUpdate extends ModificationStatement.Parsed
{
// Provided for an UPDATE
private final List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> updates;
private final List<Pair<ColumnIdentifier, Operation.RawUpdate>> updates;
private final WhereClause whereClause;
/**
@ -280,9 +261,9 @@ public class UpdateStatement extends ModificationStatement
* */
public ParsedUpdate(QualifiedName name,
Attributes.Raw attrs,
List<Pair<ColumnMetadata.Raw, Operation.RawUpdate>> updates,
List<Pair<ColumnIdentifier, Operation.RawUpdate>> updates,
WhereClause whereClause,
List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> conditions,
List<Pair<ColumnIdentifier, ColumnCondition.Raw>> conditions,
boolean ifExists)
{
super(name, StatementType.UPDATE, attrs, conditions, false, ifExists);
@ -298,9 +279,9 @@ public class UpdateStatement extends ModificationStatement
{
Operations operations = new Operations(type);
for (Pair<ColumnMetadata.Raw, Operation.RawUpdate> entry : updates)
for (Pair<ColumnIdentifier, Operation.RawUpdate> entry : updates)
{
ColumnMetadata def = getColumnDefinition(metadata, entry.left);
ColumnMetadata def = metadata.getExistingColumn(entry.left);
checkFalse(def.isPrimaryKeyColumn(), "PRIMARY KEY part %s found in SET part", def.name);

95
src/java/org/apache/cassandra/cql3/statements/schema/AlterTableStatement.java
View File

@ -17,22 +17,39 @@
*/
package org.apache.cassandra.cql3.statements.schema;
import java.util.*;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.apache.cassandra.audit.AuditLogContext;
import org.apache.cassandra.audit.AuditLogEntryType;
import org.apache.cassandra.auth.Permission;
import org.apache.cassandra.cql3.*;
import org.apache.cassandra.cql3.CQL3Type;
import org.apache.cassandra.cql3.CQLStatement;
import org.apache.cassandra.cql3.ColumnIdentifier;
import org.apache.cassandra.cql3.QualifiedName;
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.schema.*;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.schema.IndexMetadata;
import org.apache.cassandra.schema.KeyspaceMetadata;
import org.apache.cassandra.schema.Keyspaces;
import org.apache.cassandra.schema.Keyspaces.KeyspacesDiff;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.schema.TableParams;
import org.apache.cassandra.schema.ViewMetadata;
import org.apache.cassandra.schema.Views;
import org.apache.cassandra.service.ClientState;
import org.apache.cassandra.service.reads.repair.ReadRepairStrategy;
import org.apache.cassandra.transport.Event.SchemaChange;
import org.apache.cassandra.transport.Event.SchemaChange.Change;
import org.apache.cassandra.transport.Event.SchemaChange.Target;
import org.apache.cassandra.utils.FBUtilities;
import static java.lang.String.join;
@ -115,11 +132,11 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
{
private static class Column
{
private final ColumnMetadata.Raw name;
private final ColumnIdentifier name;
private final CQL3Type.Raw type;
private final boolean isStatic;
Column(ColumnMetadata.Raw name, CQL3Type.Raw type, boolean isStatic)
Column(ColumnIdentifier name, CQL3Type.Raw type, boolean isStatic)
{
this.name = name;
this.type = type;
@ -151,7 +168,7 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
TableMetadata.Builder tableBuilder,
Views.Builder viewsBuilder)
{
ColumnIdentifier name = column.name.getIdentifier(table);
ColumnIdentifier name = column.name;
AbstractType<?> type = column.type.prepare(keyspaceName, keyspace.types).getType();
boolean isStatic = column.isStatic;
@ -213,10 +230,10 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
// TODO: swap UDT refs with expanded tuples on drop
private static class DropColumns extends AlterTableStatement
{
private final Collection<ColumnMetadata.Raw> removedColumns;
private final Set<ColumnIdentifier> removedColumns;
private final Long timestamp;
private DropColumns(String keyspaceName, String tableName, Collection<ColumnMetadata.Raw> removedColumns, Long timestamp)
private DropColumns(String keyspaceName, String tableName, Set<ColumnIdentifier> removedColumns, Long timestamp)
{
super(keyspaceName, tableName);
this.removedColumns = removedColumns;
@ -230,16 +247,14 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
return keyspace.withSwapped(keyspace.tables.withSwapped(builder.build()));
}
private void dropColumn(KeyspaceMetadata keyspace, TableMetadata table, ColumnMetadata.Raw column, TableMetadata.Builder builder)
private void dropColumn(KeyspaceMetadata keyspace, TableMetadata table, ColumnIdentifier column, TableMetadata.Builder builder)
{
ColumnIdentifier name = column.getIdentifier(table);
ColumnMetadata currentColumn = table.getColumn(name);
ColumnMetadata currentColumn = table.getColumn(column);
if (null == currentColumn)
throw ire("Column %s was not found in table '%s'", name, table);
throw ire("Column %s was not found in table '%s'", column, table);
if (currentColumn.isPrimaryKeyColumn())
throw ire("Cannot drop PRIMARY KEY column %s", name);
throw ire("Cannot drop PRIMARY KEY column %s", column);
/*
* Cannot allow dropping top-level columns of user defined types that aren't frozen because we cannot convert
@ -247,7 +262,7 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
* the correct type in system_schema.dropped_columns.
*/
if (currentColumn.type.isUDT() && currentColumn.type.isMultiCell())
throw ire("Cannot drop non-frozen column %s of user type %s", name, currentColumn.type.asCQL3Type());
throw ire("Cannot drop non-frozen column %s of user type %s", column, currentColumn.type.asCQL3Type());
// TODO: some day try and find a way to not rely on Keyspace/IndexManager/Index to find dependent indexes
Set<IndexMetadata> dependentIndexes = Keyspace.openAndGetStore(table).indexManager.getDependentIndexes(currentColumn);
@ -261,7 +276,7 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
if (!isEmpty(keyspace.views.forTable(table.id)))
throw ire("Cannot drop column %s on base table %s with materialized views", currentColumn, table.name);
builder.removeRegularOrStaticColumn(name);
builder.removeRegularOrStaticColumn(column);
builder.recordColumnDrop(currentColumn, getTimestamp());
}
@ -279,9 +294,9 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
*/
private static class RenameColumns extends AlterTableStatement
{
private final Map<ColumnMetadata.Raw, ColumnMetadata.Raw> renamedColumns;
private final Map<ColumnIdentifier, ColumnIdentifier> renamedColumns;
private RenameColumns(String keyspaceName, String tableName, Map<ColumnMetadata.Raw, ColumnMetadata.Raw> renamedColumns)
private RenameColumns(String keyspaceName, String tableName, Map<ColumnIdentifier, ColumnIdentifier> renamedColumns)
{
super(keyspaceName, tableName);
this.renamedColumns = renamedColumns;
@ -299,26 +314,23 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
private void renameColumn(KeyspaceMetadata keyspace,
TableMetadata table,
ColumnMetadata.Raw oldName,
ColumnMetadata.Raw newName,
ColumnIdentifier oldName,
ColumnIdentifier newName,
TableMetadata.Builder tableBuilder,
Views.Builder viewsBuilder)
{
ColumnIdentifier oldColumnName = oldName.getIdentifier(table);
ColumnIdentifier newColumnName = newName.getIdentifier(table);
ColumnMetadata column = table.getColumn(oldColumnName);
ColumnMetadata column = table.getColumn(oldName);
if (null == column)
throw ire("Column %s was not found in table %s", oldColumnName, table);
throw ire("Column %s was not found in table %s", oldName, table);
if (!column.isPrimaryKeyColumn())
throw ire("Cannot rename non PRIMARY KEY column %s", oldColumnName);
throw ire("Cannot rename non PRIMARY KEY column %s", oldName);
if (null != table.getColumn(newColumnName))
if (null != table.getColumn(newName))
{
throw ire("Cannot rename column %s to %s in table '%s'; another column with that name already exists",
oldColumnName,
newColumnName,
oldName,
newName,
table);
}
@ -327,22 +339,19 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
if (!dependentIndexes.isEmpty())
{
throw ire("Can't rename column %s because it has dependent secondary indexes (%s)",
oldColumnName,
oldName,
join(", ", transform(dependentIndexes, i -> i.name)));
}
for (ViewMetadata view : keyspace.views.forTable(table.id))
{
if (view.includes(oldColumnName))
if (view.includes(oldName))
{
ColumnIdentifier oldViewColumn = oldName.getIdentifier(view.metadata);
ColumnIdentifier newViewColumn = newName.getIdentifier(view.metadata);
viewsBuilder.put(viewsBuilder.get(view.name()).withRenamedPrimaryKeyColumn(oldViewColumn, newViewColumn));
viewsBuilder.put(viewsBuilder.get(view.name()).withRenamedPrimaryKeyColumn(oldName, newName));
}
}
tableBuilder.renamePrimaryKeyColumn(oldColumnName, newColumnName);
tableBuilder.renamePrimaryKeyColumn(oldName, newName);
}
}
@ -402,11 +411,11 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
private final List<AddColumns.Column> addedColumns = new ArrayList<>();
// DROP
private final List<ColumnMetadata.Raw> droppedColumns = new ArrayList<>();
private final Set<ColumnIdentifier> droppedColumns = new HashSet<>();
private Long timestamp = null; // will use execution timestamp if not provided by query
// RENAME
private final Map<ColumnMetadata.Raw, ColumnMetadata.Raw> renamedColumns = new HashMap<>();
private final Map<ColumnIdentifier, ColumnIdentifier> renamedColumns = new HashMap<>();
// OPTIONS
public final TableAttributes attrs = new TableAttributes();
@ -433,18 +442,18 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
throw new AssertionError();
}
public void alter(ColumnMetadata.Raw name, CQL3Type.Raw type)
public void alter(ColumnIdentifier name, CQL3Type.Raw type)
{
kind = Kind.ALTER_COLUMN;
}
public void add(ColumnMetadata.Raw name, CQL3Type.Raw type, boolean isStatic)
public void add(ColumnIdentifier name, CQL3Type.Raw type, boolean isStatic)
{
kind = Kind.ADD_COLUMNS;
addedColumns.add(new AddColumns.Column(name, type, isStatic));
}
public void drop(ColumnMetadata.Raw name)
public void drop(ColumnIdentifier name)
{
kind = Kind.DROP_COLUMNS;
droppedColumns.add(name);
@ -455,7 +464,7 @@ public abstract class AlterTableStatement extends AlterSchemaStatement
this.timestamp = timestamp;
}
public void rename(ColumnMetadata.Raw from, ColumnMetadata.Raw to)
public void rename(ColumnIdentifier from, ColumnIdentifier to)
{
kind = Kind.RENAME_COLUMNS;
renamedColumns.put(from, to);

16
src/java/org/apache/cassandra/cql3/statements/schema/IndexTarget.java
View File

@ -44,36 +44,36 @@ public class IndexTarget
public static class Raw
{
private final ColumnMetadata.Raw column;
private final ColumnIdentifier column;
private final Type type;
private Raw(ColumnMetadata.Raw column, Type type)
private Raw(ColumnIdentifier column, Type type)
{
this.column = column;
this.type = type;
}
public static Raw simpleIndexOn(ColumnMetadata.Raw c)
public static Raw simpleIndexOn(ColumnIdentifier c)
{
return new Raw(c, Type.SIMPLE);
}
public static Raw valuesOf(ColumnMetadata.Raw c)
public static Raw valuesOf(ColumnIdentifier c)
{
return new Raw(c, Type.VALUES);
}
public static Raw keysOf(ColumnMetadata.Raw c)
public static Raw keysOf(ColumnIdentifier c)
{
return new Raw(c, Type.KEYS);
}
public static Raw keysAndValuesOf(ColumnMetadata.Raw c)
public static Raw keysAndValuesOf(ColumnIdentifier c)
{
return new Raw(c, Type.KEYS_AND_VALUES);
}
public static Raw fullCollection(ColumnMetadata.Raw c)
public static Raw fullCollection(ColumnIdentifier c)
{
return new Raw(c, Type.FULL);
}
@ -85,7 +85,7 @@ public class IndexTarget
// same syntax as an index on a regular column (i.e. the 'values' in
// 'CREATE INDEX on table(values(collection));' is optional). So we correct the target type
// when the target column is a collection & the target type is SIMPLE.
ColumnMetadata columnDef = column.prepare(table);
ColumnMetadata columnDef = table.getExistingColumn(column);
Type actualType = (type == Type.SIMPLE && columnDef.type.isCollection()) ? Type.VALUES : type;
return new IndexTarget(columnDef.name, actualType);
}

7
src/java/org/apache/cassandra/db/AbstractReadCommandBuilder.java
View File

@ -191,13 +191,6 @@ public abstract class AbstractReadCommandBuilder
protected ClusteringIndexFilter makeFilter()
{
// StatementRestrictions.isColumnRange() returns false for static compact tables, which means
// SelectStatement.makeClusteringIndexFilter uses a names filter with no clusterings for static
// compact tables, here we reproduce this behavior (CASSANDRA-11223). Note that this code is only
// called by tests.
if (cfs.metadata().isStaticCompactTable())
return new ClusteringIndexNamesFilter(new TreeSet<>(cfs.metadata().comparator), reversed);
if (clusterings != null)
{
return new ClusteringIndexNamesFilter(clusterings, reversed);

4
src/java/org/apache/cassandra/db/ColumnFamilyStore.java
View File

@ -423,7 +423,9 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean
// create the private ColumnFamilyStores for the secondary column indexes
indexManager = new SecondaryIndexManager(this);
for (IndexMetadata info : metadata.get().indexes)
{
indexManager.addIndex(info, true);
}
if (registerBookeeping)
{
@ -990,7 +992,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean
* In doing so it also tells the write operations to update the commitLogUpperBound of the memtable, so
* that we know the CL position we are dirty to, which can be marked clean when we complete.
*/
writeBarrier = keyspace.writeOrder.newBarrier();
writeBarrier = Keyspace.writeOrder.newBarrier();
// submit flushes for the memtable for any indexed sub-cfses, and our own
AtomicReference<CommitLogPosition> commitLogUpperBound = new AtomicReference<>();

146
src/java/org/apache/cassandra/db/CompactTables.java
View File

@ -1,146 +0,0 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.cassandra.db;
import java.nio.ByteBuffer;
import java.util.*;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.utils.ByteBufferUtil;
/**
* Small utility methods pertaining to the encoding of COMPACT STORAGE tables.
*
* COMPACT STORAGE tables exists mainly for the sake of encoding internally thrift tables (as well as
* exposing those tables through CQL). Note that due to these constraints, the internal representation
* of compact tables does *not* correspond exactly to their CQL definition.
*
* The internal layout of such tables is such that it can encode any thrift table. That layout is as follow:
* CREATE TABLE compact (
* key [key_validation_class],
* [column_metadata_1] [type1] static,
* ...,
* [column_metadata_n] [type1] static,
* column [comparator],
* value [default_validation_class]
* PRIMARY KEY (key, column)
* )
* More specifically, the table:
* - always has a clustering column and a regular value, which are used to store the "dynamic" thrift columns name and value.
* Those are always present because we have no way to know in advance if "dynamic" columns will be inserted or not. Note
* that when declared from CQL, compact tables may not have any clustering: in that case, we still have a clustering
* defined internally, it is just ignored as far as interacting from CQL is concerned.
* - have a static column for every "static" column defined in the thrift "column_metadata". Note that when declaring a compact
* table from CQL without any clustering (but some non-PK columns), the columns ends up static internally even though they are
* not in the declaration
*
* On variation is that if the table comparator is a CompositeType, then the underlying table will have one clustering column by
* element of the CompositeType, but the rest of the layout is as above.
*
* As far as thrift is concerned, one exception to this is super column families, which have a different layout. Namely, a super
* column families is encoded with:
* {@code
* CREATE TABLE super (
* key [key_validation_class],
* super_column_name [comparator],
* [column_metadata_1] [type1],
* ...,
* [column_metadata_n] [type1],
* "" map<[sub_comparator], [default_validation_class]>
* PRIMARY KEY (key, super_column_name)
* )
* }
* In other words, every super column is encoded by a row. That row has one column for each defined "column_metadata", but it also
* has a special map column (whose name is the empty string as this is guaranteed to never conflict with a user-defined
* "column_metadata") which stores the super column "dynamic" sub-columns.
*/
public abstract class CompactTables
{
// We use an empty value for the 1) this can't conflict with a user-defined column and 2) this actually
// validate with any comparator.
public static final ByteBuffer SUPER_COLUMN_MAP_COLUMN = ByteBufferUtil.EMPTY_BYTE_BUFFER;
private CompactTables() {}
public static ColumnMetadata getCompactValueColumn(RegularAndStaticColumns columns, boolean isSuper)
{
if (isSuper)
{
for (ColumnMetadata column : columns.regulars)
if (column.name.bytes.equals(SUPER_COLUMN_MAP_COLUMN))
return column;
throw new AssertionError("Invalid super column table definition, no 'dynamic' map column");
}
assert columns.regulars.simpleColumnCount() == 1 && columns.regulars.complexColumnCount() == 0;
return columns.regulars.getSimple(0);
}
public static boolean hasEmptyCompactValue(TableMetadata metadata)
{
return metadata.compactValueColumn.type instanceof EmptyType;
}
public static boolean isSuperColumnMapColumn(ColumnMetadata column)
{
return column.kind == ColumnMetadata.Kind.REGULAR && column.name.bytes.equals(SUPER_COLUMN_MAP_COLUMN);
}
public static DefaultNames defaultNameGenerator(Set<String> usedNames)
{
return new DefaultNames(new HashSet<>(usedNames));
}
public static class DefaultNames
{
private static final String DEFAULT_CLUSTERING_NAME = "column";
private static final String DEFAULT_COMPACT_VALUE_NAME = "value";
private final Set<String> usedNames;
private int clusteringIndex = 1;
private int compactIndex = 0;
private DefaultNames(Set<String> usedNames)
{
this.usedNames = usedNames;
}
public String defaultClusteringName()
{
while (true)
{
String candidate = DEFAULT_CLUSTERING_NAME + clusteringIndex;
++clusteringIndex;
if (usedNames.add(candidate))
return candidate;
}
}
public String defaultCompactValueName()
{
while (true)
{
String candidate = compactIndex == 0 ? DEFAULT_COMPACT_VALUE_NAME : DEFAULT_COMPACT_VALUE_NAME + compactIndex;
++compactIndex;
if (usedNames.add(candidate))
return candidate;
}
}
}
}

3
src/java/org/apache/cassandra/db/PartitionRangeReadQuery.java
View File

@ -87,7 +87,6 @@ public interface PartitionRangeReadQuery extends ReadQuery
default boolean selectsFullPartition()
{
return metadata().isStaticCompactTable() ||
(dataRange().selectsAllPartition() && !rowFilter().hasExpressionOnClusteringOrRegularColumns());
return dataRange().selectsAllPartition() && !rowFilter().hasExpressionOnClusteringOrRegularColumns();
}
}

6
src/java/org/apache/cassandra/db/SimpleBuilders.java
View File

@ -341,8 +341,8 @@ public abstract class SimpleBuilders
if (initiated)
return;
// If a CQL table, add the "row marker"
if (metadata.isCQLTable() && !noPrimaryKeyLivenessInfo)
// Adds the row liveness
if (!noPrimaryKeyLivenessInfo)
builder.addPrimaryKeyLivenessInfo(LivenessInfo.create(timestamp, ttl, nowInSec));
initiated = true;
@ -364,7 +364,7 @@ public abstract class SimpleBuilders
ColumnMetadata column = getColumn(columnName);
if (!overwriteForCollection && !(column.type.isMultiCell() && column.type.isCollection()))
throw new IllegalArgumentException("appendAll() can only be called on non-frozen colletions");
throw new IllegalArgumentException("appendAll() can only be called on non-frozen collections");
columns.add(column);

3
src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
View File

@ -994,8 +994,7 @@ public class SinglePartitionReadCommand extends ReadCommand implements SinglePar
@Override
public boolean selectsFullPartition()
{
return metadata().isStaticCompactTable() ||
(clusteringIndexFilter.selectsAllPartition() && !rowFilter().hasExpressionOnClusteringOrRegularColumns());
return clusteringIndexFilter.selectsAllPartition() && !rowFilter().hasExpressionOnClusteringOrRegularColumns();
}
@Override

2
src/java/org/apache/cassandra/db/filter/RowFilter.java
View File

@ -534,7 +534,7 @@ public abstract class RowFilter implements Iterable<RowFilter.Expression>
Operator operator = Operator.readFrom(in);
ColumnMetadata column = metadata.getColumn(name);
if (!metadata.isCompactTable() && column == null)
if (column == null)
throw new RuntimeException("Unknown (or dropped) column " + UTF8Type.instance.getString(name) + " during deserialization");
switch (kind)

30
src/java/org/apache/cassandra/db/rows/BTreeRow.java
View File

@ -18,7 +18,16 @@
package org.apache.cassandra.db.rows;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.AbstractCollection;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.Comparator;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.function.Predicate;
@ -28,14 +37,25 @@ import com.google.common.collect.Collections2;
import com.google.common.collect.Iterators;
import com.google.common.primitives.Ints;
import org.apache.cassandra.db.Clustering;
import org.apache.cassandra.db.Columns;
import org.apache.cassandra.db.DeletionPurger;
import org.apache.cassandra.db.DeletionTime;
import org.apache.cassandra.db.LivenessInfo;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.db.*;
import org.apache.cassandra.db.filter.ColumnFilter;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.partitions.PartitionUpdate;
import org.apache.cassandra.schema.DroppedColumn;
import org.apache.cassandra.utils.*;
import org.apache.cassandra.utils.AbstractIterator;
import org.apache.cassandra.utils.BiLongAccumulator;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.LongAccumulator;
import org.apache.cassandra.utils.ObjectSizes;
import org.apache.cassandra.utils.btree.BTree;
import org.apache.cassandra.utils.btree.BTreeSearchIterator;
import org.apache.cassandra.utils.btree.UpdateFunction;
@ -548,7 +568,7 @@ public class BTreeRow extends AbstractRow
private CellInLegacyOrderIterator(TableMetadata metadata, boolean reversed)
{
AbstractType<?> nameComparator = metadata.columnDefinitionNameComparator(isStatic() ? ColumnMetadata.Kind.STATIC : ColumnMetadata.Kind.REGULAR);
AbstractType<?> nameComparator = UTF8Type.instance;
this.comparator = reversed ? Collections.reverseOrder(nameComparator) : nameComparator;
this.reversed = reversed;

2
src/java/org/apache/cassandra/db/rows/UnfilteredRowIteratorWithLowerBound.java
View File

@ -232,7 +232,7 @@ public class UnfilteredRowIteratorWithLowerBound extends LazilyInitializedUnfilt
// Side-note: pre-2.1 sstable stat file had clustering value arrays whose size may not match the comparator size
// and that would break getMetadataLowerBound. We don't support upgrade from 2.0 to 3.0 directly however so it's
// not a true concern. Besides, !sstable.mayHaveTombstones already ensure this is a 3.0 sstable anyway.
return !sstable.mayHaveTombstones() && !sstable.metadata().isCompactTable();
return !sstable.mayHaveTombstones();
}
/**

3
src/java/org/apache/cassandra/db/view/View.java
View File

@ -26,6 +26,7 @@ import com.google.common.collect.Iterables;
import org.apache.cassandra.cql3.*;
import org.apache.cassandra.cql3.selection.RawSelector;
import org.apache.cassandra.cql3.selection.Selectable;
import org.apache.cassandra.cql3.statements.SelectStatement;
import org.apache.cassandra.db.*;
import org.apache.cassandra.db.rows.*;
@ -188,7 +189,7 @@ public class View
.columns()
.stream()
.map(c -> c.name.toString())
.map(ColumnMetadata.Raw::forQuoted)
.map(Selectable.RawIdentifier::forQuoted)
.map(c -> new RawSelector(c, null))
.collect(Collectors.toList());
}

32
src/java/org/apache/cassandra/db/view/ViewUpdateGenerator.java
View File

@ -166,21 +166,9 @@ public class ViewUpdateGenerator
assert !mergedBaseRow.isEmpty();
// Note that none of the base PK columns will differ since we're intrinsically dealing
// with the same base row. So we have to check 3 things:
// 1) that the clustering doesn't have a null, which can happen for compact tables. If that's the case,
// there is no corresponding entries.
// 2) if there is a column not part of the base PK in the view PK, whether it is changed by the update.
// 3) whether mergedBaseRow actually match the view SELECT filter
if (baseMetadata.isCompactTable())
{
Clustering clustering = mergedBaseRow.clustering();
for (int i = 0; i < clustering.size(); i++)
{
if (clustering.get(i) == null)
return UpdateAction.NONE;
}
}
// with the same base row. So we have to check 2 things:
// 1) if there is a column not part of the base PK in the view PK, whether it is changed by the update.
// 2) whether mergedBaseRow actually match the view SELECT filter
assert view.baseNonPKColumnsInViewPK.size() <= 1 : "We currently only support one base non-PK column in the view PK";
@ -404,13 +392,13 @@ public class ViewUpdateGenerator
// If computed deletion timestamp is from row deletion, we only need row deletion itself
if (timestamp > rowDeletion)
{
/**
* We use an expired liveness instead of a row tombstone to allow a shadowed MV
* entry to co-exist with a row tombstone, see ViewComplexTest#testCommutativeRowDeletion.
*
* TODO This is a dirty overload of LivenessInfo and we should modify
* the storage engine to properly support this on CASSANDRA-13826.
*/
/*
* We use an expired liveness instead of a row tombstone to allow a shadowed MV
* entry to co-exist with a row tombstone, see ViewComplexTest#testCommutativeRowDeletion.
*
* TODO This is a dirty overload of LivenessInfo and we should modify
* the storage engine to properly support this on CASSANDRA-13826.
*/
LivenessInfo info = LivenessInfo.withExpirationTime(timestamp, LivenessInfo.EXPIRED_LIVENESS_TTL, nowInSec);
currentViewEntryBuilder.addPrimaryKeyLivenessInfo(info);
}

21
src/java/org/apache/cassandra/index/internal/CassandraIndex.java
View File

@ -45,7 +45,6 @@ import org.apache.cassandra.db.lifecycle.SSTableSet;
import org.apache.cassandra.db.lifecycle.View;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.CollectionType;
import org.apache.cassandra.db.marshal.EmptyType;
import org.apache.cassandra.db.partitions.PartitionIterator;
import org.apache.cassandra.db.partitions.PartitionUpdate;
import org.apache.cassandra.db.rows.*;
@ -303,6 +302,7 @@ public abstract class CassandraIndex implements Index
return new CompositesSearcher(command, target.get(), this);
case KEYS:
return new KeysSearcher(command, target.get(), this);
default:
throw new IllegalStateException(String.format("Unsupported index type %s for index %s on %s",
metadata.kind,
@ -741,27 +741,12 @@ public abstract class CassandraIndex implements Index
TableMetadata.Builder builder =
TableMetadata.builder(baseCfsMetadata.keyspace, baseCfsMetadata.indexTableName(indexMetadata), baseCfsMetadata.id)
.kind(TableMetadata.Kind.INDEX)
// tables for legacy KEYS indexes are non-compound and dense
.isDense(indexMetadata.isKeys())
.isCompound(!indexMetadata.isKeys())
.partitioner(new LocalPartitioner(indexedValueType))
.addPartitionKeyColumn(indexedColumn.name, indexedColumn.type)
.addClusteringColumn("partition_key", baseCfsMetadata.partitioner.partitionOrdering());
if (indexMetadata.isKeys())
{
// A dense, compact table for KEYS indexes must have a compact
// value column defined, even though it is never used
CompactTables.DefaultNames names =
CompactTables.defaultNameGenerator(ImmutableSet.of(indexedColumn.name.toString(), "partition_key"));
builder.addRegularColumn(names.defaultCompactValueName(), EmptyType.instance);
}
else
{
// The clustering columns for a table backing a COMPOSITES index are dependent
// on the specific type of index (there are specializations for indexes on collections)
utils.addIndexClusteringColumns(builder, baseCfsMetadata, indexedColumn);
}
// Adding clustering columns, which depends on the index type.
builder = utils.addIndexClusteringColumns(builder, baseCfsMetadata, indexedColumn);
return builder.build().updateIndexTableMetadata(baseCfsMetadata.params);
}

30
src/java/org/apache/cassandra/index/internal/CassandraIndexFunctions.java
View File

@ -83,14 +83,26 @@ public interface CassandraIndexFunctions
static final CassandraIndexFunctions KEYS_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{
@Override
public CassandraIndex newIndexInstance(ColumnFamilyStore baseCfs, IndexMetadata indexMetadata)
{
return new KeysIndex(baseCfs, indexMetadata);
}
@Override
public TableMetadata.Builder addIndexClusteringColumns(TableMetadata.Builder builder,
TableMetadata baseMetadata,
ColumnMetadata columnDef)
{
// KEYS index are indexing the whole partition so have no clustering columns (outside of the
// "partition_key" one that all the 'CassandraIndex' gets (see CassandraIndex#indexCfsMetadata)).
return builder;
}
};
static final CassandraIndexFunctions REGULAR_COLUMN_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{
@Override
public CassandraIndex newIndexInstance(ColumnFamilyStore baseCfs, IndexMetadata indexMetadata)
{
return new RegularColumnIndex(baseCfs, indexMetadata);
@ -99,11 +111,13 @@ public interface CassandraIndexFunctions
static final CassandraIndexFunctions CLUSTERING_COLUMN_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{
@Override
public CassandraIndex newIndexInstance(ColumnFamilyStore baseCfs, IndexMetadata indexMetadata)
{
return new ClusteringColumnIndex(baseCfs, indexMetadata);
}
@Override
public TableMetadata.Builder addIndexClusteringColumns(TableMetadata.Builder builder,
TableMetadata baseMetadata,
ColumnMetadata columnDef)
@ -124,14 +138,6 @@ public interface CassandraIndexFunctions
}
};
static final CassandraIndexFunctions PARTITION_KEY_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{
public CassandraIndex newIndexInstance(ColumnFamilyStore baseCfs, IndexMetadata indexMetadata)
{
return new PartitionKeyIndex(baseCfs, indexMetadata);
}
};
static final CassandraIndexFunctions COLLECTION_KEY_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{
public CassandraIndex newIndexInstance(ColumnFamilyStore baseCfs, IndexMetadata indexMetadata)
@ -145,6 +151,14 @@ public interface CassandraIndexFunctions
}
};
static final CassandraIndexFunctions PARTITION_KEY_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{
public CassandraIndex newIndexInstance(ColumnFamilyStore baseCfs, IndexMetadata indexMetadata)
{
return new PartitionKeyIndex(baseCfs, indexMetadata);
}
};
static final CassandraIndexFunctions COLLECTION_VALUE_INDEX_FUNCTIONS = new CassandraIndexFunctions()
{

10
src/java/org/apache/cassandra/index/internal/keys/KeysIndex.java
View File

@ -58,8 +58,8 @@ public class KeysIndex extends CassandraIndex
}
protected ByteBuffer getIndexedValue(ByteBuffer partitionKey,
Clustering clustering,
CellPath path, ByteBuffer cellValue)
Clustering clustering,
CellPath path, ByteBuffer cellValue)
{
return cellValue;
}
@ -77,7 +77,7 @@ public class KeysIndex extends CassandraIndex
Cell cell = row.getCell(indexedColumn);
return (cell == null
|| !cell.isLive(nowInSec)
|| indexedColumn.type.compare(indexValue, cell.value()) != 0);
|| !cell.isLive(nowInSec)
|| indexedColumn.type.compare(indexValue, cell.value()) != 0);
}
}
}

8
src/java/org/apache/cassandra/index/internal/keys/KeysSearcher.java
View File

@ -149,9 +149,9 @@ public class KeysSearcher extends CassandraIndexSearcher
{
// Index is stale, remove the index entry and ignore
index.deleteStaleEntry(index.getIndexCfs().decorateKey(indexedValue),
makeIndexClustering(iterator.partitionKey().getKey(), Clustering.EMPTY),
new DeletionTime(indexHit.primaryKeyLivenessInfo().timestamp(), nowInSec),
ctx);
makeIndexClustering(iterator.partitionKey().getKey(), Clustering.EMPTY),
new DeletionTime(indexHit.primaryKeyLivenessInfo().timestamp(), nowInSec),
ctx);
iterator.close();
return null;
}
@ -160,4 +160,4 @@ public class KeysSearcher extends CassandraIndexSearcher
return iterator;
}
}
}
}

176
src/java/org/apache/cassandra/schema/ColumnMetadata.java
View File

@ -149,7 +149,7 @@ public final class ColumnMetadata extends ColumnSpecification implements Selecta
{
this(table.keyspace,
table.name,
ColumnIdentifier.getInterned(name, table.columnDefinitionNameComparator(kind)),
ColumnIdentifier.getInterned(name, UTF8Type.instance),
type,
position,
kind);
@ -426,13 +426,13 @@ public final class ColumnMetadata extends ColumnSpecification implements Selecta
((UserType)type).nameComparator().validate(path.get(0));
}
public void appendCqlTo(CqlBuilder builder, boolean ignoreStatic)
public void appendCqlTo(CqlBuilder builder)
{
builder.append(name)
.append(' ')
.append(type);
if (isStatic() && !ignoreStatic)
if (isStatic())
builder.append(" static");
}
@ -496,174 +496,4 @@ public final class ColumnMetadata extends ColumnSpecification implements Selecta
{
return type;
}
/**
* Because legacy-created tables may have a non-text comparator, we cannot determine the proper 'key' until
* we know the comparator. ColumnMetadata.Raw is a placeholder that can be converted to a real ColumnIdentifier
* once the comparator is known with prepare(). This should only be used with identifiers that are actual
* column names. See CASSANDRA-8178 for more background.
*/
public static abstract class Raw extends Selectable.Raw
{
/**
* Creates a {@code ColumnMetadata.Raw} from an unquoted identifier string.
*/
public static Raw forUnquoted(String text)
{
return new Literal(text, false);
}
/**
* Creates a {@code ColumnMetadata.Raw} from a quoted identifier string.
*/
public static Raw forQuoted(String text)
{
return new Literal(text, true);
}
/**
* Creates a {@code ColumnMetadata.Raw} from a pre-existing {@code ColumnMetadata}
* (useful in the rare cases where we already have the column but need
* a {@code ColumnMetadata.Raw} for typing purposes).
*/
public static Raw forColumn(ColumnMetadata column)
{
return new ForColumn(column);
}
/**
* Get the identifier corresponding to this raw column, without assuming this is an
* existing column (unlike {@link Selectable.Raw#prepare}).
*/
public abstract ColumnIdentifier getIdentifier(TableMetadata table);
public abstract String rawText();
@Override
public abstract ColumnMetadata prepare(TableMetadata table);
@Override
public final int hashCode()
{
return toString().hashCode();
}
@Override
public final boolean equals(Object o)
{
if(!(o instanceof Raw))
return false;
Raw that = (Raw)o;
return this.toString().equals(that.toString());
}
private static class Literal extends Raw
{
private final String text;
public Literal(String rawText, boolean keepCase)
{
this.text = keepCase ? rawText : rawText.toLowerCase(Locale.US);
}
public ColumnIdentifier getIdentifier(TableMetadata table)
{
if (!table.isStaticCompactTable())
return ColumnIdentifier.getInterned(text, true);
AbstractType<?> columnNameType = table.staticCompactOrSuperTableColumnNameType();
if (columnNameType instanceof UTF8Type)
return ColumnIdentifier.getInterned(text, true);
// We have a legacy-created table with a non-text comparator. Check if we have a matching column, otherwise assume we should use
// columnNameType
ByteBuffer bufferName = ByteBufferUtil.bytes(text);
for (ColumnMetadata def : table.columns())
{
if (def.name.bytes.equals(bufferName))
return def.name;
}
return ColumnIdentifier.getInterned(columnNameType, columnNameType.fromString(text), text);
}
public ColumnMetadata prepare(TableMetadata table)
{
if (!table.isStaticCompactTable())
return find(table);
AbstractType<?> columnNameType = table.staticCompactOrSuperTableColumnNameType();
if (columnNameType instanceof UTF8Type)
return find(table);
// We have a legacy-created table with a non-text comparator. Check if we have a match column, otherwise assume we should use
// columnNameType
ByteBuffer bufferName = ByteBufferUtil.bytes(text);
for (ColumnMetadata def : table.columns())
{
if (def.name.bytes.equals(bufferName))
return def;
}
return find(columnNameType.fromString(text), table);
}
private ColumnMetadata find(TableMetadata table)
{
return find(ByteBufferUtil.bytes(text), table);
}
private ColumnMetadata find(ByteBuffer id, TableMetadata table)
{
ColumnMetadata def = table.getColumn(id);
if (def == null)
throw new InvalidRequestException(String.format("Undefined column name %s", toString()));
return def;
}
public String rawText()
{
return text;
}
@Override
public String toString()
{
return ColumnIdentifier.maybeQuote(text);
}
}
// Use internally in the rare case where we need a ColumnMetadata.Raw for type-checking but
// actually already have the column itself.
private static class ForColumn extends Raw
{
private final ColumnMetadata column;
private ForColumn(ColumnMetadata column)
{
this.column = column;
}
public ColumnIdentifier getIdentifier(TableMetadata table)
{
return column.name;
}
public ColumnMetadata prepare(TableMetadata table)
{
assert table.getColumn(column.name) != null; // Sanity check that we're not doing something crazy
return column;
}
public String rawText()
{
return column.name.toString();
}
@Override
public String toString()
{
return column.name.toCQLString();
}
}
}
}

6
src/java/org/apache/cassandra/schema/SchemaEvent.java
View File

@ -197,14 +197,8 @@ public final class SchemaEvent extends DiagnosticEvent
ret.put("triggers", Lists.newArrayList(repr(table.triggers)));
ret.put("columns", Lists.newArrayList(table.columns.values().stream().map(this::repr).iterator()));
ret.put("droppedColumns", Lists.newArrayList(table.droppedColumns.values().stream().map(this::repr).iterator()));
ret.put("isCompactTable", table.isCompactTable());
ret.put("isCompound", table.isCompound());
ret.put("isCounter", table.isCounter());
ret.put("isCQLTable", table.isCQLTable());
ret.put("isDense", table.isDense());
ret.put("isIndex", table.isIndex());
ret.put("isStaticCompactTable", table.isStaticCompactTable());
ret.put("isSuper", table.isSuper());
ret.put("isView", table.isView());
ret.put("isVirtual", table.isVirtual());
return ret;

6
src/java/org/apache/cassandra/schema/SchemaKeyspace.java
View File

@ -954,12 +954,6 @@ public final class SchemaKeyspace
UntypedResultSet.Row row = rows.one();
Set<TableMetadata.Flag> flags = TableMetadata.Flag.fromStringSet(row.getFrozenSet("flags", UTF8Type.instance));
if (!TableMetadata.Flag.isCQLCompatible(flags))
{
throw new IllegalArgumentException(TableMetadata.COMPACT_STORAGE_HALT_MESSAGE);
}
return TableMetadata.builder(keyspaceName, tableName, TableId.fromUUID(row.getUUID("id")))
.flags(flags)
.params(createTableParamsFromRow(row))

265
src/java/org/apache/cassandra/schema/TableMetadata.java
View File

@ -26,9 +26,6 @@ import javax.annotation.Nullable;
import com.google.common.base.MoreObjects;
import com.google.common.collect.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.auth.DataResource;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.cql3.ColumnIdentifier;
@ -38,6 +35,7 @@ import org.apache.cassandra.db.*;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.service.reads.SpeculativeRetryPolicy;
import org.apache.cassandra.utils.AbstractIterator;
import org.github.jamm.Unmetered;
@ -52,27 +50,35 @@ import static org.apache.cassandra.schema.IndexMetadata.isNameValid;
@Unmetered
public final class TableMetadata implements SchemaElement
{
private static final Logger logger = LoggerFactory.getLogger(TableMetadata.class);
private static final ImmutableSet<Flag> DEFAULT_CQL_FLAGS = ImmutableSet.of(Flag.COMPOUND);
private static final ImmutableSet<Flag> DEPRECATED_CS_FLAGS = ImmutableSet.of(Flag.DENSE, Flag.SUPER);
public static final String COMPACT_STORAGE_HALT_MESSAGE =
"Compact Tables are not allowed in Cassandra starting with 4.0 version. " +
"Use `ALTER ... DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra " +
"in order to migrate off Compact Storage.";
private static final String COMPACT_STORAGE_DEPRECATION_MESSAGE =
"Incorrect set of flags is was detected in table {}.{}: '{}'. \n" +
"Starting with version 4.0, '{}' flags are deprecated and every table has to have COMPOUND flag. \n" +
"Forcing the following set of flags: '{}'";
"Detected table %s.%s with COMPACT STORAGE flags (%s). " +
"Compact Tables are not supported in Cassandra starting with version 4.0. " +
"Use the `ALTER ... DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra " +
"in order to migrate off Compact Storage before upgrading.";
// Please note that currently the only one truly useful flag is COUNTER, as the rest of the flags were about
// differencing between CQL tables and the various types of COMPACT STORAGE tables (pre-4.0). As those "compact"
// tables are not supported anymore, no tables should be either SUPER or DENSE, and they should all be COMPOUND.
public enum Flag
{
SUPER, COUNTER, DENSE, COMPOUND;
// As mentioned above, all tables on 4.0+ will have the COMPOUND flag, making the flag of little value. However,
// on upgrade from pre-4.0, we want to detect if a tables does _not_ have this flag, in which case this would
// be a compact table on which DROP COMPACT STORAGE has _not_ be used and fail startup. This is also why we
// still write this flag for all tables. Once we drop support for upgrading from pre-4.0 versions (and so are
// sure all tables do have the flag), we can stop writing this flag and ignore it when present (deprecate it).
// Later, we'll be able to drop the flag from this enum completely.
COMPOUND,
COUNTER,
// The only reason we still have those is that on the first startup after an upgrade from pre-4.0, we cannot
// guarantee some tables won't have those flags (users having forgotten to use DROP COMPACT STORAGE before
// upgrading). So we still "deserialize" those flags correctly, but otherwise prevent startup if any table
// have them. Once we drop support for upgrading from pre-4.0, we can remove those values.
@Deprecated SUPER,
@Deprecated DENSE;
public static boolean isCQLCompatible(Set<Flag> flags)
static boolean isLegacyCompactTable(Set<Flag> flags)
{
return !flags.contains(Flag.DENSE) && !flags.contains(Flag.SUPER) && flags.contains(Flag.COMPOUND);
return flags.contains(Flag.DENSE) || flags.contains(Flag.SUPER) || !flags.contains(Flag.COMPOUND);
}
public static Set<Flag> fromStringSet(Set<String> strings)
@ -123,27 +129,18 @@ public final class TableMetadata implements SchemaElement
public final AbstractType<?> partitionKeyType;
public final ClusteringComparator comparator;
/*
* For dense tables, this alias the single non-PK column the table contains (since it can only have one). We keep
* that as convenience to access that column more easily (but we could replace calls by regularAndStaticColumns().iterator().next()
* for those tables in practice).
*/
public final ColumnMetadata compactValueColumn;
// performance hacks; TODO see if all are really necessary
public final DataResource resource;
private TableMetadata(Builder builder)
{
if (!Flag.isCQLCompatible(builder.flags))
{
flags = ImmutableSet.copyOf(Sets.union(Sets.difference(builder.flags, DEPRECATED_CS_FLAGS), DEFAULT_CQL_FLAGS));
logger.warn(COMPACT_STORAGE_DEPRECATION_MESSAGE, builder.keyspace, builder.name, builder.flags, DEPRECATED_CS_FLAGS, flags);
}
else
{
flags = Sets.immutableEnumSet(builder.flags);
}
if (Flag.isLegacyCompactTable(builder.flags))
throw new IllegalStateException(format(COMPACT_STORAGE_HALT_MESSAGE,
builder.keyspace,
builder.name,
builder.flags));
flags = Sets.immutableEnumSet(builder.flags);
keyspace = builder.keyspace;
name = builder.name;
id = builder.id;
@ -171,10 +168,6 @@ public final class TableMetadata implements SchemaElement
comparator = new ClusteringComparator(transform(clusteringColumns, c -> c.type));
compactValueColumn = isCompactTable()
? CompactTables.getCompactValueColumn(regularAndStaticColumns, isSuper())
: null;
resource = DataResource.table(keyspace, name);
}
@ -236,46 +229,11 @@ public final class TableMetadata implements SchemaElement
return Optional.ofNullable(indexName);
}
/*
* We call dense a CF for which each component of the comparator is a clustering column, i.e. no
* component is used to store a regular column names. In other words, non-composite static "thrift"
* and CQL3 CF are *not* dense.
*/
public boolean isDense()
{
return flags.contains(Flag.DENSE);
}
public boolean isCompound()
{
return flags.contains(Flag.COMPOUND);
}
public boolean isSuper()
{
return flags.contains(Flag.SUPER);
}
public boolean isCounter()
{
return flags.contains(Flag.COUNTER);
}
public boolean isCQLTable()
{
return !isSuper() && !isDense() && isCompound();
}
public boolean isCompactTable()
{
return !isCQLTable();
}
public boolean isStaticCompactTable()
{
return !isSuper() && !isDense() && !isCompound();
}
public ImmutableCollection<ColumnMetadata> columns()
{
return columns.values();
@ -296,11 +254,6 @@ public final class TableMetadata implements SchemaElement
return clusteringColumns;
}
public ImmutableList<ColumnMetadata> createStatementClusteringColumns()
{
return isStaticCompactTable() ? ImmutableList.of() : clusteringColumns;
}
public RegularAndStaticColumns regularAndStaticColumns()
{
return regularAndStaticColumns;
@ -318,22 +271,12 @@ public final class TableMetadata implements SchemaElement
/*
* An iterator over all column definitions but that respect the order of a SELECT *.
* This also "hide" the clustering/regular columns for a non-CQL3 non-dense table for backward compatibility
* sake.
*/
public Iterator<ColumnMetadata> allColumnsInSelectOrder()
{
boolean isStaticCompactTable = isStaticCompactTable();
boolean noNonPkColumns = isCompactTable() && CompactTables.hasEmptyCompactValue(this);
Iterator<ColumnMetadata> partitionKeyIter = partitionKeyColumns.iterator();
Iterator<ColumnMetadata> clusteringIter =
isStaticCompactTable ? Collections.emptyIterator() : clusteringColumns.iterator();
Iterator<ColumnMetadata> otherColumns =
noNonPkColumns
? Collections.emptyIterator()
: (isStaticCompactTable ? staticColumns().selectOrderIterator()
: regularAndStaticColumns.selectOrderIterator());
Iterator<ColumnMetadata> clusteringIter = clusteringColumns.iterator();
Iterator<ColumnMetadata> otherColumns = regularAndStaticColumns.selectOrderIterator();
return columnsIterator(partitionKeyIter, clusteringIter, otherColumns);
}
@ -343,16 +286,9 @@ public final class TableMetadata implements SchemaElement
*/
public Iterator<ColumnMetadata> allColumnsInCreateOrder()
{
boolean isStaticCompactTable = isStaticCompactTable();
boolean noNonPkColumns = isCompactTable() && CompactTables.hasEmptyCompactValue(this);
Iterator<ColumnMetadata> partitionKeyIter = partitionKeyColumns.iterator();
Iterator<ColumnMetadata> clusteringIter = createStatementClusteringColumns().iterator();
Iterator<ColumnMetadata> otherColumns =
noNonPkColumns
? Collections.emptyIterator()
: (isStaticCompactTable ? staticColumns().iterator()
: regularAndStaticColumns.iterator());
Iterator<ColumnMetadata> clusteringIter = clusteringColumns.iterator();
Iterator<ColumnMetadata> otherColumns = regularAndStaticColumns.iterator();
return columnsIterator(partitionKeyIter, clusteringIter, otherColumns);
}
@ -383,7 +319,25 @@ public final class TableMetadata implements SchemaElement
{
return columns.get(name.bytes);
}
/**
* Returns the column of the provided name if it exists, but throws a user-visible exception if that column doesn't
* exist.
*
* <p>This method is for finding columns from a name provided by the user, and as such it does _not_ returne hidden
* columns (throwing that the column is unknown instead).
*
* @param name the name of an existing non-hidden column of this table.
* @return the column metadata corresponding to {@code name}.
*
* @throws InvalidRequestException if there is no non-hidden column named {@code name} in this table.
*/
public ColumnMetadata getExistingColumn(ColumnIdentifier name)
{
ColumnMetadata def = getColumn(name);
if (def == null)
throw new InvalidRequestException(format("Undefined column name %s in table %s", name.toCQLString(), this));
return def;
}
/*
* In general it is preferable to work with ColumnIdentifier to make it
* clear that we are talking about a CQL column, not a cell name, but there
@ -442,7 +396,7 @@ public final class TableMetadata implements SchemaElement
if (isCounter())
{
for (ColumnMetadata column : regularAndStaticColumns)
if (!(column.type.isCounter()) && !CompactTables.isSuperColumnMapColumn(column))
if (!(column.type.isCounter()) && !isSuperColumnMapColumnName(column.name))
except("Cannot have a non counter column (\"%s\") in a counter table", column.name);
}
else
@ -456,16 +410,30 @@ public final class TableMetadata implements SchemaElement
if (partitionKeyColumns.isEmpty())
except("Missing partition keys for table %s", toString());
// A compact table should always have a clustering
if (isCompactTable() && clusteringColumns.isEmpty())
except("For table %s, isDense=%b, isCompound=%b, clustering=%s", toString(), isDense(), isCompound(), clusteringColumns);
if (!indexes.isEmpty() && isSuper())
except("Secondary indexes are not supported on super column families");
indexes.validate(this);
}
/**
* To support backward compatibility with thrift super columns in the C* 3.0+ storage engine, we encode said super
* columns as a CQL {@code map<blob, blob>}. To ensure the name of this map did not conflict with any other user
* defined columns, we used the empty name (which is otherwise not allowed for user created columns).
* <p>
* While all thrift-based tables must have been converted to "CQL" ones with "DROP COMPACT STORAGE" (before
* upgrading to C* 4.0, which stop supporting non-CQL tables completely), a converted super-column table will still
* have this map with an empty name. And the reason we need to recognize it still, is that for backward
* compatibility we need to support counters in values of this map while it's not supported in any other map.
*
* TODO: it's probably worth lifting the limitation of not allowing counters as map values. It works fully
* internally (since we had to support it for this special map) and doesn't feel particularly dangerous to
* support. Doing so would remove this special case, but would also let user that do have an upgraded super-column
* table with counters to rename that weirdly name map to something more meaningful (it's not possible today
* as after renaming the validation in {@link #validate)} would trigger).
*/
private static boolean isSuperColumnMapColumnName(ColumnIdentifier columnName)
{
return !columnName.bytes.hasRemaining();
}
void validateCompatibility(TableMetadata previous)
{
if (isIndex())
@ -530,35 +498,6 @@ public final class TableMetadata implements SchemaElement
return new ClusteringComparator(partitionKeyColumns.stream().map(c -> c.type).collect(toList()));
}
/**
* The type to use to compare column names in "static compact"
* tables or superColum ones.
* <p>
* This exists because for historical reasons, "static compact" tables as
* well as super column ones can have non-UTF8 column names.
* <p>
* This method should only be called for superColumn tables and "static
* compact" ones. For any other table, all column names are UTF8.
*/
AbstractType<?> staticCompactOrSuperTableColumnNameType()
{
if (isSuper())
{
assert compactValueColumn != null && compactValueColumn.type instanceof MapType;
return ((MapType) compactValueColumn.type).nameComparator();
}
assert isStaticCompactTable();
return clusteringColumns.get(0).type;
}
public AbstractType<?> columnDefinitionNameComparator(ColumnMetadata.Kind kind)
{
return (isSuper() && kind == ColumnMetadata.Kind.REGULAR) || (isStaticCompactTable() && kind == ColumnMetadata.Kind.STATIC)
? staticCompactOrSuperTableColumnNameType()
: UTF8Type.instance;
}
/**
* Generate a table name for an index corresponding to the given column.
* This is NOT the same as the index's name! This is only used in sstable filenames and is not exposed to users.
@ -704,7 +643,7 @@ public final class TableMetadata implements SchemaElement
@Override
public String toString()
{
return String.format("%s.%s", ColumnIdentifier.maybeQuote(keyspace), ColumnIdentifier.maybeQuote(name));
return format("%s.%s", ColumnIdentifier.maybeQuote(keyspace), ColumnIdentifier.maybeQuote(name));
}
public String toDebugString()
@ -735,7 +674,7 @@ public final class TableMetadata implements SchemaElement
private Kind kind = Kind.REGULAR;
private TableParams.Builder params = TableParams.builder();
// Setting compound as default as "normal" CQL tables are compound and that's what we want by default
// See the comment on Flag.COMPOUND definition for why we (still) inconditionally add this flag.
private Set<Flag> flags = EnumSet.of(Flag.COMPOUND);
private Triggers triggers = Triggers.none();
private Indexes indexes = Indexes.none();
@ -884,26 +823,11 @@ public final class TableMetadata implements SchemaElement
return this;
}
public Builder isSuper(boolean val)
{
return flag(Flag.SUPER, val);
}
public Builder isCounter(boolean val)
{
return flag(Flag.COUNTER, val);
}
public Builder isDense(boolean val)
{
return flag(Flag.DENSE, val);
}
public Builder isCompound(boolean val)
{
return flag(Flag.COMPOUND, val);
}
private Builder flag(Flag flag, boolean set)
{
if (set) flags.add(flag); else flags.remove(flag);
@ -1195,21 +1119,7 @@ public final class TableMetadata implements SchemaElement
assert !isView();
String createKeyword = "CREATE";
if (!isCQLTable())
{
builder.append("/*")
.newLine()
.append("Warning: Table ")
.append(toString())
.append(" omitted because it has constructs not compatible with CQL (was created via legacy API).")
.newLine()
.append("Approximate structure, for reference:")
.newLine()
.append("(this should not be used to reproduce this schema)")
.newLine()
.newLine();
}
else if (isVirtual())
if (isVirtual())
{
builder.append(String.format("/*\n" +
"Warning: Table %s is a virtual table and cannot be recreated with CQL.\n" +
@ -1243,7 +1153,7 @@ public final class TableMetadata implements SchemaElement
builder.decreaseIndent();
if (!isCQLTable() || isVirtual())
if (isVirtual())
{
builder.newLine()
.append("*/");
@ -1268,7 +1178,7 @@ public final class TableMetadata implements SchemaElement
if (includeDroppedColumns && droppedColumns.containsKey(column.name.bytes))
continue;
column.appendCqlTo(builder, isStaticCompactTable());
column.appendCqlTo(builder);
if (hasSingleColumnPrimaryKey && column.isPartitionKey())
builder.append(" PRIMARY KEY");
@ -1285,7 +1195,7 @@ public final class TableMetadata implements SchemaElement
while (iterDropped.hasNext())
{
DroppedColumn dropped = iterDropped.next();
dropped.column.appendCqlTo(builder, isStaticCompactTable());
dropped.column.appendCqlTo(builder);
if (!hasSingleColumnPrimaryKey || iter.hasNext())
builder.append(',');
@ -1298,7 +1208,7 @@ public final class TableMetadata implements SchemaElement
void appendPrimaryKey(CqlBuilder builder)
{
List<ColumnMetadata> partitionKeyColumns = partitionKeyColumns();
List<ColumnMetadata> clusteringColumns = createStatementClusteringColumns();
List<ColumnMetadata> clusteringColumns = clusteringColumns();
builder.append("PRIMARY KEY (");
if (partitionKeyColumns.size() > 1)
@ -1331,12 +1241,7 @@ public final class TableMetadata implements SchemaElement
.newLine()
.append("AND ");
if (isCompactTable())
builder.append("COMPACT STORAGE")
.newLine()
.append("AND ");
List<ColumnMetadata> clusteringColumns = createStatementClusteringColumns();
List<ColumnMetadata> clusteringColumns = clusteringColumns();
if (!clusteringColumns.isEmpty())
{
builder.append("CLUSTERING ORDER BY (")
@ -1380,7 +1285,7 @@ public final class TableMetadata implements SchemaElement
.append(toString())
.append(" ADD ");
column.appendCqlTo(builder, false);
column.appendCqlTo(builder);
builder.append(';');
}

6
src/java/org/apache/cassandra/schema/ViewMetadata.java
View File

@ -142,14 +142,10 @@ public final class ViewMetadata implements SchemaElement
public ViewMetadata withRenamedPrimaryKeyColumn(ColumnIdentifier from, ColumnIdentifier to)
{
// convert whereClause to Relations, rename ids in Relations, then convert back to whereClause
ColumnMetadata.Raw rawFrom = ColumnMetadata.Raw.forQuoted(from.toString());
ColumnMetadata.Raw rawTo = ColumnMetadata.Raw.forQuoted(to.toString());
return new ViewMetadata(baseTableId,
baseTableName,
includeAllColumns,
whereClause.renameIdentifier(rawFrom, rawTo),
whereClause.renameIdentifier(from, to),
metadata.unbuild().renamePrimaryKeyColumn(from, to).build());
}

45
src/java/org/apache/cassandra/service/pager/PagingState.java
View File

@ -25,7 +25,6 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.primitives.Ints;
import org.apache.cassandra.db.Clustering;
import org.apache.cassandra.db.CompactTables;
import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.BytesType;
import org.apache.cassandra.db.marshal.CompositeType;
@ -359,8 +358,7 @@ public class PagingState
if (!cells.hasNext())
{
// If the last returned row has no cell, this means in 2.1/2.2 terms that we stopped on the row
// marker. Note that this shouldn't happen if the table is COMPACT.
assert !metadata.isCompactTable();
// marker.
mark = encodeCellName(metadata, row.clustering(), EMPTY_BYTE_BUFFER, null);
}
else
@ -388,25 +386,14 @@ public class PagingState
: Clustering.serializer.deserialize(mark, MessagingService.VERSION_30, makeClusteringTypes(metadata));
}
// Old (pre-3.0) encoding of cells. We need that for the protocol v3 as that is how things where encoded
// Old (pre-3.0) encoding of cells. We need that for the protocol v3 as that is how things were encoded
private static ByteBuffer encodeCellName(TableMetadata metadata, Clustering clustering, ByteBuffer columnName, ByteBuffer collectionElement)
{
boolean isStatic = clustering == Clustering.STATIC_CLUSTERING;
if (!metadata.isCompound())
{
if (isStatic)
return columnName;
assert clustering.size() == 1 : "Expected clustering size to be 1, but was " + clustering.size();
return clustering.get(0);
}
// We use comparator.size() rather than clustering.size() because of static clusterings
int clusteringSize = metadata.comparator.size();
int size = clusteringSize + (metadata.isDense() ? 0 : 1) + (collectionElement == null ? 0 : 1);
if (metadata.isSuper())
size = clusteringSize + 1;
int size = clusteringSize + 1 + (collectionElement == null ? 0 : 1);
ByteBuffer[] values = new ByteBuffer[size];
for (int i = 0; i < clusteringSize; i++)
{
@ -425,23 +412,9 @@ public class PagingState
values[i] = v;
}
if (metadata.isSuper())
{
// We need to set the "column" (in thrift terms) name, i.e. the value corresponding to the subcomparator.
// What it is depends if this a cell for a declared "static" column or a "dynamic" column part of the
// super-column internal map.
assert columnName != null; // This should never be null for supercolumns, see decodeForSuperColumn() above
values[clusteringSize] = columnName.equals(CompactTables.SUPER_COLUMN_MAP_COLUMN)
? collectionElement
: columnName;
}
else
{
if (!metadata.isDense())
values[clusteringSize] = columnName;
if (collectionElement != null)
values[clusteringSize + 1] = collectionElement;
}
values[clusteringSize] = columnName;
if (collectionElement != null)
values[clusteringSize + 1] = collectionElement;
return CompositeType.build(isStatic, values);
}
@ -452,12 +425,10 @@ public class PagingState
if (csize == 0)
return Clustering.EMPTY;
if (metadata.isCompound() && CompositeType.isStaticName(value))
if (CompositeType.isStaticName(value))
return Clustering.STATIC_CLUSTERING;
List<ByteBuffer> components = metadata.isCompound()
? CompositeType.splitName(value)
: Collections.singletonList(value);
List<ByteBuffer> components = CompositeType.splitName(value);
return Clustering.make(components.subList(0, Math.min(csize, components.size())).toArray(new ByteBuffer[csize]));
}

86
test/unit/org/apache/cassandra/SchemaLoader.java
View File

@ -135,15 +135,8 @@ public class SchemaLoader
standardCFMD(ks1, "StandardGCGS0").gcGraceSeconds(0).build(),
standardCFMD(ks1, "StandardLong1").build(),
standardCFMD(ks1, "StandardLong2").build(),
superCFMD(ks1, "Super1", LongType.instance).build(),
superCFMD(ks1, "Super2", LongType.instance).build(),
superCFMD(ks1, "Super3", LongType.instance).build(),
superCFMD(ks1, "Super4", UTF8Type.instance).build(),
superCFMD(ks1, "Super5", bytes).build(),
superCFMD(ks1, "Super6", LexicalUUIDType.instance, UTF8Type.instance).build(),
keysIndexCFMD(ks1, "Indexed1", true).build(),
keysIndexCFMD(ks1, "Indexed2", false).build(),
superCFMD(ks1, "SuperDirectGC", BytesType.instance).gcGraceSeconds(0).build(),
jdbcCFMD(ks1, "JdbcUtf8", UTF8Type.instance).addColumn(utf8Column(ks1, "JdbcUtf8")).build(),
jdbcCFMD(ks1, "JdbcLong", LongType.instance).build(),
jdbcCFMD(ks1, "JdbcBytes", bytes).build(),
@ -162,8 +155,6 @@ public class SchemaLoader
// Column Families
standardCFMD(ks2, "Standard1").build(),
standardCFMD(ks2, "Standard3").build(),
superCFMD(ks2, "Super3", bytes).build(),
superCFMD(ks2, "Super4", TimeUUIDType.instance).build(),
keysIndexCFMD(ks2, "Indexed1", true).build(),
compositeIndexCFMD(ks2, "Indexed2", true).build(),
compositeIndexCFMD(ks2, "Indexed3", true).gcGraceSeconds(0).build())));
@ -180,10 +171,7 @@ public class SchemaLoader
KeyspaceParams.simple(3),
Tables.of(
standardCFMD(ks4, "Standard1").build(),
standardCFMD(ks4, "Standard3").build(),
superCFMD(ks4, "Super3", bytes).build(),
superCFMD(ks4, "Super4", TimeUUIDType.instance).build(),
superCFMD(ks4, "Super5", TimeUUIDType.instance, BytesType.instance).build())));
standardCFMD(ks4, "Standard3").build())));
// Keyspace 5
schema.add(KeyspaceMetadata.create(ks5,
@ -192,8 +180,8 @@ public class SchemaLoader
// Keyspace 6
schema.add(KeyspaceMetadata.create(ks6,
KeyspaceParams.simple(1),
Tables.of(keysIndexCFMD(ks6, "Indexed1", true).build())));
KeyspaceParams.simple(1),
Tables.of(keysIndexCFMD(ks6, "Indexed1", true).build())));
// Keyspace 7
schema.add(KeyspaceMetadata.create(ks7,
@ -399,44 +387,6 @@ public class SchemaLoader
.addRegularColumn("val2", AsciiType.instance);
}
public static TableMetadata.Builder denseCFMD(String ksName, String cfName)
{
return denseCFMD(ksName, cfName, AsciiType.instance);
}
public static TableMetadata.Builder denseCFMD(String ksName, String cfName, AbstractType cc)
{
return denseCFMD(ksName, cfName, cc, null);
}
public static TableMetadata.Builder denseCFMD(String ksName, String cfName, AbstractType cc, AbstractType subcc)
{
AbstractType comp = cc;
if (subcc != null)
comp = CompositeType.getInstance(Arrays.asList(new AbstractType<?>[]{cc, subcc}));
return TableMetadata.builder(ksName, cfName)
.isDense(true)
.isCompound(subcc != null)
.addPartitionKeyColumn("key", AsciiType.instance)
.addClusteringColumn("cols", comp)
.addRegularColumn("val", AsciiType.instance)
.compression(getCompressionParameters());
}
// TODO: Fix superCFMD failing on legacy table creation. Seems to be applying composite comparator to partition key
public static TableMetadata.Builder superCFMD(String ksName, String cfName, AbstractType subcc)
{
return superCFMD(ksName, cfName, BytesType.instance, subcc);
}
public static TableMetadata.Builder superCFMD(String ksName, String cfName, AbstractType cc, AbstractType subcc)
{
return superCFMD(ksName, cfName, "cols", cc, subcc);
}
public static TableMetadata.Builder superCFMD(String ksName, String cfName, String ccName, AbstractType cc, AbstractType subcc)
{
return standardCFMD(ksName, cfName);
}
public static TableMetadata.Builder compositeIndexCFMD(String ksName, String cfName, boolean withRegularIndex) throws ConfigurationException
{
return compositeIndexCFMD(ksName, cfName, withRegularIndex, false);
@ -514,25 +464,23 @@ public class SchemaLoader
public static TableMetadata.Builder keysIndexCFMD(String ksName, String cfName, boolean withIndex)
{
TableMetadata.Builder builder =
TableMetadata.builder(ksName, cfName)
.isCompound(false)
.isDense(true)
.addPartitionKeyColumn("key", AsciiType.instance)
.addClusteringColumn("c1", AsciiType.instance)
.addStaticColumn("birthdate", LongType.instance)
.addStaticColumn("notbirthdate", LongType.instance)
.addRegularColumn("value", LongType.instance)
.compression(getCompressionParameters());
TableMetadata.builder(ksName, cfName)
.addPartitionKeyColumn("key", AsciiType.instance)
.addClusteringColumn("c1", AsciiType.instance)
.addStaticColumn("birthdate", LongType.instance)
.addStaticColumn("notbirthdate", LongType.instance)
.addRegularColumn("value", LongType.instance)
.compression(getCompressionParameters());
if (withIndex)
{
IndexMetadata index =
IndexMetadata.fromIndexTargets(
Collections.singletonList(new IndexTarget(new ColumnIdentifier("birthdate", true),
IndexTarget.Type.VALUES)),
cfName + "_birthdate_composite_index",
IndexMetadata.Kind.KEYS,
Collections.EMPTY_MAP);
IndexMetadata.fromIndexTargets(
Collections.singletonList(new IndexTarget(new ColumnIdentifier("birthdate", true),
IndexTarget.Type.VALUES)),
cfName + "_birthdate_composite_index",
IndexMetadata.Kind.KEYS,
Collections.EMPTY_MAP);
builder.indexes(Indexes.builder().add(index).build());
}
@ -543,8 +491,6 @@ public class SchemaLoader
{
TableMetadata.Builder builder =
TableMetadata.builder(ksName, cfName)
.isCompound(false)
.isDense(true)
.addPartitionKeyColumn("key", AsciiType.instance)
.addClusteringColumn("c1", AsciiType.instance)
.addRegularColumn("value", LongType.instance)

7
test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
View File

@ -63,8 +63,6 @@ public class ColumnFamilyStoreTest
public static final String KEYSPACE2 = "ColumnFamilyStoreTest2";
public static final String CF_STANDARD1 = "Standard1";
public static final String CF_STANDARD2 = "Standard2";
public static final String CF_SUPER1 = "Super1";
public static final String CF_SUPER6 = "Super6";
public static final String CF_INDEX1 = "Indexed1";
@BeforeClass
@ -76,9 +74,6 @@ public class ColumnFamilyStoreTest
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD1),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD2),
SchemaLoader.keysIndexCFMD(KEYSPACE1, CF_INDEX1, true));
// TODO: Fix superCFMD failing on legacy table creation. Seems to be applying composite comparator to partition key
// SchemaLoader.superCFMD(KEYSPACE1, CF_SUPER1, LongType.instance));
// SchemaLoader.superCFMD(KEYSPACE1, CF_SUPER6, "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", LexicalUUIDType.instance, UTF8Type.instance),
SchemaLoader.createKeyspace(KEYSPACE2,
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD(KEYSPACE2, CF_STANDARD1));
@ -89,8 +84,6 @@ public class ColumnFamilyStoreTest
{
Keyspace.open(KEYSPACE1).getColumnFamilyStore(CF_STANDARD1).truncateBlocking();
Keyspace.open(KEYSPACE1).getColumnFamilyStore(CF_STANDARD2).truncateBlocking();
// Keyspace.open(KEYSPACE1).getColumnFamilyStore(CF_SUPER1).truncateBlocking();
Keyspace.open(KEYSPACE2).getColumnFamilyStore(CF_STANDARD1).truncateBlocking();
}

7
test/unit/org/apache/cassandra/db/PartitionRangeReadTest.java
View File

@ -69,9 +69,12 @@ public class PartitionRangeReadTest
SchemaLoader.createKeyspace(KEYSPACE1,
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD1),
SchemaLoader.denseCFMD(KEYSPACE1, CF_STANDARDINT, IntegerType.instance),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARDINT,
0,
AsciiType.instance,
AsciiType.instance,
IntegerType.instance),
TableMetadata.builder(KEYSPACE1, CF_COMPACT1)
.isCompound(false)
.addPartitionKeyColumn("key", AsciiType.instance)
.addClusteringColumn("column1", AsciiType.instance)
.addRegularColumn("value", AsciiType.instance)

4
test/unit/org/apache/cassandra/db/PartitionTest.java
View File

@ -53,7 +53,6 @@ public class PartitionTest
private static final String KEYSPACE1 = "Keyspace1";
private static final String CF_STANDARD1 = "Standard1";
private static final String CF_TENCOL = "TenColumns";
private static final String CF_COUNTER1 = "Counter1";
@BeforeClass
public static void defineSchema() throws ConfigurationException
@ -62,8 +61,7 @@ public class PartitionTest
SchemaLoader.createKeyspace(KEYSPACE1,
KeyspaceParams.simple(1),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD1),
SchemaLoader.standardCFMD(KEYSPACE1, CF_TENCOL, 10, AsciiType.instance),
SchemaLoader.denseCFMD(KEYSPACE1, CF_COUNTER1, BytesType.instance));
SchemaLoader.standardCFMD(KEYSPACE1, CF_TENCOL, 10, AsciiType.instance));
}
@Test

2
test/unit/org/apache/cassandra/db/ReadCommandTest.java
View File

@ -172,7 +172,7 @@ public class ReadCommandTest
TableMetadata.Builder metadata7 =
TableMetadata.builder(KEYSPACE, CF7)
.flags(EnumSet.of(TableMetadata.Flag.COUNTER))
.flags(EnumSet.of(TableMetadata.Flag.COUNTER, TableMetadata.Flag.COMPOUND))
.addPartitionKeyColumn("key", BytesType.instance)
.addClusteringColumn("col", AsciiType.instance)
.addRegularColumn("c", CounterColumnType.instance);

4
test/unit/org/apache/cassandra/db/ScrubTest.java
View File

@ -88,7 +88,6 @@ public class ScrubTest
public static final String CF_INDEX2 = "Indexed2";
public static final String CF_INDEX1_BYTEORDERED = "Indexed1_ordered";
public static final String CF_INDEX2_BYTEORDERED = "Indexed2_ordered";
public static final String COL_INDEX = "birthdate";
public static final String COL_NON_INDEX = "notbirthdate";
@ -521,7 +520,6 @@ public class ScrubTest
CompactionManager.instance.performScrub(cfs2, false, false, 2);
}
@Test /* CASSANDRA-5174 */
public void testScrubKeysIndex_preserveOrder() throws IOException, ExecutionException, InterruptedException
{
@ -563,7 +561,7 @@ public class ScrubTest
@Test /* CASSANDRA-5174 */
public void testScrubTwice() throws IOException, ExecutionException, InterruptedException
{
testScrubIndex(CF_INDEX1, COL_INDEX, false, true, true);
testScrubIndex(CF_INDEX2, COL_INDEX, true, true, true);
}
private void testScrubIndex(String cfName, String colName, boolean composite, boolean ... scrubs)

2
test/unit/org/apache/cassandra/db/commitlog/CommitLogUpgradeTest.java
View File

@ -79,8 +79,6 @@ public class CommitLogUpgradeTest
static TableMetadata metadata =
TableMetadata.builder(KEYSPACE, TABLE)
.isCompound(false)
.isDense(true)
.addPartitionKeyColumn("key", AsciiType.instance)
.addClusteringColumn("col", AsciiType.instance)
.addRegularColumn("val", BytesType.instance)

4
test/unit/org/apache/cassandra/db/compaction/CompactionControllerTest.java
View File

@ -61,14 +61,10 @@ public class CompactionControllerTest extends SchemaLoader
SchemaLoader.createKeyspace(KEYSPACE,
KeyspaceParams.simple(1),
TableMetadata.builder(KEYSPACE, CF1)
.isCompound(false)
.isDense(true)
.addPartitionKeyColumn("pk", AsciiType.instance)
.addClusteringColumn("ck", AsciiType.instance)
.addRegularColumn("val", AsciiType.instance),
TableMetadata.builder(KEYSPACE, CF2)
.isCompound(false)
.isDense(true)
.addPartitionKeyColumn("pk", AsciiType.instance)
.addClusteringColumn("ck", AsciiType.instance)
.addRegularColumn("val", AsciiType.instance));

64
test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java
View File

@ -93,10 +93,6 @@ public class CompactionsTest
private static final String CF_STANDARD2 = "Standard2";
private static final String CF_STANDARD3 = "Standard3";
private static final String CF_STANDARD4 = "Standard4";
private static final String CF_SUPER1 = "Super1";
private static final String CF_SUPER5 = "Super5";
private static final String CF_SUPERGC = "SuperDirectGC";
@BeforeClass
public static void defineSchema() throws ConfigurationException
{
@ -110,17 +106,11 @@ public class CompactionsTest
SchemaLoader.createKeyspace(KEYSPACE1,
KeyspaceParams.simple(1),
SchemaLoader.denseCFMD(KEYSPACE1, CF_DENSE1)
.compaction(CompactionParams.stcs(compactionOptions)),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD1)
.compaction(CompactionParams.stcs(compactionOptions)),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD2),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD3),
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD4),
SchemaLoader.superCFMD(KEYSPACE1, CF_SUPER1, AsciiType.instance),
SchemaLoader.superCFMD(KEYSPACE1, CF_SUPER5, AsciiType.instance),
SchemaLoader.superCFMD(KEYSPACE1, CF_SUPERGC, AsciiType.instance)
.gcGraceSeconds(0));
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARD4));
}
public static long populate(String ks, String cf, int startRowKey, int endRowKey, int ttl)
@ -147,14 +137,14 @@ public class CompactionsTest
public void testSingleSSTableCompaction() throws Exception
{
Keyspace keyspace = Keyspace.open(KEYSPACE1);
ColumnFamilyStore store = keyspace.getColumnFamilyStore(CF_DENSE1);
ColumnFamilyStore store = keyspace.getColumnFamilyStore(CF_STANDARD1);
store.clearUnsafe();
MigrationManager.announceTableUpdate(store.metadata().unbuild().gcGraceSeconds(1).build(), true);
// disable compaction while flushing
store.disableAutoCompaction();
long timestamp = populate(KEYSPACE1, CF_DENSE1, 0, 9, 3); //ttl=3s
long timestamp = populate(KEYSPACE1, CF_STANDARD1, 0, 9, 3); //ttl=3s
store.forceBlockingFlush();
assertEquals(1, store.getLiveSSTables().size());
@ -180,44 +170,6 @@ public class CompactionsTest
assertMaxTimestamp(store, timestamp);
}
@Test
public void testSuperColumnTombstones()
{
Keyspace keyspace = Keyspace.open(KEYSPACE1);
ColumnFamilyStore cfs = keyspace.getColumnFamilyStore("Super1");
TableMetadata table = cfs.metadata();
cfs.disableAutoCompaction();
DecoratedKey key = Util.dk("tskey");
ByteBuffer scName = ByteBufferUtil.bytes("TestSuperColumn");
// a subcolumn
new RowUpdateBuilder(table, FBUtilities.timestampMicros(), key.getKey())
.clustering(ByteBufferUtil.bytes("cols"))
.add("val", "val1")
.build().applyUnsafe();
cfs.forceBlockingFlush();
// shadow the subcolumn with a supercolumn tombstone
RowUpdateBuilder.deleteRow(table, FBUtilities.timestampMicros(), key.getKey(), ByteBufferUtil.bytes("cols")).applyUnsafe();
cfs.forceBlockingFlush();
CompactionManager.instance.performMaximal(cfs, false);
assertEquals(1, cfs.getLiveSSTables().size());
// check that the shadowed column is gone
SSTableReader sstable = cfs.getLiveSSTables().iterator().next();
AbstractBounds<PartitionPosition> bounds = new Bounds<>(key, sstable.getPartitioner().getMinimumToken().maxKeyBound());
UnfilteredRowIterator ai;
try (ISSTableScanner scanner = sstable.getScanner())
{
ai = scanner.next();
final Unfiltered next = ai.next();
assertTrue(next.isRow());
assertFalse(ai.hasNext());
}
}
@Test
public void testUncheckedTombstoneSizeTieredCompaction() throws Exception
{
@ -301,16 +253,6 @@ public class CompactionsTest
assertEquals(maxTimestampExpected, maxTimestampObserved);
}
@Test
public void testDontPurgeAccidentally() throws InterruptedException
{
testDontPurgeAccidentally("test1", "Super5");
// Use CF with gc_grace=0, see last bug of CASSANDRA-2786
testDontPurgeAccidentally("test1", "SuperDirectGC");
}
@Test
public void testUserDefinedCompaction() throws Exception
{

6
test/unit/org/apache/cassandra/db/marshal/CompositeTypeTest.java
View File

@ -69,7 +69,11 @@ public class CompositeTypeTest
SchemaLoader.prepareServer();
SchemaLoader.createKeyspace(KEYSPACE1,
KeyspaceParams.simple(1),
SchemaLoader.denseCFMD(KEYSPACE1, CF_STANDARDCOMPOSITE, composite));
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARDCOMPOSITE,
0,
AsciiType.instance, // key
AsciiType.instance, // value
composite)); // clustering
}
@Test

6
test/unit/org/apache/cassandra/db/marshal/DynamicCompositeTypeTest.java
View File

@ -72,7 +72,11 @@ public class DynamicCompositeTypeTest
SchemaLoader.prepareServer();
SchemaLoader.createKeyspace(KEYSPACE1,
KeyspaceParams.simple(1),
SchemaLoader.denseCFMD(KEYSPACE1, CF_STANDARDDYNCOMPOSITE, dynamicComposite));
SchemaLoader.standardCFMD(KEYSPACE1, CF_STANDARDDYNCOMPOSITE,
0,
AsciiType.instance, // key
AsciiType.instance, // value
dynamicComposite)); // clustering
}
@Test

11
test/unit/org/apache/cassandra/index/internal/CassandraIndexTest.java
View File

@ -793,9 +793,7 @@ public class CassandraIndexTest extends CQLTester
{
assertFalse(resultSet.isEmpty());
TableMetadata cfm = getCurrentColumnFamilyStore().metadata();
int columnCount = cfm.partitionKeyColumns().size();
if (cfm.isCompound())
columnCount += cfm.clusteringColumns().size();
int columnCount = cfm.partitionKeyColumns().size() + cfm.clusteringColumns().size();
Object[] expected = copyValuesFromRow(row, columnCount);
assertArrayEquals(expected, copyValuesFromRow(getRows(resultSet)[0], columnCount));
}
@ -835,17 +833,12 @@ public class CassandraIndexTest extends CQLTester
private Stream<ColumnMetadata> getPrimaryKeyColumns()
{
TableMetadata cfm = getCurrentColumnFamilyStore().metadata();
if (cfm.isCompactTable())
return cfm.partitionKeyColumns().stream();
else
return Stream.concat(cfm.partitionKeyColumns().stream(), cfm.clusteringColumns().stream());
return Stream.concat(cfm.partitionKeyColumns().stream(), cfm.clusteringColumns().stream());
}
private Object[] getPrimaryKeyValues(Object[] row)
{
TableMetadata cfm = getCurrentColumnFamilyStore().metadata();
if (cfm.isCompactTable())
return getPartitionKeyValues(row);
return copyValuesFromRow(row, cfm.partitionKeyColumns().size() + cfm.clusteringColumns().size());
}

9
tools/stress/src/org/apache/cassandra/stress/operations/userdefined/CASQuery.java
View File

@ -28,6 +28,7 @@ import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.ResultSet;
import org.antlr.runtime.RecognitionException;
import org.apache.cassandra.cql3.CQLFragmentParser;
import org.apache.cassandra.cql3.ColumnIdentifier;
import org.apache.cassandra.cql3.CqlParser;
import org.apache.cassandra.cql3.conditions.ColumnCondition;
import org.apache.cassandra.cql3.statements.ModificationStatement;
@ -81,13 +82,13 @@ public class CASQuery extends SchemaStatement
throw new IllegalArgumentException("could not parse update query:" + statement.getQueryString(), e);
}
final List<Pair<ColumnMetadata.Raw, ColumnCondition.Raw>> casConditionList = modificationStatement.getConditions();
final List<Pair<ColumnIdentifier, ColumnCondition.Raw>> casConditionList = modificationStatement.getConditions();
List<Integer> casConditionIndex = new ArrayList<>();
boolean first = true;
StringBuilder casReadConditionQuery = new StringBuilder();
casReadConditionQuery.append("SELECT ");
for (final Pair<ColumnMetadata.Raw, ColumnCondition.Raw> condition : casConditionList)
for (final Pair<ColumnIdentifier, ColumnCondition.Raw> condition : casConditionList)
{
if (!condition.right.getValue().getText().equals("?"))
{
@ -98,8 +99,8 @@ public class CASQuery extends SchemaStatement
{
casReadConditionQuery.append(", ");
}
casReadConditionQuery.append(condition.left.rawText());
casConditionIndex.add(getDataSpecification().partitionGenerator.indexOf(condition.left.rawText()));
casReadConditionQuery.append(condition.left.toString());
casConditionIndex.add(getDataSpecification().partitionGenerator.indexOf(condition.left.toString()));
first = false;
}
casReadConditionQuery.append(" FROM ").append(tableName).append(" WHERE ");