mirror of https://github.com/apache/cassandra
Several paths report on-heap allocation through UpdateFunction.onAllocatedOnHeap in a way that diverges from the heap actually retained (BTree.sizeOnHeapOf), so the memtable's owned-heap counter drifts. Five under/over-counting causes are fixed so that reported allocation matches sizeOnHeapOf: 1) BTree.update over-counts on node split/overflow and never counts branch sizeMaps. The Updater's running 'allocated' was not a true net delta: leaf drain() cleared the source before subtracting it, the redistribute/overflow paths added new nodes without releasing the source they replace, and branch sizeMaps were never counted. Fix: account each node net - add every newly retained node's shallow heap (array plus sizeMap) and subtract it for every replaced source, releasing before the source is cleared; and record the root as the top builder's source so the old root is released too. Test: BTreeUpdateHeapAccountingTest (randomized small / contiguous-block / overlapping / height-4, coverage verified by JaCoCo). 2) BTreeRow.merge does not release a row's column tree when a row tombstone shadows its cells: the retain branch rebuilds it smaller via BTree.transformAndFilter (node accounting disabled) but never releases the freed structure. Fix: report sizeOnHeapOf(retained) - sizeOnHeapOf(existing) when the filter shrinks the tree, as ColumnData.Reconciler.merge does. 3) BTreeRow.merge does not account the row's LivenessInfo/Deletion change (e.g. a tombstone replacing a live row). Fix: account (reconciled liveness+deletion) - (existing liveness+deletion). 4) Allocation and release disagree on the branch sizeMap: allocation used sizeOfStructureOnHeap (excludes it), release used sizeOnHeapOf (includes it). Fix: remove sizeOfStructureOnHeap and use sizeOnHeapOf everywhere. 5) ColumnData.removeShadowed does not release a shadowed complex (collection) column's own structure: it releases the inner cells via recordDeletion.delete but not the column's cell tree (which can span multiple nodes) nor, when the column is dropped, its wrapper - both counted as owned when written. Fix: report (EMPTY_SIZE + sizeOnHeapOf(tree)) after - before (after is 0 when dropped); a no-op on the update side (recordDeletion == noOp), as required by CASSANDRA-21469. Tests for 2-5: PartitionRowAccountingTest.rowTombstoneOverExistingRowDoesNotInflateOwnership and .rowTombstoneOverExistingCollectionDoesNotInflateOwnership require two logically identical partitions reached via different merge paths to own exactly the same on-heap (only with all fixes does it match); SetCellAccountingTest guards that a grow/reset op mix on a set<text> column never drives the owned heap negative. patch by Dmitry Konstantinov; reviewed by Caleb Rackliffe for CASSANDRA-21472 |
||
|---|---|---|
| .build | ||
| .circleci | ||
| .jenkins | ||
| bin | ||
| conf | ||
| debian | ||
| doc | ||
| examples/triggers | ||
| ide | ||
| lib | ||
| pylib | ||
| redhat | ||
| src | ||
| test | ||
| tools | ||
| .gitignore | ||
| .snyk | ||
| CASSANDRA-14092.txt | ||
| CHANGES.txt | ||
| CONTRIBUTING.md | ||
| LICENSE.txt | ||
| NEWS.txt | ||
| NOTICE.txt | ||
| README.asc | ||
| TESTING.md | ||
| build-shaded-dtest-jar.sh | ||
| build.properties.default | ||
| build.xml | ||
| eclipse_compiler.properties | ||
| relocate-dependencies.pom | ||
README.asc
Apache Cassandra
-----------------
Apache Cassandra is a highly-scalable partitioned row store. Rows are organized into tables with a required primary key.
https://cwiki.apache.org/confluence/display/CASSANDRA2/Partitioners[Partitioning] means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster.
https://cwiki.apache.org/confluence/display/CASSANDRA2/DataModel[Row store] means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.
For more information, see https://cassandra.apache.org/[the Apache Cassandra web site].
Requirements
------------
. Java >= 1.8 (OpenJDK and Oracle JVMS have been tested)
. Python 3.6+ (for cqlsh; 2.7 works but is deprecated)
Getting started
---------------
This short guide will walk you through getting a basic one node cluster up
and running, and demonstrate some simple reads and writes. For a more-complete guide, please see the Apache Cassandra website's https://cassandra.apache.org/doc/4.0/cassandra/getting_started/index.html[Getting Started Guide].
First, we'll unpack our archive:
$ tar -zxvf apache-cassandra-$VERSION.tar.gz
$ cd apache-cassandra-$VERSION
After that we start the server. Running the startup script with the -f argument will cause
Cassandra to remain in the foreground and log to standard out; it can be stopped with ctrl-C.
$ bin/cassandra -f
Now let's try to read and write some data using the Cassandra Query Language:
$ bin/cqlsh
The command line client is interactive so if everything worked you should
be sitting in front of a prompt:
----
Connected to Test Cluster at localhost:9160.
[cqlsh 2.2.0 | Cassandra 1.2.0 | CQL spec 3.0.0 | Thrift protocol 19.35.0]
Use HELP for help.
cqlsh>
----
As the banner says, you can use 'help;' or '?' to see what CQL has to
offer, and 'quit;' or 'exit;' when you've had enough fun. But lets try
something slightly more interesting:
----
cqlsh> CREATE KEYSPACE schema1
WITH replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
cqlsh> USE schema1;
cqlsh:Schema1> CREATE TABLE users (
user_id varchar PRIMARY KEY,
first varchar,
last varchar,
age int
);
cqlsh:Schema1> INSERT INTO users (user_id, first, last, age)
VALUES ('jsmith', 'John', 'Smith', 42);
cqlsh:Schema1> SELECT * FROM users;
user_id | age | first | last
---------+-----+-------+-------
jsmith | 42 | john | smith
cqlsh:Schema1>
----
If your session looks similar to what's above, congrats, your single node
cluster is operational!
For more on what commands are supported by CQL, see
https://cassandra.apache.org/doc/4.0/cassandra/cql/[the CQL reference]. A
reasonable way to think of it is as, "SQL minus joins and subqueries, plus collections."
Wondering where to go from here?
* Join us in #cassandra on the https://s.apache.org/slack-invite[ASF Slack] and ask questions
* Subscribe to the Users mailing list by sending a mail to
user-subscribe@cassandra.apache.org
* Visit the https://cassandra.apache.org/community/[community section] of the Cassandra website for more information on getting involved.
* Visit the https://cassandra.apache.org/doc/latest/development/index.html[development section] of the Cassandra website for more information on how to contribute.