Add trie memtable docs-storage engine, create and alter table; revised per blambov's comments.

Authored by Lorina Poland (polandll); Reviewed by Branimir Lambov (blambov) for CASSANDRA-18236

doc/modules/cassandra/examples/CQL/cyclist_id-table.cql

@@ -17,11 +17,34 @@ CREATE TABLE IF NOT EXISTS cycling.cyclist_id (
 );
 // end::comp_pk[]
 
+// tag::trie-memtable[]
+CREATE TABLE IF NOT EXISTS cycling.cyclist_id (
+  lastname text,
+  firstname text,
+  age int,
+  id UUID,
+  PRIMARY KEY ((lastname, firstname), age)
+) WITH memtable = {'trie'};
+// end::trie-memtable[]
+
+// tag::skiplist-memtable[]
+CREATE TABLE IF NOT EXISTS cycling.cyclist_id (
+  lastname text,
+  firstname text,
+  age int,
+  id UUID,
+  PRIMARY KEY ((lastname, firstname), age)
+) WITH memtable = {'skiplist'};
+// end::skiplist-memtable[]
+
 // tag::altercdc[]
-ALTER TABLE cycling.cyclist_id
+ALTER TABLE cycling.cyclist_id WITH CDC = false;
-WITH CDC = false;
 // end::altercdc[]
 
+// tag::table-alter-memtable[]
+ALTER TABLE cycling.cyclist_id WITH memtable = 'default';
+// end::table-alter-memtable[]
+
 INSERT INTO cycling.cyclist_id (
   lastname, firstname, age, id
 ) VALUES (

doc/modules/cassandra/pages/architecture/storage-engine.adoc

@@ -111,6 +111,55 @@ If a node stops working, replaying the commit log restores writes to the memtabl
 
 Data in the commit log is purged after its corresponding data in the memtable is flushed to an SSTable on disk.
 
+=== Trie memtables
+
+An alternative memtable implementation based on tries, also called prefix trees, is provided alongside the legacy skip list solution. 
+The implementation is activated using the memtable API (https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-11%3A+Pluggable+memtable+implementations[CEP-11] / https://issues.apache.org/jira/browse/CASSANDRA-17034[CASSANDRA-17034]).
+Trie memtables improve on the legacy solution in modification and lookup performance, as well as the size of the structure for a given amount of data by fitting more data.
+
+Trie memtables use a data structure called a trie to organize data. 
+This structure makes them very efficient at modifying and querying data, as well as more compact in memory. 
+These features result in higher write throughput, lower latency for accessing recently-written data, while fitting more of it in memory.
+
+Trie memtables have internal memory management mechanisms, which drastically reduce the amount of work needed for garbage collection, reducing GC-inflicted pauses and higher-percentile latencies.
+This improvement is crucial to {cassandra} as it reduces the impact of GC on the system and allows for more predictable performance.
+
+Trie memtables reduce write amplification, a common problem in database systems, by buffering and organizing writes until they fill up their allocated memory. 
+By accepting up to 30% more data for the same memory allocation, trie memtables reduce write amplification further.
+
+In the trie memtable implementation, the concurrent skip-list partitions map is replaced with a sharded single-writer trie. 
+To maintain partition order, all keys are mapped to their byte-comparable representations. 
+To minimize the size of the structure, the keys are only stored in the trie paths, and converted back to the standard format on retrieval.
+
+// In later iterations this will be expanded to include the partition-to-row maps, forming a direct map to rows and doing away with most of the complexity and overhead of maintaining separate partition maps.
+
+The trie memtable implementation is a pluggable memtable implementation, and can be enabled by setting the `memtable` configuration in `cassandra.yaml` to `trie`.
+
+[source, yaml]
+----
+memtable:
+    configurations:
+        skiplist:
+          class_name: SkipListMemtable
+        trie:
+          class_name: TrieMemtable
+        default:
+          inherits: trie
+----
+
+This configuration allows for the use of different memtable implementations for different tables, or for different stages of development or testing.
+
+Cassandra currently comes with two memtable implementations:
+
+* SkipListMemtable is the default and matches the memtable format of Cassandra versions up to 4.1. It organizes partitions into a single concurrent skip list.
+* TrieMemtable is a novel solution that organizes partitions into an in-memory trie which places the partition indexing structure in a buffer, off-heap if desired, which significantly improves garbage collection efficiency. 
+It also improves the memtable's space efficiency and lookup performance. 
+Its configuration uses a single parameter:
+** shards: the number of shards to split into, defaulting to the number of CPU cores on the machine
+
+Trie memtables, once configured in `cassandra.yaml`, can be used by setting the `memtable` configuration in the table definition in xref:cassandra:reference/cql-commands/create-table.adoc[`CREATE TABLE`] or xref:cassandra:reference/cql-commands/alter-table.adoc[`ALTER TABLE`] statements.
+The other method is to change the default setting in the `cassandra.yaml` to inherit trie.
+
 == SSTables
 
 https://cassandra.apache.org/_/glossary.html#sstable[SSTables] are the immutable data files that Cassandra uses for persisting data on disk.

doc/modules/cassandra/pages/new/index.adoc

@@ -15,7 +15,7 @@ This section covers the new features in Apache Cassandra 6.0.
 This section covers the new features in Apache Cassandra 5.0.
 
 * Storage Attached Indexes: xref:cassandra:developing/cql/indexing/sai/sai-overview.adoc[Docs], https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-7%3A+Storage+Attached+Index[CEP-7],  https://issues.apache.org/jira/browse/CASSANDRA-16052[JIRA ticket]
-* Trie memtables: https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-19%3A+Trie+memtable+implementation[CEP-19], https://issues.apache.org/jira/browse/CASSANDRA-17240[JIRA ticket]
+* Trie memtables: xref:cassandra:architecture/storage-engine.adoc#trie-memtables[Docs], https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-19%3A+Trie+memtable+implementation[CEP-19], https://issues.apache.org/jira/browse/CASSANDRA-17240[JIRA ticket]
 * Trie SSTables: https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format[CEP-25], https://issues.apache.org/jira/browse/CASSANDRA-18398[JIRA ticket]
 * JDK 17: xref:cassandra:reference/java17.adoc[Docs], https://issues.apache.org/jira/browse/CASSANDRA-16895[JIRA ticket]
 * More guardrails: https://github.com/apache/cassandra/blob/trunk/NEWS.txt[NEWS.txt]

doc/modules/cassandra/pages/reference/cql-commands/alter-table.adoc

@@ -336,6 +336,17 @@ Lists the number of pending tasks for a compaction strategy.
  "time":1470083447967,"strategyId":"1","pending":200}
 ----
 
+== Alter a table with memtable
+
+To alter a table with memtable, a memtable configuration must be enabled in `cassandra.yaml`. See xref:architecture/storage-engine/memtable.adoc[Memtable] for more information.
+
+To alter a table with a default memtable:
+
+[source,language-cql]
+----
+include::cassandra:example$CQL/cyclist_id-table.cql[tag=table-alter-memtable]
+----
+
 === Reviewing the table definition
 
 Use `DESCRIBE` or `DESC` to view the table definition.

doc/modules/cassandra/pages/reference/cql-commands/create-table-examples.adoc

@@ -68,6 +68,28 @@ After the disk space limit is reached, writes to CDC-enabled tables are rejected
 See https://docs.datastax.com/en/dse/6.8/dse-admin/datastax_enterprise/config/configCassandra_yaml.html#cdcSpaceSection[Change-data-capture (CDC) space settings] for information about available CDC settings.
 ====
 
+== Create a table with a trie memtable
+
+To create a table with a trie memtable, a memtable configuration must be enabled in `cassandra.yaml`. See xref:architecture/storage-engine/memtable.adoc[Memtable] for more information.
+
+To create a table with a trie memtable:
+
+[source,language-cql]
+----
+include::cassandra:example$CQL/cyclist_id-table.cql[tag=trie-memtable]
+----
+
+== Create a table with a skiplist memtable
+
+To create a table with a skiplist memtable, a memtable configuration must be enabled in `cassandra.yaml`. See xref:architecture/storage-engine/memtable.adoc[Memtable] for more information.
+
+To create a table with a skiplist memtable:
+
+[source,language-cql]
+----
+include::cassandra:example$CQL/cyclist_id-table.cql[tag=skiplist-memtable]
+----
+
 == Storing data in descending order
 
 The following example shows a table definition that stores the categories with the highest points first.

doc/modules/cassandra/partials/table-properties.adoc

@@ -105,6 +105,19 @@ If your application uses batch operations, consider the possibility that decreas
 The configuration/cass_yaml_file.html#batchlog_replay_throttle[batchlog_replay_throttle] property in the cassandra.yaml file give some control of the batch replay process.
 The most important factors, however, are the size and scope of the batches you use.
 
+*memtable*::
+The memtable implementation can be configured per table by setting the memtable property in the table definition to one of the choices defined in the `cassandra.yaml` file:
+
+* skiplist
+* trie
+* default
+
+If the memtable property is not set, the "default" configuration will be used.
+In the `cassandra.yaml` file, the default is set to `skiplist`.
+
+The `trie` option has the parameter `shards`, which is the number of shards to use for the memtable.
+
+
 *memtable_flush_period_in_ms* ::
 Milliseconds before `memtables` associated with the table are flushed.
 When memtable_flush_period_in_ms=0, the memtable will flush when: