Address comments
This commit is contained in:
parent
fb5c197ec2
commit
52c08ffc85
|
|
@ -121,6 +121,13 @@ the BitmapIndex for the Stripe will be queried for `"LAND"` and will return an i
|
|||
These correspond to the row numbers which match the value
|
||||
(i.e. only these rows should be read into memory), the rest can be skipped.
|
||||
|
||||
For queries with multiple values like `SELECT * FROM animals WHERE type=LAND or type=AERIAL;`,
|
||||
BitmapIndex will perform two lookups. A union will be performed on the two Bitmaps to get the final result
|
||||
For queries with multiple values like `SELECT * FROM animals WHERE type=LAND or type=AERIAL;`, BitmapIndex will perform two lookups. A union will be performed on the two Bitmaps to
|
||||
get the final result
|
||||
(i.e. `[001000] UNION [100011] = [101011]`), therefore the returned iterator will be `[1, 3, 5, 6]`.
|
||||
|
||||
## Disk usage
|
||||
|
||||
Bitmap index internally has a btree structure which uses disk to serialize. Therefore, sufficient space in the system's temporary directory is required for both creation and
|
||||
filtering.
|
||||
|
||||
Check [hindex-statements](./hindex-statements.md) for how to change the temp folder path.
|
||||
|
|
@ -116,7 +116,7 @@ The file name and offsets can be used to filter out Splits which do not match th
|
|||
Additionally, the last modified time can be used to ensure that the index is still valid. If the original ORC file had been modified since the index was created, then the index is
|
||||
invalid and should not be used for filtering.
|
||||
|
||||
## 磁盘使用
|
||||
## Disk usage
|
||||
|
||||
BTree index uses disk to serialize its internal tree structure. Therefore, sufficient space in the system's temporary directory is required for both creation and filtering.
|
||||
|
||||
|
|
|
|||
|
|
@ -107,12 +107,15 @@ Monkey, LAND
|
|||
通过使用BitmapIndex,我们可以改进此过程。而不是读取Stripe中的所有行。
|
||||
BitmapIndex可以返回应读取的匹配行的列表。这样既可以减少内存消耗,又可以缩短查询执行时间。
|
||||
|
||||
如果我们在`type`列上创建BitmapIndex,则在从Stripe读取数据之前,
|
||||
将为Stripe的BitmapIndex查询`LAND`,并将返回具有以下值的迭代器:
|
||||
如果我们在`type`列上创建BitmapIndex,则在从Stripe读取数据之前, 将为Stripe的BitmapIndex查询`LAND`,并将返回具有以下值的迭代器:
|
||||
`[1, 5, 6]`
|
||||
|
||||
这些对应于与值匹配的行号(即仅应将这些行读入内存),其余的可以跳过。
|
||||
|
||||
对于具有多个值的queries,例如`SELECT * FROM animes WHERE type=LAND OR type=AERIAL;`,
|
||||
BitmapIndex将执行两次查找。将对两个Bitmaps执行联合以得到最终结果
|
||||
(例如,`[001000] UNION [100011] = [101011]`),因此返回的迭代器将为`[1、3、5、6]`。
|
||||
对于具有多个值的queries,例如`SELECT * FROM animes WHERE type=LAND OR type=AERIAL;`, BitmapIndex将执行两次查找。将对两个Bitmaps执行联合以得到最终结果 (例如,`[001000] UNION [100011] = [101011]`),因此返回的迭代器将为`[1、3、5、6]`。
|
||||
|
||||
## 磁盘使用
|
||||
|
||||
Bitmap索引内部使用btree数据来序列化。因此,无论是创建还是在执行语句时使用索引,都需要本地临时磁盘空间。
|
||||
|
||||
参见 [hindex-statements](./hindex-statements.md)中的“磁盘使用”章节来指定使用的临时路径。
|
||||
|
|
@ -63,7 +63,7 @@ public class HeuristicIndexFactory
|
|||
return getIndexObjFromID(indexType).getClass().getConstructor().newInstance();
|
||||
}
|
||||
catch (NoSuchMethodException | InvocationTargetException | InstantiationException | IllegalAccessException e) {
|
||||
throw new RuntimeException(String.format("Error creating new instance of class: %s. Check server log for details.", indexType), e);
|
||||
throw new RuntimeException(String.format("Error creating new instance of class: %s. Check the coordinator log for more details.", indexType), e);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -118,6 +118,9 @@ public class BTreeIndex
|
|||
if (dbe.getCause() instanceof IOException) {
|
||||
throw (IOException) dbe.getCause();
|
||||
}
|
||||
else {
|
||||
throw new IOException("Error setting up local mapdb: ", dbe);
|
||||
}
|
||||
}
|
||||
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
|
||||
try {
|
||||
|
|
|
|||
Loading…
Reference in New Issue