!703 add FAQ pages in openLooKeng website
* add FAQ pages in openLooKeng website
This commit is contained in:
xudezhi
Raghunandan
parent
ea8c669962
commit
ebb10c0846
|
|
@ -0,0 +1,239 @@
|
|||
# openLooKeng FAQs
|
||||
|
||||
## Common Questions
|
||||
|
||||
1. What license is used by the openLooKeng?
|
||||
|
||||
> The openLooKeng uses Apache License 2.0.
|
||||
|
||||
2. Does the openLooKeng support x86 and ARM? What OSs are supported?
|
||||
|
||||
> The openLooKeng supports both x86 and ARM. Currently, openLooKeng can run on Linux OSs, such as openEuler, CentOS, and openSUSE.
|
||||
|
||||
3. What should I do if I find a vulnerability in the openLooKeng?
|
||||
|
||||
> The community has a CVE handling process. If a suspected security issue is found, you can report it the community through secrets@openLooKeng.io. We will respond and handle the issue as soon as possible.
|
||||
|
||||
4. How do I report a bug?
|
||||
|
||||
> You can go the **Issues** tab page (https://gitee.com/organizations/openLooKeng/issues) of the openLooKeng community, find the repository to which the bug belongs, add an issue, and set the issue type to **Bug**.
|
||||
|
||||
5. How do I submit a patch?
|
||||
|
||||
> Any person can submit a patch in the form of pull request (PR) in the openLooKeng community. The PR will be reviewed, fed back, and integrated by the module maintainer. For details, visit https://gitee.com/help/articles/4122.
|
||||
|
||||
6. What are the version plan and roadmap of the openLooKeng community?
|
||||
|
||||
> On June 30, 2020, the code was released to the community in the openLooKeng v0.1.0 version for sharing with global developers. It is planned that an iterative version is released every three months to continuously launch new features. The roadmap of the openLooKeng will be released as required. For details, visit the openLooKeng commuity bi-weekly PMC meeting (https://etherpad.openlookeng.io/p/PMC-weeklymeeting) or the openLooKeng Slack channel (https://openLooKeng.io/zh-cn/communication.html).
|
||||
|
||||
7. How can I obtain openLooKeng information and communicate with other developers about openLooKeng technologies?
|
||||
|
||||
> Community issues: https://gitee.com/openlookeng
|
||||
>
|
||||
> Official website: https://openlookeng.io
|
||||
>
|
||||
> Slack channel: https://openlookeng.slack.com/
|
||||
>
|
||||
> openLooKeng WeChat Official Account: **openLooKeng**
|
||||
>
|
||||
> openLooKeng WeChat group: add the assistant's account **openLooKengoss**, and the assistant will invite you to join the group.
|
||||
>
|
||||
> openLooKeng Bilibili channel: Search **openLooKeng** on https://www.bilibili.com/**.**
|
||||
|
||||
## Functions
|
||||
|
||||
1. What connectors does the openLooKeng support?
|
||||
|
||||
> Data source connectors supported by the openLooKeng
|
||||
>
|
||||
> | Data Source| Performance Optimization|
|
||||
> ----------|----------
|
||||
> CarbonData| No
|
||||
> Elasticsearch| No
|
||||
> Greenplum| Operator pushdown
|
||||
> Hana| Operator pushdown
|
||||
> HBase| Sharding rule optimization
|
||||
> Hive| Dynamic filtering, table scan reuse, ORC pushdown, and DM optimization
|
||||
> Apache Kafka| No
|
||||
> MySQL| Operator pushdown
|
||||
> PostgreSQL| No
|
||||
> SQL Server| No
|
||||
> Oracle| Operator pushdown
|
||||
> MongoDB| No
|
||||
> openGauss| No
|
||||
|
||||
> Other connectors supported by the openLooKeng
|
||||
>
|
||||
> | Connector | Description |
|
||||
> ------------ | -------------------------------------------------------
|
||||
> Data center connector | The data center connector allows querying remote openLooKeng data centers. Converged analysis can be performed on data between different openLooKeng clusters in the local openLooKeng environment.
|
||||
> JMX connector | The JMX connector provides the capability of querying JMX information from all nodes in the openLooKeng cluster. This is useful for viewing or debugging.
|
||||
> Local file connector | The local file connector allows you to query data stored in the local file system of each worker node.
|
||||
> Memory connector | The memory connector stores all data and metadata in the RAM on the worker node. When the openLooKeng is restarted, the data and metadata are discarded.
|
||||
> System connector | The system connector provides information and metrics of the currently running openLooKeng cluster. The information and metrics are obtained through normal SQL queries.
|
||||
> TPCDS connector | The TPCDS connector provides a set of schemas to support TPC Benchmark™ DS (TPC-DS). TPC-DS is a database benchmark used to evaluate the performance of complex decision support databases.
|
||||
> TPCH connector | The TPCH connector provides a set of schemas to support TPC Benchmark™ H (TPC-H). TPC-H is a database benchmark used to evaluate the performance of highly complex decision support databases.
|
||||
> Virtual data market connector | In some scenarios, multiple data sources need to be managed or accessed in the same session or view, and users do not need to pay attention to data distribution and sources. The virtual data market (VDM) connector is designed to introduce this feature into the openLooKeng.
|
||||
|
||||
2. What external interfaces does the openLooKeng provide?
|
||||
|
||||
> openLooKeng provides the following external interfaces: JDBC, ODBC, and RESTful APIs. The SQL syntax complies with the SQL:2003 standard.
|
||||
|
||||
3. How do I develop UDFs for the openLooKeng?
|
||||
|
||||
> The openLooKeng provides the user-defined function feature. For details, see https://openlookeng.io/docs/docs/develop/functions.html. The openLooKeng also supports user-defined functions in the Hive data source. For details, see https://openlookeng.io/docs/docs/migration/hetu-hive-functions.html.
|
||||
|
||||
4. Can I manage manually added data sources after the dynamic directory function is enabled?
|
||||
|
||||
> If the dynamic directory is enabled, use the dynamic directory to add, delete, and update directories. Do not access the installation directory of the openLooKeng to manually modify the directories. Otherwise, the configuration may be faulty.
|
||||
|
||||
5. After the dynamic directory is configured, the error message "*XXX* path must be at user workspace" is displayed when the openLooKeng is started.
|
||||
|
||||
> Check **catalog.config-dir** and **catalog.share.config-dir** in the **etc/node.properties** file, ensure that the two paths are in the allowlist: \["/tmp", "/opt/hetu", "/opt/openLooKeng", "/etc/hetu", "/etc/openLooKeng", *work directory*]. Note: Do not select the root directory. The path cannot contain **../**.
|
||||
>
|
||||
> If **node.data\_dir** is configured, the current work directory is the parent directory of **node.data\_dir**. Otherwise, the current work directory is the directory of the openLooKeng server.
|
||||
|
||||
6. Are there any cURL command examples for adding or deleting dynamic directories?
|
||||
|
||||
> cURL examples for performing operations on dynamic directories: Adding the dynamic data source MySQL
|
||||
>
|
||||
> ```
|
||||
> curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin' -F 'catalogInformation={ "catalogName" : "mysql", "connectorName" : "mysql", "properties" : { "connection-url" : "jdbc:mysql://127.0.0.1:3306/", "connection-user" : "mysql", "connection-password" : "password"}}'
|
||||
> ```
|
||||
>
|
||||
> Adding the dynamic data source memory
|
||||
>
|
||||
> ```
|
||||
> curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin' -F 'catalogInformation={ "catalogName" : "memory", "connectorName" : "memory", "properties" : { } }'
|
||||
> ```
|
||||
>
|
||||
> Adding the dynamic data source TPC-H
|
||||
>
|
||||
> ```
|
||||
> `curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin' -F 'catalogInformation={ "catalogName" : "tpch", "connectorName" : "tpch", "properties" : { } }'`
|
||||
> ```
|
||||
>
|
||||
> Adding the dynamic data source Hive
|
||||
>
|
||||
> ```
|
||||
> curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin'
|
||||
> -F 'catalogInformation={ "catalogName" : "hive", "connectorName" : "hive-hadoop2", "properties" : {"hive.metastore.uri":"thrift://hadoop-master:9083","hive.allow-drop-table":"true","hive.max-partitions-per-writers":"1000","hive.config.resources":"core-site.xml,hdfs-site.xml"}}'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/core-site.xml'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/hdfs-site.xml'
|
||||
> ```
|
||||
>
|
||||
> Updating the dynamic data source Hive
|
||||
>
|
||||
> ```
|
||||
> curl -X UPDATE http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin'
|
||||
> -F 'catalogInformation={ "catalogName" : "hive", "connectorName" : "hive-hadoop2", "properties" : {"hive.metastore.uri":"thrift://hadoop-master:9083","hive.allow-drop-table":"true","hive.max-partitions-per-writers":"1000","hive.config.resources":"core-site.xml,hdfs-site.xml"}}'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/core-site.xml'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/hdfs-site.xml'
|
||||
> ```
|
||||
>
|
||||
> Deleting the dynamic data source TPC-H
|
||||
>
|
||||
> ```
|
||||
> curl -X DELETE -H "X-Presto-User: admin" "http://127.0.0.1:8080/v1/catalog/tpch"
|
||||
> ```
|
||||
|
||||
7. How many historical records are saved when SQL statements are executed on the WebUI?
|
||||
|
||||
> In the current version 1.1.0, the SQL statement execution on the WebUI is not persisted. Therefore, the number of historical records that can be saved depends on the maximum size of the memory in the actual running environment. Historical records will be persisted to files in the future.
|
||||
|
||||
8. How do I print error stack information in openLooKeng logs during fault locating?
|
||||
|
||||
> Set **stack-trace-visible** to **true** in the **config.properties** file and restart the openLooKeng.
|
||||
|
||||
9. Do data sources support heuristic indexes?
|
||||
|
||||
> Currently, only the ORC data format of the Hive connector supports index creation.
|
||||
|
||||
## Data Sources
|
||||
|
||||
1. Are SQL statements of the openLooKeng case-sensitive? If SQL statements of the data source are case-sensitive, how does the openLooKeng process them?
|
||||
|
||||
> SQL statements of the openLooKeng are case-insensitive. Uppercase letters in SQL statements are converted to lowercase letters. If SQL statements of the data source are case-sensitive, queries may fail. For example, SQL statements of the Elasticsearch is case-sensitive. If the column name in Elasticsearch is in uppercase, an error occurs when the column is queried in the openLooKeng.
|
||||
|
||||
2. What do I do If the error message "cannot change stats state for a transactional table" is displayed when an update operation fails after the openLooKeng connects to the Hive?
|
||||
|
||||
> This problem occurs on the open-source Hive 4. To solve this problem in the current version, set **hive.txn.stats.enabled** to **false**.
|
||||
|
||||
3. What do I do if the error message "unpartitioned hive tables are immutable" is displayed when an update operation fails after the openLooKeng connects to the Hive?
|
||||
|
||||
> Check whether **hive.immutable-partitions** in **hive.properties** is set to **true**. If yes, change it to **false**.
|
||||
|
||||
4. What can I do if tables whose names contain underscores (\_) cannot be queried when the openLooKeng connects to the Oracle Database 10g?
|
||||
|
||||
> The Oracle Database 10g is an old version. If the ojdbc of a later version is used, incompatibility may occur. After multiple ojdbc versions have been used, it is found that the openLooKeng can use ojdbc6 to query data in the Oracle Database 10g.
|
||||
|
||||
5. When the openLooKeng connects to the Oracle database, why the data type queried in the Oracle database is NCLOB while the data type in the table created by the openLooKeng is varchar?
|
||||
|
||||
> There is a mapping between openLooKeng and Oracle data types. For details, see "Mapping Data Types Between openLooKeng and Oracle" in the **Oracle Connector** section on the openLooKeng official website. You can use varchar(n) in openLooKeng, which is varchar2(n) in Oracle.
|
||||
|
||||
6. How do I configure the openLooKeng to connect to the Hive data source of Huawei FusionInsight 6.5.1?
|
||||
|
||||
> The Hive of FusionInsight 6.5.1 is different from the open-source Apache Hive. Therefore, configure the connector based on the open-source Hive configuration and then perform the following modification:
|
||||
>
|
||||
> (1) Set the **dfs.client.failover.proxy.provider.hacluster** configuration item in the **hdfs-site.xml** file to **org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider**.
|
||||
>
|
||||
> (2) If data in the Hive of FusionInsight 6.5.1 is in the .txt format, copy the **hadoop-plugins-1.0.jar** file in the FusionInsight 6.5.1 installation directory to the **plugin/hive-hadoop2** directory in the openLooKeng installation directory and restart the openLooKeng.
|
||||
|
||||
## Security
|
||||
|
||||
2. How do I add openLooKeng service components on the Ranger page?
|
||||
|
||||
> You can use cURL commands to register the Ranger service type definition. An example is as follows:
|
||||
>
|
||||
> ```
|
||||
> curl -u admin:password -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d @service-defs/Ranger-servicedef-openlookeng.json http://Ranger-admin-host:port/service/plugins/definitions
|
||||
> ```
|
||||
>
|
||||
> The openLooKeng service type can be viewed on the Ranger management page only after the openLooKeng service is registered. Then, you can add the openLooKeng service on the Ranger management page.
|
||||
|
||||
3. How do I set openLooKeng service parameters in the Ranger page?
|
||||
|
||||
> An example is as follows:
|
||||
>
|
||||
> 
|
||||
|
||||
4. Why is the warning message "WARN PolicyRefresher(serviceName=openlookengdev)-196 org.apache.Ranger.admin.client.RangerAdminRESTClient Error getting Roles. secureMode=false, user=root (auth:SIMPLE), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=openlookengdev" displayed in the openLooKeng log after the openLooKeng is connected to the Ranger?
|
||||
|
||||
> This problem is caused by the low RangerAdmin version. The Ranger plugin provided by the openLooKeng is used to connect to Ranger 2.1. Update the Ranger plugin.
|
||||
|
||||
5. Is the internal communication security of the openLooKeng mandatory?
|
||||
|
||||
> This configuration is not mandatory. Setting the openLooKeng to the non-security mode is acceptable. However, if LDAP authentication is required, SSL must be configured. For details, see the **LDAP Authentication** section on the openLooKeng website.
|
||||
|
||||
6. Does the openLooKeng support application log monitoring and audit? Is secondary development required?
|
||||
|
||||
> Currently, the openLooKeng has the audit function for SQL creation and completion. If more user events need to be audited, for example, WebUI login events, you can perform secondary development to add the function.
|
||||
|
||||
7. What do I do if the field association function on the RangerAdmin is unavailable when the openLooKeng is connected to the Ranger after Kerberos authentication is enabled?
|
||||
|
||||
> The association function of RangerAdmin needs to send SQL statements to query olk-related resources. Therefore, when adding the openLooKeng service on RangerAdmin, you need to add the Kerberos information to the **jdbc.url** parameter. An example is as follows:
|
||||
>
|
||||
> ```
|
||||
> jdbc:lk://server1:10001/catalog?SSL=true&SSLKeyStorePath=/etc/openLooKeng_keystore.jks&SSLKeyStorePassword=keystore_password&KerberosRemoteServiceName=openLooKeng&KerberosConfigPath=/etc/krb5.conf&KerberosPrincipal=cpk@EXAMPLE.COM&KerberosKeytabPath=/etc/security/keytabs/cpk.keytab
|
||||
> ```
|
||||
|
||||
8. How do I connect the openLooKeng to two Hive data sources configured with Kerberos?
|
||||
|
||||
> Ensure that the domain names of the two Hive data sources configured with Kerberos are different.
|
||||
>
|
||||
> Combine the **krb5.conf** files of the two data sources, and add the realm information of one data source to the **krb5.conf** file of the other data source.
|
||||
>
|
||||
> Save the combined **krb5.conf** file to the path configured in **jvm.config**. An example is as follows:
|
||||
>
|
||||
> 
|
||||
|
||||
9. How do I connect the openLooKeng to a Hive data source configured with Kerberos and a Hive data source not configured with Kerberos at the same time?
|
||||
|
||||
> Add the following configuration to the **core-site.xml** file of the Hive data source that is not configured with Kerberos:
|
||||
>
|
||||
> ```
|
||||
> <property>
|
||||
> <name>ipc.client.fallback-to-simple-auth-allowed</name>
|
||||
> <value>true</value>
|
||||
> </property>
|
||||
> ```
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 386 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
|
|
@ -207,3 +207,5 @@ headless: true
|
|||
- [1.0.1]({{< relref "./docs/releasenotes/releasenotes-1.0.1.md" >}})
|
||||
- [1.0.0]({{< relref "./docs/releasenotes/releasenotes-1.0.0.md" >}})
|
||||
- [0.1.0]({{< relref "./docs/releasenotes/releasenotes-0.1.0.md" >}})
|
||||
|
||||
- [FAQ]({{< relref "./docs/faq/faq.md" >}})
|
||||
|
|
|
|||
|
|
@ -0,0 +1,239 @@
|
|||
# openLooKeng FAQ
|
||||
|
||||
## 通用问题
|
||||
|
||||
1. openLooKeng开源使用的是什么许可证?
|
||||
|
||||
> openLooKeng使用Apache License 2.0。
|
||||
|
||||
2. openLooKeng是否支持X86和ARM？可以运行在哪些操作系统上?
|
||||
|
||||
> openLooKeng对于X86/ARM均可以支持；对于操作系统，目前openLooKeng支持openEuler、CentOS、openSUSE等Linux操作系统。
|
||||
|
||||
4. 如果发现openLooKeng存在漏洞，将如何处理？
|
||||
|
||||
> 社区有CVE处理流程，如果发现疑似安全问题，可通过securities@openLooKeng.io 通知社区，我们将在第一时间响应并处理。
|
||||
|
||||
5. 如何上报一个Bug？
|
||||
|
||||
> 可以访问openLooKeng社区Issue板块（https://gitee.com/organizations/openLooKeng/issues），找到Bug所属的仓库新建Issue并将问题类型选择为缺陷。
|
||||
|
||||
6. 如何提交补丁？
|
||||
|
||||
> 任何人员都可以在openLooKeng社区以Pull Request（简称PR）的形式提交补丁，PR将会由模块Maintainer检视、反馈、合入。具体操作请参考：https://gitee.com/help/articles/4122
|
||||
|
||||
7. openLooKeng的社区版本计划和roadmap？
|
||||
|
||||
> 2020年6月30日 正式将代码开放到社区，发布openLooKeng v0.1.0版本，与全球开发者一起共建社区。后续规划每3个月发布一个迭代release版本，持续上线新特性。openLooKeng的roadmap会按需发布，请关注openLooKeng社区双周PMC Meeting：https://etherpad.openlookeng.io/p/PMC-weeklymeeting，或到openLooKeng Slack频道查看。openLooKeng Slack入口连接在：https://openLooKeng.io/zh-cn/communication.html。
|
||||
|
||||
8. 哪些渠道可以获取openLooKeng的相关资讯和进行技术交流？
|
||||
|
||||
> 社区issue：https://gitee.com/openlookeng
|
||||
> 官网主页：https://openlookeng.io
|
||||
> Slack频道: https://openlookeng.slack.com/
|
||||
> openLooKeng公众号：公众号搜索openLooKeng
|
||||
> openLooKeng微信群：请添加openLooKeng小助手微信号：openLooKengoss，小助手会拉您入群
|
||||
> openLooKeng B站：https://www.bilibili.com/ 搜索openLooKeng
|
||||
|
||||
## 功能
|
||||
|
||||
1. openLooKeng目前支持哪些连接器?
|
||||
|
||||
>openLooKeng目前支持的数据源连接器：
|
||||
>
|
||||
>| 数据源 | 是否有性能优化 |
|
||||
> ------------- | --------------------------------------------------------
|
||||
> CarbonData | 否
|
||||
> Elasticsearch | 否
|
||||
> Greenplum | 有，算子下推
|
||||
> Hana | 有，算子下推
|
||||
> HBase | 有，优化分片规则
|
||||
> Hive | 有， 动态过滤、table scan reuse， ORC pushdown，DM优化等
|
||||
> Apache Kafka | 否
|
||||
> MySQL | 有，算子下推
|
||||
> PostgreSQL | 否
|
||||
> SQL Server | 否
|
||||
> Oracle | 有，算子下推
|
||||
> MongoDB | 否
|
||||
> openGauss | 否
|
||||
>
|
||||
>openLooKeng支持的其他连接器：
|
||||
>
|
||||
>| 其他连接器 | 说明 |
|
||||
> ------------ | -------------------------------------------------------
|
||||
> 数据中心连接器 | 数据中心连接器允许查询远程openLooKeng数据中心。可实现来自本地openLooKeng环境的不同openLooKeng集群之间的数据融合分析。
|
||||
> JMX连接器 | JMX连接器提供从openLooKeng集群中的所有节点查询JMX信息的能力。这对于检视或调试非常有用。
|
||||
> 本地文件连接器 | 本地文件连接器允许查询每个工作节点的本地文件系统上存储的数据。
|
||||
> 内存连接器 | 内存连接器将所有数据和元数据存储在工作节点上的RAM中，当openLooKeng重新启动时，数据和元数据都将被丢弃。
|
||||
> 系统连接器 | 系统连接器提供当前运行的openLooKeng集群的信息和指标。这通过正常SQL查询实现。
|
||||
> TPCDS连接器 | TPCDS连接器提供一组模式来支持TPC Benchmark™ DS (TPC-DS)。TPC-DS是一个数据库基准，用于衡量复杂决策支持数据库的性能。
|
||||
> TPCH连接器 | TPCH连接器提供一组模式来支持TPC Benchmark™ H (TPC-H)。TPC-H是一个数据库基准，用于衡量高度复杂决策支持数据库的性能。
|
||||
> 虚拟数据市场连接器 | 在一些场景中，需要在同一个会话或视图中管理或访问多个数据源，同时，用户不需要关心数据的分布和来源。虚拟数据市场（VDM）连接器旨在将该特性引入openLooKeng。
|
||||
>
|
||||
|
||||
2. openLooKeng对外提供哪些接口？
|
||||
|
||||
> openLooKeng提供以下对外接口：JDBC、ODBC、Rest-API。SQL语法遵守SQL2003标准。
|
||||
|
||||
3. openLooKeng的UDF如何开发？
|
||||
|
||||
> openLooKeng提供User Defined Function能力，详情参考：https://openlookeng.io/zh-cn/docs/docs/develop/functions.html
|
||||
> openLooKeng也支持hive数据源中用户自定义函数，详细参考：https://openlookeng.io/docs/docs/migration/hetu-hive-functions.html
|
||||
|
||||
4. 动态目录功能打开后，是否还能管理手动添加的数据源？
|
||||
|
||||
> 如果启用了动态目录，那么请完全使用动态目录来添加、删除和更新目录，而不要再人为进入openLooKeng的安装目录手动修改，以免引起配置混乱的问题。
|
||||
|
||||
5. 配置动态目录后，启动openLooKeng报错：XXX path must be at user workspace
|
||||
|
||||
> 检查etc/node.properties文件中的catalog.config-dir和catalog.share.config-dir，请确保这两个路径配置在白名单中：["/tmp”, “/opt/hetu”, “/opt/openLooKeng”, “/etc/hetu”, “/etc/openLooKeng”, 工作目录]。
|
||||
> 注意：避免选择根目录；路径不能包含../；
|
||||
>
|
||||
> 工作目录是指：如果配置了node.data_dir,那么当前工作目录为node.data_dir的父目录； 如果没有配置，那么当前工作目录为openLooKeng server的目录。
|
||||
|
||||
6. 动态目录的添加删除是否有curl命令样例？
|
||||
|
||||
> 动态目录操作的curl样例参考：
|
||||
> 添加动态数据源mysql
|
||||
>
|
||||
> ```
|
||||
> curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin' -F 'catalogInformation={ "catalogName" : "mysql", "connectorName" : "mysql", "properties" : { "connection-url" : "jdbc:mysql://127.0.0.1:3306/", "connection-user" : "mysql", "connection-password" : "password"}}'
|
||||
> ```
|
||||
>
|
||||
> 添加动态数据源memory
|
||||
>
|
||||
> ```
|
||||
> curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin' -F 'catalogInformation={ "catalogName" : "memory", "connectorName" : "memory", "properties" : { } }'
|
||||
> ```
|
||||
>
|
||||
> 添加动态数据源tpch
|
||||
>
|
||||
> ```
|
||||
> `curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin' -F 'catalogInformation={ "catalogName" : "tpch", "connectorName" : "tpch", "properties" : { } }'`
|
||||
> ```
|
||||
>
|
||||
> 添加动态数据源hive
|
||||
>
|
||||
> ```
|
||||
> curl -X POST http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin'
|
||||
> -F 'catalogInformation={ "catalogName" : "hive", "connectorName" : "hive-hadoop2", "properties" : {"hive.metastore.uri":"thrift://hadoop-master:9083","hive.allow-drop-table":"true","hive.max-partitions-per-writers":"1000","hive.config.resources":"core-site.xml,hdfs-site.xml"}}'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/core-site.xml'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/hdfs-site.xml'
|
||||
> ```
|
||||
>
|
||||
> 更新动态数据源hive
|
||||
>
|
||||
> ```
|
||||
> curl -X UPDATE http://127.0.0.1:8080/v1/catalog -H 'X-Presto-User: admin'
|
||||
> -F 'catalogInformation={ "catalogName" : "hive", "connectorName" : "hive-hadoop2", "properties" : {"hive.metastore.uri":"thrift://hadoop-master:9083","hive.allow-drop-table":"true","hive.max-partitions-per-writers":"1000","hive.config.resources":"core-site.xml,hdfs-site.xml"}}'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/core-site.xml'
|
||||
> -F 'catalogConfigurationFiles=@/path-dir/hdfs-site.xml'
|
||||
> ```
|
||||
>
|
||||
> 删除动态数据源tpch
|
||||
>
|
||||
> ```
|
||||
> curl -X DELETE -H "X-Presto-User: admin" "http://127.0.0.1:8080/v1/catalog/tpch"
|
||||
> ```
|
||||
|
||||
8. webUI中执行sql，历史记录保存多少条记录？
|
||||
|
||||
> 目前版本1.1.0中，WebUI上的sql执行没有持久化，所以实际运行环境的内存能放多少是多少，后续会考虑将历史记录持久化到文件。
|
||||
|
||||
9. 问题定位时openLooKeng日志中缺少错误栈信息，如何才能打印出来？
|
||||
|
||||
> 请在config.properties文件中配置stack-trace-visible=true，然后重启openLooKeng。
|
||||
|
||||
10. 数据源都支持使用启发式索引吗？
|
||||
|
||||
> 当前只有hive connector的orc数据格式，才支持创建索引。
|
||||
|
||||
## 数据源
|
||||
|
||||
1. openLooKeng的SQL是否是大小写敏感的？如果数据源是大小写敏感的，openLooKeng如何处理？
|
||||
|
||||
> openLooKeng对大小写不敏感，对于sql中的大写，openLooKeng都会转换成小写处理。如果数据源本身是大小写敏感的，那么就可能查询失败。比如ES是大小写敏感的，所以如果ES中的列名使用大写，那么openLooKeng查询该列就会出错。
|
||||
|
||||
2. 对接hive使用update失败，报错：cannot change stats state for a transactional table，是什么原因？
|
||||
|
||||
> 此问题在开源的hive4上面会发生，当前版本的解决方法是设置参数hive.txn.stats.enabled=false
|
||||
|
||||
3. 对接hive使用update失败，报错：unpartitioned hive tables are immutable
|
||||
|
||||
> 检查hive.properties中的配置是否将hive.immutable-partitions设置成了true，如果是，需要修改为false
|
||||
|
||||
6. openLooKeng对接oracle10g，无法查询表名含下划线的表，如何解决？
|
||||
|
||||
> oracle10g是一个比较旧的oracle版本了，使用高版本的ojdbc存在不兼容的问题。尝试使用过多个ojdbc版本，发现openLooKeng使用ojdbc6可以正常查询oracle10g。
|
||||
|
||||
8. 对接oracle时，openLooKeng创建的表中的数据类型varchar，为什么在oracle中查询到的却是NCLOB？
|
||||
|
||||
> openLooKeng和oracle的数据类型存在一个映射关系，请查看openLooKeng官网上oracle连接器章节里的“openLooKeng到Oracle类型映射”。可以在openLooKeng中使用varchar(n)，这样在oracle中就是varchar2(n)
|
||||
|
||||
9. openLooKeng对接华为FusionInsight6.5.1的hive数据源，要如何配置？
|
||||
|
||||
>FusionInsight6.5.1的hive和开源apache hive有些差别，所以配置connector的时候，首先按照开源hive的配置方案配置，然后再做如下修改：
|
||||
>(1). hdfs-site.xml文件中的dfs.client.failover.proxy.provider.hacluster配置项设置为：org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider。
|
||||
>(2). 如果FusionInsight6.5.1的hive中数据采用的是txt格式，那么请将FusionInsight6.5.1安装目录下的hadoop-plugins-1.0.jar文件拷贝到openLooKeng安装目录的plugin/hive-hadoop2目录中，再重启openLooKeng。
|
||||
|
||||
## 安全
|
||||
|
||||
2. Ranger界面如何添加openLooKeng的服务组件？
|
||||
|
||||
> 可以通过curl命令进行Ranger服务类型定义的注册，举例：
|
||||
>
|
||||
> ```
|
||||
> curl -u admin:password -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d @service-defs/Ranger-servicedef-openlookeng.json http://Ranger-admin-host:port/service/plugins/definitions
|
||||
> ```
|
||||
>
|
||||
> 服务注册后，才能在Ranger的管理界面看到openLooKeng服务类型，然后就可以在管理界面添加openLooKeng服务了。
|
||||
>
|
||||
|
||||
3. Ranger中如何配置openLooKeng服务的参数？
|
||||
|
||||
> 参考如下：
|
||||
>
|
||||
> 
|
||||
|
||||
4. 对接Ranger后，openLooKeng日志告警：WARN PolicyRefresher(serviceName=openlookengdev)-196 org.apache.Ranger.admin.client.RangerAdminRESTClient Error getting Roles. secureMode=false, user=root (auth:SIMPLE), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=openlookengdev，这是什么原因？
|
||||
|
||||
> 这是RangerAdmin端版本过低的问题，openLooKeng提供的Ranger插件是对接Ranger2.1版本的，请升级Ranger版本。
|
||||
|
||||
5. openLooKeng的内部通信安全是必须配置的吗？
|
||||
|
||||
> 这个不是必须配置，最简单地将openLooKeng配置为非安全模式也是可以的。但是如果需要用到LDAP认证，这个ssl就是必须配置的，详情参考openLooKeng网站的LDAP配置一节。
|
||||
|
||||
6. openLooKeng有没有实现应用日志监控审计功能，需要二次开发吗？
|
||||
|
||||
> openLooKeng目前对sql的创建和完成是有审计功能的，如果需要审计更多的用户事件，例如webUI登录，可以二次开发来增加。
|
||||
|
||||
7. openLooKeng开启kerberos认证后再对接Ranger，RangerAdmin上的字段联想功能不可用怎么办？
|
||||
|
||||
> RangerAdmin的联想功能需要发送sql查询olk的相关资源，所以在RangerAdmin上添加openLooKeng服务时，jdbc.url参数需要配置kerberos信息，举例如下：
|
||||
>
|
||||
> ```
|
||||
> jdbc:lk://server1:10001/catalog?SSL=true&SSLKeyStorePath=/etc/openLooKeng_keystore.jks&SSLKeyStorePassword=keystore_password&KerberosRemoteServiceName=openLooKeng&KerberosConfigPath=/etc/krb5.conf&KerberosPrincipal=cpk@EXAMPLE.COM&KerberosKeytabPath=/etc/security/keytabs/cpk.keytab
|
||||
> ```
|
||||
|
||||
8. openLooKeng如何连接两个配置了kerberos的hive数据源？
|
||||
|
||||
> 首先保证两个配置了kerberos的hive数据源域名不相同；
|
||||
>
|
||||
> 然后合并两个数据源的krb5.conf文件，将其中一个的realm信息加入另一个krb5.conf文件即可；
|
||||
>
|
||||
> 将合并后的krb5.conf文件放在jvm.config中配置的路径下。举例如下：
|
||||
>
|
||||
> 
|
||||
|
||||
9. openLooKeng如何同时连接一个配置了kerberos的hive和一个没有配置kerberos的hive数据源？
|
||||
|
||||
> 在没有配置kerberos的hive数据源的core-site.xml文件中增加如下配置即可：
|
||||
>
|
||||
> ```
|
||||
> <property>
|
||||
> <name>ipc.client.fallback-to-simple-auth-allowed</name>
|
||||
> <value>true</value>
|
||||
> </property>
|
||||
> ```
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 386 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
|
|
@ -207,3 +207,5 @@ headless: true
|
|||
- [1.0.1]({{< relref "./docs/releasenotes/releasenotes-1.0.1.md" >}})
|
||||
- [1.0.0]({{< relref "./docs/releasenotes/releasenotes-1.0.0.md" >}})
|
||||
- [0.1.0]({{< relref "./docs/releasenotes/releasenotes-0.1.0.md" >}})
|
||||
|
||||
- [FAQ]({{< relref "./docs/faq/faq.md" >}})
|
||||
|
|
|
|||
Loading…
Reference in New Issue