add tcp-ip for HA deployment documentation

hetu-docs/en/installation/deployment-ha.md

@@ -4,32 +4,108 @@
 The openLooKeng HA solves single point failure of coordinators. Users can submit queries to any coordinator to balance the workloads.
 
 ## Installing HA
-openLooKeng with HA is required to be installed with minimum of 2 coordinators in the cluster.
+openLooKeng with HA is required to be installed with minimum of 2 coordinators in the cluster. Make sure the time on all coordinators have been sync up.
 Please follow [Deploying openLooKeng Manually](./deployment.md) or [Deploying openLooKeng Automatically](./deployment-auto.md) for basic installation.
 
 ## Configuring HA
+- There are 2 mechanisms of configuring HA
+    1. TCP-IP : Coordinators discover each other based on their seeds (ie. coordinator's IP address).
+    2. Multicast : Coordinators discover each other under the same network.
 
-- Create an `etc\state-store.properties` file inside both coordinators and workers installation directories.
+- Multicast mechanism is not recommended for production since UDP is often blocked in production environments.
 
+### Configuring HA with TCP-IP
 
-### State Store Properties
+#### State Store Properties
 
-The state store properties file, `etc/state-store.properties`, contains configuration to state store.
+Create an `etc\state-store.properties` property file inside both coordinators and workers installation directories.
-State store is used to store states that are shared between coordinators and workers. The following is a minimal configuration.
+
+State store is used to store states that are shared between coordinators and workers.
 
 ``` properties
 state-store.type=hazelcast
 state-store.name=query
 state-store.cluster=cluster1
+
+hazelcast.discovery.mode=tcp-ip   
+hazelcast.discovery.port=5701       #optional, the default port is 5701
+```
+
+The above properties are described below:
+- `state-store.type`: The type of the state store. For now, only support hazelcast.
+- `state-store.name`: User defined name of state store.
+- `state-store.cluster`: User defined cluster name of state store.
+- `hazelcast.discovery.mode` : The discovery mode of hazelcast state store, now support tcp-ip and multicast(default).  
+- `hazelcast.discovery.port` : The user defined port of hazelcast state store to launch. 
+
+#### Seed Store Properties
+Create an `etc\seed-store.properties` property file inside both coordinators and workers installation directories.
+
+Seed store is used to store seeds of coordinators to discover each other.
+
+```
+seed-store.type=filebased
+seed-store.filesystem.seed-dir=/tmp/openlookeng/ha
+seed-store.filesystem.profile=hdfs-config-default
 ```
 
 The above properties are described below:
 
-- `state-store.type`: The type of the state store. For now, only support hazelcast
+- `seed-store.type`: The type of the seed store. For now, only support filebased.
-- `state-store.name`: User defined name of state store
+- `seed-store.filesystem.seed-dir`: The directory to store seeds.
-- `state-store.cluster`: User defined cluster name of state store
+- `seed-store.filesystem.profile.`: The name of filesystem configuration file under `etc\filesystem` directory.
 
-### Coordinator and Worker Properties
+#### Filesystem Properties
+
+Create an `etc\filesystem\hdfs-config-default.properties` property file inside both coordinators and workers installation directories.
+
+Filesystem is required to be distributed filesystem so that all coordinators and workers can access  (ie. HDFS).
+```
+fs.client.type=hdfs
+hdfs.config.resources=/path/to/core-site.xml,/path/to/hdfs-site.xml
+hdfs.authentication.type=NONE
+fs.hdfs.impl.disable.cache=true
+```
+The above properties are described in [Filesystem Access Utilities](../develop/filesystem.md).
+
+#### Coordinator and Worker Properties
+
+Add following configuration to `etc/config.properties` on all coordinators.
+
+``` properties
+hetu.multiple-coordinator.enabled=true
+hetu.embedded-state-store.enabled=true
+hetu.seed-store.enabled=true
+```
+
+Add following configuration to ``etc/config.properties`` on all workers.
+
+``` properties
+hetu.multiple-coordinator.enabled=true
+hetu.seed-store.enabled=true
+```
+
+The above properties are described below:
+
+-  `hetu.multiple-coordinator.enabled`: Multiple coordinator mode is enabled.
+-  `hetu.embedded-state-store.enabled`: Allow coordinators to use embedded state store.
+-  `hetu.seed-store.enabled`: Allow coordinator and worker to use seed store.
+
+### Configuring HA with Multicast
+
+#### State Store Properties
+
+Create an `etc\state-store.properties` file inside both coordinators and workers installation directories.
+``` properties
+state-store.type=hazelcast
+state-store.name=query
+state-store.cluster=cluster1
+
+hazelcast.discovery.mode=multicast  #optional, the default is multicast 
+hazelcast.discovery.port=5701       #optional, the default port is 5701
+```
+
+#### Coordinator and Worker Properties
 
 Add following configuration to `etc/config.properties` on all coordinators.
 
@@ -44,11 +120,6 @@ Add following configuration to ``etc/config.properties`` on all workers.
 hetu.multiple-coordinator.enabled=true
 ```
 
-The above properties are described below:
-
--  `hetu.multiple-coordinator.enabled`: Multiple coordinator mode is enabled
--  `hetu.embedded-state-store.enabled`: Allow this coordinator to use embedded state store
-
 ## HA Cluster behind Reverse Proxy
 
 To achieve the full benefit of HA, clients(ie. openLooKeng CLI, JDBC, etc) are encouraged to _not_ connect to specific coordinators directly. Instead, they should connect to multiple coordinators through some sort of _reverse proxy_, for example through a load balancer, or a Kubernetes service. This allows the client to continue to work even if a specific coordinator is not working as expected.
@@ -59,7 +130,7 @@ When the connection is made through a reverse proxy, it is required for a given
 
 ### Configure Reverse Proxy Example (Nginx)
 
-Please include configuration below in the Nginx configuration file (ie. `nginx.conf`)
+Please include configuration below in the Nginx configuration file (ie. `nginx.conf`).
 
 ```
 http {

hetu-docs/zh/installation/deployment-ha.md

@@ -5,20 +5,28 @@ openLooKeng HA解决协调节点单点故障问题。用户可以在任何协调
 
 ## 安装HA
 
-安装具有HA的openLooKeng要求集群中至少有2个协调节点。请按照[手动部署openLooKeng](deployment.md)或[自动部署openLooKeng](deployment-auto.md)进行基本设置。
+安装具有HA的openLooKeng要求集群中至少有2个协调节点。请确保协调节点上的时间一致性。请按照[手动部署openLooKeng](deployment.md)或[自动部署openLooKeng](deployment-auto.md)进行基本设置。
 
 ## 配置HA
+- 目前有2种方法配置HA
+    1. TCP-IP : 协调节点根据种子（即协调节点ip地址）相互发现。
+    2. Multicast : 协调节点在同一网络下相互发现。
+- 不建议在生产环境中使用Multicast机制，因为UDP通常在生产环境中被阻止。
 
-- 在协调节点和工作节点安装目录中创建文件`etc\state-store.properties`。
+### TCP-IP方法配置HA
 
-### 状态存储属性
+#### 状态存储属性
+在协调节点和工作节点安装目录中创建文件`etc\state-store.properties`。
 
-状态存储属性文件`etc/state-store.properties`包含状态存储的配置。状态存储用于与协调节点和工作节点共享的存储状态。以下是最基本配置。
+状态存储用于与协调节点和工作节点共享的存储状态。
 
 ``` properties
 state-store.type=hazelcast
 state-store.name=query
 state-store.cluster=cluster1
+
+hazelcast.discovery.mode=tcp-ip   
+hazelcast.discovery.port=5701       #可选，默认端口为5701
 ```
 
 上述属性说明如下：
@@ -26,8 +34,76 @@ state-store.cluster=cluster1
 - `state-store.type`：状态存储的类型。目前仅支持Hazelcast。
 - `state-store.name`：用户定义的状态存储名称。
 - `state-store.cluster`：用户定义的状态存储集群名称。
+- `hazelcast.discovery.mode` : Hazelcast状态存储发现模式, 目前支持tcp-ip和multicast（默认）。  
+- `hazelcast.discovery.port` : 用户定义hazelcast启动端口。
 
-### 协调节点和工作节点属性
+#### 种子存储属性
+在协调节点和工作节点安装目录中创建文件`etc\seed-store.properties`。
+
+种子存储用于存储协调节点的种子以相互发现。
+
+```
+seed-store.type=filebased
+seed-store.filesystem.seed-dir=/tmp/openlookeng/ha
+seed-store.filesystem.profile=hdfs-config-default
+```
+
+上述属性说明如下：
+
+- `seed-store.type`: 种子存储的类型。目前仅支持Filebased。
+- `seed-store.filesystem.seed-dir`: 存储种子的目录。
+- `seed-store.filesystem.profile.`: `etc\filesystem`目录下的文件系统中配置文件的名称。
+
+
+#### 文件系统属性
+
+在协调节点和工作节点安装目录中创建文件`etc\filesystem\hdfs-config-default.properties`。
+
+文件系统必须是分布式文件系统，以便所有协调节点和工作节点都可以访问（例如HDFS）。
+```
+fs.client.type=hdfs
+hdfs.config.resources=/path/to/core-site.xml,/path/to/hdfs-site.xml
+hdfs.authentication.type=NONE
+fs.hdfs.impl.disable.cache=true
+```
+上述属性说明请参考[文件系统访问实用程序](../develop/filesystem.md)。 
+
+#### 协调节点和工作节点属性
+
+在所有协调节点上的`etc/config.properties`中添加如下配置。
+
+``` properties
+hetu.multiple-coordinator.enabled=true
+hetu.embedded-state-store.enabled=true
+hetu.seed-store.enabled=true
+```
+
+在所有工作节点上的`etc/config.properties`中添加如下配置。
+
+``` properties
+hetu.multiple-coordinator.enabled=true
+hetu.seed-store.enabled=true
+```
+
+上述属性说明如下：
+
+- `hetu.multiple-coordinator.enabled`：开启了多协调节点模式。
+- `hetu.embedded-state-store.enabled`：允许协调节点使用嵌入式状态存储。
+- `hetu.seed-store.enabled`: 允许协调节点和工作节点使用嵌入式状态存储。
+
+### Multicast方法配置HA
+
+#### 状态存储属性
+
+状态存储属性文件`etc/state-store.properties`包含状态存储的配置。
+
+``` properties
+state-store.type=hazelcast
+state-store.name=query
+state-store.cluster=cluster1
+```
+
+#### 协调节点和工作节点属性
 
 在所有协调节点上的`etc/config.properties`中添加如下配置。
 
@@ -42,11 +118,6 @@ hetu.embedded-state-store.enabled=true
 hetu.multiple-coordinator.enabled=true
 ```
 
-上述属性说明如下：
-
-- `hetu.multiple-coordinator.enabled`：开启了多协调节点模式。
-- `hetu.embedded-state-store.enabled`：允许此协调节点使用嵌入式状态存储。
-
 ## HA及反向代理
 
 要尽可能的显示HA的优势，建议客户端(例如openLooKeng CLI, JDBC等)不直接连接特定的协调节点，而是通过反向代理连接多个协调节点，例如使用负载平衡器或者Kubernetes服务，这样即使某个协调节点无法正常工作，客户端也可以继续使用其它的协调节点。
@@ -57,7 +128,7 @@ hetu.multiple-coordinator.enabled=true
 
 ### 配置反向代理 (Nginx)
 
-请在Nginx配置文件中包含以下配置（nginx.conf）
+请在Nginx配置文件中包含以下配置（nginx.conf）。
 
 ```
 http {