Commit Graph

166 Commits

Author SHA1 Message Date
青年 b385c45c35
chore:remove copyright in license header (#2090)
Co-authored-by: imbajin <jin@apache.org>
2023-01-13 23:26:52 +08:00
vaughn 85499f389b
fix: keep binary file (#2052) 2022-12-15 16:07:34 +08:00
imbajin 9cbb61862c
chore: prepare for apache v1.0.0 release (#2048)
* chore: apache release

* chore: update release script & remove redundant pom config
2022-12-14 17:52:01 +08:00
青年 df05b2b1b0
let sub-modules use revision (#2039) 2022-11-30 00:32:17 +08:00
青年 67a689c86c refact: adapt all to apache package and dependencies (Breaking Changes) 2022-11-23 21:16:14 +08:00
青年 8abf3d5524
chore: support check license header with RAT & fill the gaps (#1988) 2022-10-26 19:43:41 +08:00
Simon Cheung 767bdfc397
chore: fix GraphOfTheMoviesExample exception (#1953) 2022-10-08 19:31:36 +08:00
May cf767dc2d2
fix hugegraph-hbase and hugegraph-example checkstyle issue (#1864)
Co-authored-by: imbajin <17706099+imbajin@users.noreply.github.com>
2022-05-09 17:40:48 +08:00
May 5b48b3b4ba
fix hugegraph-example/Example checkstyle issue (#1838)
Co-authored-by: imbajin <17706099+imbajin@users.noreply.github.com>
2022-04-25 22:39:54 +08:00
Jermy Li 8d52abb2a0
HugeGraph-622: Bump up to version 0.13.0 (#1740) 2022-04-21 14:15:46 +08:00
Josh Soref 5368f3205e
Spelling corrections (#1798)
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-30 17:12:18 +08:00
Jermy Li 43b8ae7bff
Bfs optimize vertex-out-out query (#1359)
* not-commit: optimize-query-vertex-out-out-with-in-and
* fix cassandra IN query
* fix batch query missing limit and page info
* add feature supportsQueryWithInCondition()
* improve TableSerializer.writeQueryEdgeCondition()
* impprove flatten IN
* rename BatchQuery to BatchConditionQuery
* fix query from RamTable not flatten
* use BatchMapperIterator to fetch intermediate results(by batch avoid OOM)
* rename to HugeVertexStepByBatch and just enable when needed
* set batch list capacity
* set QUERY_BATCH = 100 for scylladb limit

Change-Id: I1b3c2e942cbfdc9f51cfc43a3ff5aa61b62e22d0
2021-11-03 19:08:59 +08:00
imbajin 23beacd2bc
refact: invalidate auth client cache & merge auth filter log (#1528)
* refact: invalidate auth client cache & merge auth filter log

1. merge auth filter log into audit.log
2. invalidate auth client role cache with CUD operation
2021-06-30 16:38:02 +08:00
imbajin f22d03fdd6
refact(auth): cache login role in auth client to avoid sending rpc repeatedly (#1507)
* refact(auth): cache login role in auth client to avoid sending rpc repeatedly
* fix: don't create Id every time
* perf: remove useless toLowerCase
* add config for expire time in auth client
* fix StandardAuthManager.inputPassword in auth client
* add option for auth max_capacity
* unified cache_capacity & expired time (also separate invalid cache for pwd)
* add option for token expiration

Co-authored-by: guoshoujing <guoshoujing@baidu.com>
2021-06-25 21:52:40 +08:00
imbajin fceaaa2fc0
feat(dist): support auto reload config & compress audit log (#1492)
Also enhance:
- add time strategy to avoid old logs overwriting by current logs
- auto create log folder if it does not exist
- add some comment for auth config

* use strict mode for config property & separate rest configs
* reduce log format length
* Update rest-server.properties
* update all configs (TODO: push to other modules)
2021-06-18 19:35:59 +08:00
Jermy Li bc9a5a85a4
Add compact api for rocksdb/cassandra/hbase backend (#1378)
* add compact api for rocksdb/cassandra/hbase backend
* increase and standardize some metrics
* increase metrics for cassandra/hbase and unify display format
* fix codecov missing some files coverage by jacococli
* add ScyllaDBMetrics
* make cassandra compact in the scope of keyspace

Change-Id: I26660b28b64089f3b636020e721659db3466267b
2021-06-11 15:57:20 +08:00
zhoney 23e3c31a54
Collection optimize for OLTP algorithm and HugeElement (#1409)
* add Eclipse Collection dependency
* use primitive map for properties of HugeElement
* from Map<Id, HugeProperty<?>> to MutableIntObjectMap<HugeProperty<?>>
* allow config oltp collections(JCF or EC)
* add fastutil choice for collection
* change Set<HugeEdge> of HugeVertex to List<HugeEdge>
* use IntObjectMap and IntIntMap to improve shortest path
* improve paths api by remove redundant sourcesAll and targetsAll
* improve paths api performance by using IntObjectHashMap<IntHashSet>
* extract shortestpath and paths record to hold traversed paths
* implement Int2IntsMap for array paths record
* extract Record and Records for paths and shortest path
* adapt primitive collection for kout/kneighbor
* add objectintmapping unit test
* add UT for CollectionFactory and IdSet

Change-Id: I501a18a70054fad3c40bd6b9e852a82a5c185949
2021-06-09 17:36:47 +08:00
imbajin 1ede97c7ab
refact: use batch flush & async way to output log (#1448)
* refact: use batch flush & async way to output log

More details refer to [doc](xx), here are some points:
1. change log4j2 appender  to `RandomAccessFile` to improve perf (default is BufferedOutputStream)
2. change log4j2 to not flush immediately  (default is true)
3. use bigger (512K) buffer size (default is 4k)
4. use (LMAX Disruptor) async way to improve log perf (default is sync)
5. separate audit log & server log to keep sever log clean & readable
6. use simple layout format for audit log
7. use `StringBuilder` to reduce about `9%` cost for audit log
8. record mysql connection url info for user issue

* format align & remove mysql log(todo: replace other log4j2.xml later)
2021-05-18 13:37:08 +08:00
Linary e05850b5d5
Upgrade version to 0.12.0 (#1372)
Change-Id: Ia7eab6e7c88816234204ec6e60fc807f25e71212
2021-03-26 18:00:34 +08:00
Jermy Li 314fd8a6c9
support: invalid cache through rpc (#1357)
* support: invalid cache through rpc
* add RpcServiceConfig* interfaces
* add FanoutCluster to broadcast reuqest
* fix GraphManager.close() is not called by HttpServer.DESTROY_FINISHED
* notify cache with Id[] arg to avoid too many events
* improve perf of raft notify-cache
* only enable cache-rpc service for shared storage
* check raft mode can only be enabled if non-shared store

Change-Id: I3fe18cd8c893a84a56fb66df651b78c5e0fe5f4b
2021-03-11 21:20:46 +08:00
zhoney f6c9850d5c
fix example bug (#1308)
Change-Id: I145e99de88aab45defe6cc00b960d945821edd1e
2020-12-23 22:05:09 +08:00
Linary 2df4f680d4
Update cache from raft StateMachine (#1119)
Change-Id: I743de35eb35409ebf4ff2507564cfe2c82c79fe9
2020-07-29 18:07:37 +08:00
Linary a25000e91c
Initially implement distributed consistency with jraft (#1020)
* Initially implement distributed consistency with jraft

* support save and load snapshot

* Let StoreCommand register and apply at StateMachine

* merge tx operation into one store command

* compress task data

* let follower can forward request to leader

* handle remote exception, let it report backend busy

* Check connection failed for address: xxx, maybe write overflow!

* add rate limit at StateMachine

* Let grizzly-http thread wait raft task applied

Change-Id: I5bdec682ec35daead67cbcf92477b0d761d31610
2020-07-28 10:43:31 +08:00
zhoney a0ea78249c
fix mysql execute() not always auto-commit (#1064)
* update root cause for not set autocommit if commit failed

Change-Id: Ib6749a8a5e003c312cc284795a7e37312755ead1
2020-07-01 14:55:22 +08:00
imbajin 81466fdecb
fix: change default minimum log level to TRACE (#1050) 2020-06-22 16:40:28 +08:00
Jermy Li 409bdbb83b
add grant permission api (#985)
* add grant permission api
* upgrade version to 0.11.1
* share usermanager

Change-Id: I58e6e5a8c950ad74fc1ccbcee8e8ca28d48a27e9
2020-05-11 15:33:41 +08:00
Jermy Li 26c4498463
fix permission overflow from return results (#969)
* fix permission overflow from return results

also:
 * hide private methods and fields for reflection that require permissions
 * delete stale index of task directly
 
add HugeGraph.java and TaskScheduler.java: separate commits to avoid no git diff

implement: #209
Change-Id: Iccfbd47c213e1e13122190d1242778031e8a957c
2020-04-24 15:42:43 +08:00
Linary 1fe09d0dd2
Increase task result limit (#925)
* Increase task result limit
* Let task input and result property datatype as blob
* Support config task input and result size limit

Change-Id: Ib2c4bdf25e4604d8bea47802e724c0f0934170fb
2020-04-06 19:26:23 +08:00
Jermy Li b38ca33a81
check for unexpected sysprop query (#862)
fix: #861

* workaround: cassandra column with null value can't be queryed with CONTAINS

* string2HugeKey() disallow pass syspprop from gremlin

Change-Id: I319703286f65fd38819b2de91d7c9a18203ceafd
2020-03-30 11:27:57 +08:00
Jermy Li dee579d989
fix task-cancel error with saving failure task (#827)
Change-Id: Ib61c8cdae2e73d346dd611791d47faa30eac6e5a
2020-02-24 09:27:15 +08:00
Linary dbf5eb53cb Upgrade version to 0.11.0 (#739)
Change-Id: I565dbbe34f746ca5d7fed0f29bdd6e0bf46b4675
2019-11-11 14:24:13 +08:00
Linary c19fbea2de Unified license related files name (#733)
Change-Id: I0f9fcc0120c33408765485a8aab7c60b876a27c0
2019-11-05 14:07:42 +08:00
Jermy Li 95eba359e9
support paging for tasks API (#720)
Change-Id: Ie036d433ef74d7c6724bf869425c3fd528f08941
related: #718
2019-11-04 11:26:43 +08:00
imbajin 9d8ceb4dfc Optimize update logic & add a new strategy (#673)
1. if original data is not null but new data is null, use original data instead of null
2. add update strategy "OVERRIDE"
2019-09-24 20:26:07 +08:00
Linary 1c062b8947 Upgrade tinkerpop version to 3.4.3 (#648)
Change-Id: I5443b6a05eea74205a373f093dad27536e04f0be
2019-09-12 19:04:30 +08:00
Linary 94c2d34ce8 Support to verify license (#645)
Change-Id: Ia3fb6b38a032442e1d8bf7cee4d7b4032dc219df
2019-09-09 20:58:41 +08:00
Jermy Li 26e48a27f1 encode numeric values in secondary index and primary-key (#676)
Change-Id: I6acfad358745a54284f80d236243bfbef61d2deb
2019-09-02 17:15:24 +08:00
Jermy Li 97090e65df fix example1 query error (#638)
Change-Id: I0410c64be5a2ccec2f81a94e6620517a87cf2d7d
2019-08-14 20:43:18 +08:00
zhoney 0e66c64fec support shard-index and vertex + sortkey prefix query (#574)
implemented: #270 and #398

Change-Id: I444b67b7f57a18fd32c29d54d776cf411ccd847a
2019-08-06 11:16:52 +08:00
Linary b01549a579 Not allowed perform sensitive operations via gremlin (#176)
Implement #145

Change-Id: I9a590fe40d3b5a808b569ed0af8fd83214a2941a
2019-07-29 14:21:40 +08:00
Linary abebfac25c Add api white list and rate limiter for gc (#522)
Also add ProfileAPI
Change-Id: Ia7ac5dc5d7d91676701738f2a0ae5073f9ceb01e
2019-05-31 17:06:40 +08:00
Linary 9db6d6a0ef Uniform variable name: vertexes to vertices (#529)
Change-Id: I05c0f52438e3ab74360b396d1c3a278d885a5426
2019-05-27 14:49:40 +08:00
Linary bed8a6ea03 Bump up to version 0.10.0 (#471)
Change-Id: I49314313781c73436e3370d2a6334adfbc142d96
2019-05-06 22:43:01 -05:00
Linary d8d8fe2fa1 Improve some comments (#472)
Change-Id: I237f3e4175eec538aff01c521ec6e93cc8c2fc87
2019-04-19 18:37:05 +08:00
Linary 2890c96e67 Add personalrank and neighborrank RESTful API (#274)
Change-Id: I66e4224b5813aa2d5eadbfe540fb9de203db766b
2019-04-18 22:14:27 -05:00
Jermy Li 727a0e4947 Uniform variable name: Iterator iter (#438)
Change-Id: Iabf708702778a8ecd5eb19bdcf012c4133edff8f
2019-04-09 18:45:17 +08:00
Linary c50c34a904 Support pagination when single index query (#328)
Support pagination when single index query

Change-Id: Iab355a9f68cebd8ebc666cf126842eb19fa40240
2019-03-27 16:22:34 +08:00
liningrui 0e6359477c Some small modification
Change-Id: I80ff7c0778e52b4ca939f42c7a4e111da5b09406
2019-01-14 15:41:39 +08:00
Zhangmei Li 1885132b5b fix bug: perf example cost time should exclude time of env-init
Change-Id: Iba9964542dc43c8c4d4feb0bb6a2883309e9326a
2019-01-11 17:13:28 +08:00
Zhangmei Li 7164a459ba Support range sortKey feature
implement #271

Change-Id: Ib618c45bc6169a0fb7632dbbf0fb29b7ee0a08e3
2018-12-29 14:15:54 +08:00