Commit Graph

13 Commits

Author SHA1 Message Date
jonwe 353b3b7944 optimize mindrecord writer performance 2020-04-22 11:09:28 -04:00
panfengfeng b31946750b 回退 'Pull Request !182 : Tuning mindrecord writer performance' 2020-04-22 19:19:57 +08:00
mindspore-ci-bot c73301bad9 !182 Tuning mindrecord writer performance
Merge pull request !182 from jonathan_yan/write_a
2020-04-22 17:43:46 +08:00
jonwe f4bae5f364 optimize mindrecord writer performance 2020-04-21 09:12:13 -04:00
liyong f1542a90a3 add pk sampler 2020-04-21 10:08:38 +08:00
chenjianping 1286767d0e support building on windows 2020-04-15 09:36:48 +00:00
liyong 0ce83e39e1 fix TestShardSampleWrongNumber
adjust internal api
2020-04-13 21:31:45 +08:00
liyong 11403492ae add mindrecord subset random sampler 2020-04-11 18:48:23 +08:00
jonyguo a9443635b7 fix: mindpage enhance parameter check and search by filename failed 2020-04-11 10:19:14 +08:00
mindspore-ci-bot bd7ac259af !147 Parallelize mindrecord index writer via std::thread
Merge pull request !147 from ZiruiWu/master
2020-04-09 05:31:44 +08:00
Zirui Wu 5637f80692 implemented multi-thread index writer for mindrecord
num threads cannot be more than num shards

minor fix

clang style fix

address review comments
2020-04-08 10:03:19 -04:00
jonyguo 20d1b64443 fix: error info is not exactly when column list invalid 2020-04-08 16:11:13 +08:00
zhunaipan 930a1fb0a8 initial version
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
2020-03-27 22:54:54 +08:00