11 Commits

Author	SHA1	Message	Date
yanghaitao	1cb3bead32	skip black line for TextFileDataset	2020-04-27 14:54:26 +08:00
qianlong	db80f4ff92	The num_samples and numRows in schema for TFRecordDataset are conflict	2020-04-21 17:08:59 +08:00
liyong	f1542a90a3	add pk sampler	2020-04-21 10:08:38 +08:00
mindspore-ci-bot	6369cf27bd	!406 added first row crc check for when reading tfrecord files ... Merge pull request !406 from Peilin/first-row-crc-check	2020-04-21 00:10:18 +08:00
xulei2020	c705ea5e5b	add filterOp code	2020-04-20 22:32:19 +08:00
Peilin Wang	9bc2134cb7	added checking of first row crc to find invalid tfrecord files ... addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file fixed clang format fixed cppcheck fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments more clang whitespace fixes... changed print to logger.info	2020-04-20 10:25:26 -04:00
yanghaitao	2795e492ff	TextFileDataset	2020-04-20 11:18:55 +08:00
Cathy Wong	59a714c654	Correct shuffle UT buffer_size > #dataset-row as valid	2020-04-09 15:22:33 -04:00
yanzhenxiang2020	5a4f17bfb6	fix mindrecord ut long time	2020-04-02 19:18:49 +08:00
Cathy Wong	2e881276ab	Enable skipped dataset zip python unit tests ... Signed-off-by: Cathy Wong <cathy.wong@huawei.com>	2020-03-31 16:47:13 -04:00
zhunaipan	930a1fb0a8	initial version ... Signed-off-by: leonwanghui <leon.wanghui@huawei.com>	2020-03-27 22:54:54 +08:00