forked from huawei/mindspore2022
!30329 update chinese api doc
Merge pull request !30329 from luoyang/code_docs_chinese
This commit is contained in:
Gitee
commit
db7662fa5e
|
|
@ -0,0 +1,15 @@
|
|||
mindspore.dataset.BatchInfo
|
||||
===========================
|
||||
|
||||
.. py:class:: mindspore.dataset.BatchInfo
|
||||
|
||||
此类提供了两种方法获取数据集的批处理数量（batch size）和迭代数（epoch），
|
||||
用于 `batch` 操作中的输入参数 `batch_size` 和 `per_batch_map`。
|
||||
|
||||
.. py:method:: get_batch_num()
|
||||
|
||||
返回数据集的批处理数量（batch size）。
|
||||
|
||||
.. py:method:: get_epoch_num()
|
||||
|
||||
返回数据集的迭代数（epoch）。
|
||||
|
|
@ -34,7 +34,7 @@ mindspore.dataset.Caltech101Dataset
|
|||
- **ValueError** - `target_type` 参数取值不为'category'、'annotation'或'all'。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数值错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ mindspore.dataset.Caltech256Dataset
|
|||
- **ValueError** - `target_type` 参数取值不为'category'、'annotation'或'all'。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数值错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ mindspore.dataset.CelebADataset
|
|||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **ValueError** - `usage` 参数取值不为'train'、'valid'、'test'或'all'。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数值错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ mindspore.dataset.Cifar100Dataset
|
|||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **ValueError** - `usage` 参数取值不为'train'、'test'或'all'。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数错误（小于0或者大于等于 `num_shards`）。
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ mindspore.dataset.Cifar10Dataset
|
|||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **ValueError** - `usage` 参数取值不为'train'、'test'或'all'。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ mindspore.dataset.CityscapesDataset
|
|||
- **RuntimeError** - `dataset_dir` 路径下不包含任何数据文件。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `dataset_dir` 路径非法或不存在。
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@
|
|||
|
||||
- **RuntimeError** - `dataset_dir` 路径下不包含任何数据文件。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **RuntimeError** - 解析 `annotation_file` 指定的JSON文件失败。
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ mindspore.dataset.DIV2KDataset
|
|||
|
||||
读取和解析DIV2K数据集的源文件构建数据集。
|
||||
|
||||
生成的数据集有两列 `[hr_image, lr_image]` 。'hr_image` 列和 `lr_image` 列的数据类型都为uint8。
|
||||
生成的数据集有两列 `[hr_image, lr_image]` 。 `hr_image` 列和 `lr_image` 列的数据类型都为uint8。
|
||||
|
||||
**参数：**
|
||||
|
||||
|
|
@ -28,13 +28,15 @@ mindspore.dataset.DIV2KDataset
|
|||
- **RuntimeError** - `dataset_dir` 路径下不包含任何数据文件。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `dataset_dir` 路径非法或不存在。
|
||||
- **ValueError** - `usage` 参数取值不为'train'、'valid'或'all'。
|
||||
- **ValueError** - `downgrade` 参数取值不为'bicubic'、'unknown'、'mild'、'difficult'或'wild'。
|
||||
- **ValueError** - `scale` 参数取值不在给定的字段中，或与 `downgrade` 参数的值不匹配。
|
||||
- **ValueError** - `scale` 参数取值为8，但 `downgrade` 参数的值不为 'bicubic'。
|
||||
- **ValueError** - `downgrade` 参数取值为'mild'、'difficult'或'wild'，但 `scale` 参数的值不为4。
|
||||
- **ValueError** - `shard_id` 参数值错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@
|
|||
- **RuntimeError** - `column_names` 参数指定的列名数量与 `source` 参数输出的数据数量不匹配。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过最大线程数。
|
||||
- **ValueError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **ValueError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **ValueError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **ValueError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **ValueError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数值错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ mindspore.dataset.ImageFolderDataset
|
|||
- **RuntimeError** - `dataset_dir` 不包含任何数据文件。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **RuntimeError** - `class_indexing` 参数的类型不是dict。
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@
|
|||
- **RuntimeError** - `dataset_files` 路径下不包含任何数据文件。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **RuntimeError** - `class_indexing` 参数的类型不是dict。
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ mindspore.dataset.MnistDataset
|
|||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **ValueError** - `usage` 参数取值不为'train'、'test'或'all'。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ mindspore.dataset.NumpySlicesDataset
|
|||
- **RuntimeError** - `column_names` 列表的长度与数据的输出列表长度不匹配。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
- **ValueError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **ValueError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **ValueError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **ValueError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **ValueError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `shard_id` 参数值错误（小于0或者大于等于 `num_shards` ）。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
mindspore.dataset.OutputFormat
|
||||
==============================
|
||||
|
||||
.. py:class:: mindspore.dataset.OutputFormat
|
||||
|
||||
通过API `get_all_neighbors` 获取所有相邻节点时，指定节点的存储格式。
|
||||
|
||||
- **OutputFormat.NORMAL** - 普通格式，不进行压缩。
|
||||
- **OutputFormat.COO** - 采用COO（Coordinate）格式进行存储。
|
||||
- **OutputFormat.CSR** - 采用CSR（Compressed Sparse Row）格式进行存储。
|
||||
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
mindspore.dataset.SamplingStrategy
|
||||
==================================
|
||||
|
||||
.. py:class:: mindspore.dataset.SamplingStrategy
|
||||
|
||||
指定图数据采样策略的枚举类。
|
||||
|
||||
- **SamplingStrategy.RANDOM** - 随机抽样，带放回采样。
|
||||
- **SamplingStrategy.EDGE_WEIGHT** - 以边缘的权重为概率进行采样。
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
mindspore.dataset.Shuffle
|
||||
=========================
|
||||
|
||||
.. py:class:: mindspore.dataset.Shuffle
|
||||
|
||||
指定混洗模式的枚举类。
|
||||
|
||||
- **Shuffle.GLOBAL** - 混洗文件和文件中的数据。
|
||||
- **Shuffle.FILES** - 仅混洗文件。
|
||||
- **Shuffle.INFILE** - 保持读入文件的序列，仅混洗每个文件中的数据。
|
||||
|
|
@ -37,7 +37,7 @@ mindspore.dataset.VOCDataset
|
|||
- **RuntimeError** - 读取的xml文件缺失 `object` 属性。
|
||||
- **RuntimeError** - 读取的xml文件缺失 `bndbox` 属性。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `shuffle` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数。
|
||||
- **RuntimeError** - 同时指定了 `sampler` 和 `num_shards` 参数或同时指定了 `sampler` 和 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `num_shards` 参数，但是未指定 `shard_id` 参数。
|
||||
- **RuntimeError** - 指定了 `shard_id` 参数，但是未指定 `num_shards` 参数。
|
||||
- **ValueError** - `num_parallel_workers` 参数超过系统最大线程数。
|
||||
|
|
|
|||
|
|
@ -28,29 +28,29 @@ mindspore.dataset.text.transforms.JiebaTokenizer
|
|||
- **TypeError** - 参数 `hmm_path` 和 `mp_path` 类型不为string。
|
||||
- **TypeError** - 参数 `with_offsets` 类型不为bool。
|
||||
|
||||
.. py:method:: add_word(self, word, freq=None)
|
||||
.. py:method:: add_word(self, word, freq=None)
|
||||
|
||||
将用户定义的词添加到 JiebaTokenizer 的字典中。
|
||||
将用户定义的词添加到 JiebaTokenizer 的字典中。
|
||||
|
||||
**参数：**
|
||||
**参数：**
|
||||
|
||||
- **word** (str) - 要添加到 JiebaTokenizer 词典中的单词，注意通过此接口添加的单词不会被写入本地的模型文件中。
|
||||
- **freq** (int，可选) - 要添加的单词的频率。频率越高，单词被分词的机会越大。默认值：None，使用默认频率。
|
||||
- **word** (str) - 要添加到 JiebaTokenizer 词典中的单词，注意通过此接口添加的单词不会被写入本地的模型文件中。
|
||||
- **freq** (int，可选) - 要添加的单词的频率。频率越高，单词被分词的机会越大。默认值：None，使用默认频率。
|
||||
|
||||
.. py:method:: add_dict(self, user_dict)
|
||||
.. py:method:: add_dict(self, user_dict)
|
||||
|
||||
将用户定义的词添加到 JiebaTokenizer 的字典中。
|
||||
将用户定义的词添加到 JiebaTokenizer 的字典中。
|
||||
|
||||
**参数：**
|
||||
**参数：**
|
||||
|
||||
- **user_dict** (Union[str, dict]) - 有两种输入方式。可以通过指定jieba字典格式的文件路径加载。
|
||||
要求的jieba字典格式为：[word，freq]，如：
|
||||
- **user_dict** (Union[str, dict]) - 有两种输入方式。可以通过指定jieba字典格式的文件路径加载。
|
||||
要求的jieba字典格式为：[word，freq]，如：
|
||||
|
||||
.. code-block::
|
||||
.. code-block::
|
||||
|
||||
word1 freq1
|
||||
word2 None
|
||||
word3 freq3
|
||||
word1 freq1
|
||||
word2 None
|
||||
word3 freq3
|
||||
|
||||
也可以通过Python dict加载，要求的 Python 字典格式为：{word1:freq1, word2:freq2,...}。
|
||||
只有用户提供的文件中有效的词对才会被添加到字典中，无的效输入行将被忽略，且不返回错误或警告状态。
|
||||
也可以通过Python dict加载，要求的 Python 字典格式为：{word1:freq1, word2:freq2,...}。
|
||||
只有用户提供的文件中有效的词对才会被添加到字典中，无的效输入行将被忽略，且不返回错误或警告状态。
|
||||
|
|
|
|||
|
|
@ -16,5 +16,5 @@ mindspore.dataset.text.transforms.SentencePieceTokenizer
|
|||
|
||||
**异常：**
|
||||
|
||||
- **TypeError** - 参数 `mode` 的类型不是string或SentencePieceVocab。
|
||||
- **TypeError** - 参数 `out_type` 的类型不是SPieceTokenizerOutType。
|
||||
- **TypeError** - 参数 `mode` 的类型不是string或 :class:`mindspore.dataset.text.SentencePieceVocab`。
|
||||
- **TypeError** - 参数 `out_type` 的类型不是 :class::`mindspore.dataset.text.SPieceTokenizerOutType`。
|
||||
|
|
|
|||
|
|
@ -21,4 +21,4 @@ mindspore.dataset.text.transforms.WordpieceTokenizer
|
|||
- **TypeError** - 参数 `max_bytes_per_token` 的类型不为int。
|
||||
- **TypeError** - 参数 `unknown_token` 的类型不为string。
|
||||
- **TypeError** - 参数 `with_offsets` 的类型不为bool。
|
||||
- **TypeError** - 参数 `max_bytes_per_token` 的值为非负数。
|
||||
- **ValueError** - 参数 `max_bytes_per_token` 的值为负数。
|
||||
|
|
|
|||
|
|
@ -7,8 +7,9 @@ mindspore.dataset.transforms.c_transforms.Mask
|
|||
|
||||
**参数：**
|
||||
|
||||
- **operator** (Relational) - 关系操作符，可以取值为 Relational.EQ、Relational.NE、Relational.LT、
|
||||
Relational.GT、Relational.LE、Relational.GE。以Relational.EQ为例，将找出Tensor中与 `constant` 相等的元素。
|
||||
- **operator** (:class:`mindspore.dataset.transforms.c_transforms.Relational`) - 关系操作符，可以取值为
|
||||
Relational.EQ、Relational.NE、Relational.LT、 Relational.GT、Relational.LE、Relational.GE。
|
||||
以Relational.EQ为例，将找出Tensor中与 `constant` 相等的元素。
|
||||
- **constant** (Union[str, int, float, bool]) - 要进行比较的基准值。
|
||||
- **dtype** (mindspore.dtype, 可选) - 生成的掩码Tensor的数据类型，默认值：mstype.bool。
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,10 @@ mindspore.dataset.transforms.py_transforms.Compose
|
|||
|
||||
将多个数据增强算子组合使用。
|
||||
|
||||
.. note:: Compose可以将 `mindspore.dataset.transforms.py_transforms` 模块中的数据增强算子以及用户自定义的Python可调用对象
|
||||
合并成单个数据增强。对于用户定义的Python可调用对象，要求其返回值是numpy.ndarray类型。有关如何使用，请参阅Compose的示例，或阅读
|
||||
:class:`mindspore.dataset.vision.py_transforms.FiveCrop` 的示例，学习如何与用户自定义Python可调用对象配合使用。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **transforms** (list) - 一个数据增强的列表。
|
||||
|
|
@ -14,3 +18,11 @@ mindspore.dataset.transforms.py_transforms.Compose
|
|||
- **TypeError** - 参数 `transforms` 类型不为 list。
|
||||
- **ValueError** - 参数 `transforms` 的长度为空。
|
||||
- **TypeError** - 参数 `transforms` 的元素不是Python的callable对象。
|
||||
|
||||
.. py:method:: reduce(operations)
|
||||
|
||||
静态方法，可以调用这个函数将多个数据增强算子组合起来。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **operations** (list) - 数据增强的列表。
|
||||
|
|
@ -8,7 +8,7 @@ mindspore.dataset.transforms.py_transforms.OneHotOp
|
|||
**参数：**
|
||||
|
||||
- **num_classes** (int) - 数据集的类别数，它应该大于数据集中最大的label编号。
|
||||
- **num_classes** (float，可选) - 标签平滑的系数，默认值：0.0。
|
||||
- **smoothing_rate** (float，可选) - 标签平滑的系数，默认值：0.0。
|
||||
|
||||
**异常：**
|
||||
|
||||
|
|
|
|||
|
|
@ -13,5 +13,5 @@ mindspore.dataset.vision.c_transforms.RandomPosterize
|
|||
**异常：**
|
||||
|
||||
- **TypeError** - 如果 `bits` 不是int或整数序列类型。
|
||||
- **ValueErro：** - 如果 `bits` 不在 [1, 8] 范围内。
|
||||
- **ValueError** - 如果 `bits` 不在 [1, 8] 范围内。
|
||||
- **RuntimeError** - 如果输入图像的shape不是 <H, W> 或 <H, W, C>。
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
mindspore.dataset.vision.py_transforms.Pad
|
||||
==========================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.Pad(padding, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.Pad(padding, fill_value=0, padding_mode=Border.CONSTANT)
|
||||
|
||||
对输入PIL图像的各边进行填充。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
mindspore.dataset.vision.py_transforms.RandomAffine
|
||||
===================================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomAffine(degrees, translate=None, scale=None, shear=None, resample=<Inter.NEAREST: 0>, fill_value=0)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomAffine(degrees, translate=None, scale=None, shear=None, resample=Inter.NEAREST, fill_value=0)
|
||||
|
||||
对输入PIL图像进行随机仿射变换。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
mindspore.dataset.vision.py_transforms.RandomCrop
|
||||
=================================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomCrop(size, padding=None, pad_if_needed=False, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomCrop(size, padding=None, pad_if_needed=False, fill_value=0, padding_mode=Border.CONSTANT)
|
||||
|
||||
在输入PIL图像上的随机位置，裁剪指定大小的子图。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
mindspore.dataset.vision.py_transforms.RandomPerspective
|
||||
========================================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomPerspective(distortion_scale=0.5, prob=0.5, interpolation=<Inter.BICUBIC: 3>)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomPerspective(distortion_scale=0.5, prob=0.5, interpolation=Inter.BICUBIC)
|
||||
|
||||
按照指定的概率对输入PIL图像进行透视变换。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
mindspore.dataset.vision.py_transforms.RandomResizedCrop
|
||||
========================================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomResizedCrop(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomResizedCrop(size, scale=(0.08, 1.0), ratio=(3. / 4., 4. / 3.), interpolation=Inter.BILINEAR, max_attempts=10)
|
||||
|
||||
在输入PIL图像上的随机位置裁剪子图，并放缩到指定大小。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,12 @@
|
|||
mindspore.dataset.vision.py_transforms.RandomRotation
|
||||
=====================================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomRotation(degrees, resample=<Inter.NEAREST: 0>, expand=False, center=None, fill_value=0)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.RandomRotation(degrees, resample=Inter.NEAREST, expand=False, center=None, fill_value=0)
|
||||
|
||||
将输入PIL图像旋转随机角度。
|
||||
|
||||
.. note:: 参阅Pillow的 `rotate <https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.rotate>`_ 功能以了解更多用法。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **degrees** (Union[int, float, sequence]) - 旋转角度的随机选取范围。若输入单个数字，则从(-degrees, degrees)中随机生成旋转角度；若输入2元素序列，需按(min, max)顺序排列。
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
mindspore.dataset.vision.py_transforms.Resize
|
||||
=============================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.Resize(size, interpolation=<Inter.BILINEAR: 2>)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.Resize(size, interpolation=Inter.BILINEAR)
|
||||
|
||||
将输入PIL图像放缩为指定大小。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,12 @@
|
|||
mindspore.dataset.vision.py_transforms.ToTensor
|
||||
===============================================
|
||||
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.ToTensor(output_type=<class 'numpy.float32'>)
|
||||
.. py:class:: mindspore.dataset.vision.py_transforms.ToTensor(output_type=numpy.float32)
|
||||
|
||||
将输入的PIL或numpy.ndarray图像转换为指定数据类型的numpy.ndarray图像，此时像素值取值将由[0, 255]变为[0.0, 1.0]，图像的shape将由(H, W, C)变为(C, H, W)。
|
||||
|
||||
.. note:: 输入图像中的像素值将从 [0, 255] 缩放为 [0.0, 1.0]。图像的数据类型将被转换为由 `output_type` 参数指定的类型，图像通道数保持不变。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **output_type** (numpy.dtype，可选) - 输出图像的数据类型，默认值：numpy.float32。
|
||||
|
|
|
|||
|
|
@ -557,9 +557,9 @@ class Dataset:
|
|||
>>> # and drops the last incomplete batch if there is one.
|
||||
>>> dataset = dataset.batch(100, True)
|
||||
>>> # resize image according to its batch number, if it's 5-th batch, resize to (5^2, 5^2) = (25, 25)
|
||||
>>> def np_resize(col, batchInfo):
|
||||
>>> def np_resize(col, BatchInfo):
|
||||
... output = col.copy()
|
||||
... s = (batchInfo.get_batch_num() + 1) ** 2
|
||||
... s = (BatchInfo.get_batch_num() + 1) ** 2
|
||||
... index = 0
|
||||
... for c in col:
|
||||
... img = Image.fromarray(c.astype('uint8')).convert('RGB')
|
||||
|
|
|
|||
|
|
@ -64,13 +64,13 @@ class CMUArcticDataset(MappableDataset, AudioBaseDataset):
|
|||
|
||||
Raises:
|
||||
RuntimeError: If source raises an exception during execution.
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- CMUArctic dataset doesn't support PKSampler.
|
||||
|
|
@ -199,13 +199,13 @@ class GTZANDataset(MappableDataset, AudioBaseDataset):
|
|||
|
||||
Raises:
|
||||
RuntimeError: If source raises an exception during execution.
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- GTZAN doesn't support PKSampler.
|
||||
|
|
@ -334,13 +334,13 @@ class LibriTTSDataset(MappableDataset, AudioBaseDataset):
|
|||
|
||||
Raises:
|
||||
RuntimeError: If source raises an exception during execution.
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- LibriTTS dataset doesn't support PKSampler.
|
||||
|
|
@ -472,19 +472,19 @@ class LJSpeechDataset(MappableDataset, AudioBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -617,13 +617,13 @@ class SpeechCommandsDataset(MappableDataset, AudioBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -748,19 +748,19 @@ class TedliumDataset(MappableDataset, AudioBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain stm files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain stm files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -949,13 +949,13 @@ class YesNoDataset(MappableDataset, AudioBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
|
|||
|
|
@ -68,18 +68,18 @@ class CSVDataset(SourceDataset, UnionBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_files are not valid or do not exist.
|
||||
ValueError: If field_delim is invalid.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> csv_dataset_dir = ["/path/to/csv_dataset_file"] # contains 1 or multiple csv files
|
||||
|
|
@ -129,8 +129,8 @@ class MindDataset(MappableDataset, UnionBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, 'num_samples' reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
sampler (Sampler, optional): Object used to choose samples from the
|
||||
dataset (default=None, sampler is exclusive
|
||||
with shuffle and block_reader). Support list: SubsetRandomSampler,
|
||||
|
|
@ -146,10 +146,10 @@ class MindDataset(MappableDataset, UnionBaseDataset):
|
|||
|
||||
Raises:
|
||||
ValueError: If dataset_files are not valid or do not exist.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -270,21 +270,21 @@ class TFRecordDataset(SourceDataset, UnionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
shard_equal_rows (bool, optional): Get equal rows for all shards(default=False). If shard_equal_rows
|
||||
is false, number of rows of each shard may be not equal, and may lead to a failure in distributed training.
|
||||
When the number of samples of per TFRecord file are not equal, it is suggested to set to true.
|
||||
This argument should only be specified when num_shards is also specified.
|
||||
This argument should only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
ValueError: If dataset_files are not valid or do not exist.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> from mindspore import dtype as mstype
|
||||
|
|
|
|||
|
|
@ -62,8 +62,8 @@ class AGNewsDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, 'num_samples' reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
|
|
@ -154,18 +154,18 @@ class AmazonReviewDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the mindspore.dataset.config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> amazon_review_dataset_dir = "/path/to/amazon_review_dataset_dir"
|
||||
|
|
@ -245,8 +245,8 @@ class CLUEDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
|
|
@ -396,10 +396,10 @@ class CLUEDataset(SourceDataset, TextBaseDataset):
|
|||
ValueError: If dataset_files are not valid or do not exist.
|
||||
ValueError: task is not in 'AFQMC', 'TNEWS', 'IFLYTEK', 'CMNLI', 'WSC' or 'CSL'.
|
||||
ValueError: usage is not in 'train', 'test' or 'eval'.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> clue_dataset_dir = ["/path/to/clue_dataset_file"] # contains 1 or multiple clue files
|
||||
|
|
@ -479,18 +479,18 @@ class CoNLL2000Dataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> conll2000_dataset_dir = "/path/to/conll2000_dataset_dir"
|
||||
|
|
@ -541,17 +541,17 @@ class DBpediaDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> dbpedia_dataset_dir = "/path/to/dbpedia_dataset_directory"
|
||||
|
|
@ -630,8 +630,8 @@ class EnWik9Dataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
|
|
@ -705,19 +705,19 @@ class IMDBDataset(MappableDataset, TextBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- The shape of the test column.
|
||||
|
|
@ -855,18 +855,18 @@ class IWSLT2016Dataset(SourceDataset, TextBaseDataset):
|
|||
- Shuffle.FILES: Shuffle files only.
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> iwslt2016_dataset_dir = "/path/to/iwslt2016_dataset_dir"
|
||||
|
|
@ -976,18 +976,18 @@ class IWSLT2017Dataset(SourceDataset, TextBaseDataset):
|
|||
- Shuffle.FILES: Shuffle files only.
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> iwslt2017_dataset_dir = "/path/to/iwslt207_dataset_dir"
|
||||
|
|
@ -1075,18 +1075,18 @@ class Multi30kDataset(SourceDataset, TextBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If usage is not 'train', 'test', 'valid' or 'all'.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
RuntimeError: If `usage` is not 'train', 'test', 'valid' or 'all'.
|
||||
RuntimeError: If the length of language_pair is not equal to 2.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
RuntimeError: If num_samples is less than 0.
|
||||
|
||||
Examples:
|
||||
|
|
@ -1174,8 +1174,8 @@ class PennTreebankDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, 'num_samples' reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
|
|
@ -1261,18 +1261,18 @@ class SogouNewsDataset(SourceDataset, TextBaseDataset):
|
|||
- Shuffle.FILES: Shuffle files only.
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> sogou_news_dataset_dir = "/path/to/sogou_news_dataset_dir"
|
||||
|
|
@ -1353,16 +1353,16 @@ class SQuADDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> squad_dataset_dir = "/path/to/squad_dataset_file"
|
||||
|
|
@ -1457,17 +1457,17 @@ class TextFileDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
ValueError: If dataset_files are not valid or do not exist.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> text_file_dataset_dir = ["/path/to/text_file_dataset_file"] # contains 1 or multiple text files
|
||||
|
|
@ -1514,18 +1514,18 @@ class UDPOSDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> udpos_dataset_dir = "/path/to/udpos_dataset_dir"
|
||||
|
|
@ -1570,8 +1570,8 @@ class WikiTextDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, 'num_samples' reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
|
|
@ -1654,17 +1654,17 @@ class YahooAnswersDataset(SourceDataset, TextBaseDataset):
|
|||
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> yahoo_answers_dataset_dir = "/path/to/yahoo_answers_dataset_directory"
|
||||
|
|
@ -1745,18 +1745,18 @@ class YelpReviewDataset(SourceDataset, TextBaseDataset):
|
|||
- Shuffle.FILES: Shuffle files only.
|
||||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
When this argument is specified, `num_samples` reflects the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
num_parallel_workers (int, optional): Number of workers to read the data
|
||||
(default=None, number set in the config).
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
|
||||
Examples:
|
||||
>>> yelp_review_dataset_dir = "/path/to/yelp_review_dataset_dir"
|
||||
|
|
|
|||
|
|
@ -476,7 +476,7 @@ class GeneratorDataset(MappableDataset, UnionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
Random accessible input is required. When this argument is specified, `num_samples` reflects the maximum
|
||||
sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This argument must be specified only
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This argument must be specified only
|
||||
when num_shards is also specified. Random accessible input is required.
|
||||
python_multiprocessing (bool, optional): Parallelize Python operations with multiple worker process. This
|
||||
option could be beneficial if the Python operation is computational heavy (default=True).
|
||||
|
|
@ -486,12 +486,12 @@ class GeneratorDataset(MappableDataset, UnionBaseDataset):
|
|||
Raises:
|
||||
RuntimeError: If source raises an exception during execution.
|
||||
RuntimeError: If len of column_names does not match output len of source.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If sampler and shuffle are specified at the same time.
|
||||
ValueError: If sampler and sharding are specified at the same time.
|
||||
ValueError: If num_shards is specified but shard_id is None.
|
||||
ValueError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If shard_id is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- Input `source` accept user defined Python function(PyFuncs), Do not add network computing operators from
|
||||
|
|
@ -801,7 +801,7 @@ class NumpySlicesDataset(GeneratorDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
Random accessible input is required. When this argument is specified, `num_samples` reflects the max
|
||||
sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This argument must be specified only
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This argument must be specified only
|
||||
when num_shards is also specified. Random accessible input is required.
|
||||
|
||||
Note:
|
||||
|
|
@ -836,12 +836,12 @@ class NumpySlicesDataset(GeneratorDataset):
|
|||
|
||||
Raises:
|
||||
RuntimeError: If len of column_names does not match output len of data.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If sampler and shuffle are specified at the same time.
|
||||
ValueError: If sampler and sharding are specified at the same time.
|
||||
ValueError: If num_shards is specified but shard_id is None.
|
||||
ValueError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If shard_id is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> # 1) Input data can be a list
|
||||
|
|
|
|||
|
|
@ -122,9 +122,9 @@ class Caltech101Dataset(GeneratorDataset):
|
|||
dataset_dir (str): Path to the root directory that contains the dataset. This root directory contains two
|
||||
subdirectories, one is called 101_ObjectCategories, which stores images,
|
||||
and the other is called Annotations, which stores annotations.
|
||||
target_type (str, optional): Target of the image. If target_type is 'category', return category represents
|
||||
the target class. If target_type is 'annotation', return annotation.
|
||||
If target_type is 'all', return category and annotation (default=None, means 'category').
|
||||
target_type (str, optional): Target of the image. If `target_type` is 'category', return category represents
|
||||
the target class. If `target_type` is 'annotation', return annotation.
|
||||
If `target_type` is 'all', return category and annotation (default=None, means 'category').
|
||||
num_samples (int, optional): The number of images to be included in the dataset
|
||||
(default=None, all images).
|
||||
num_parallel_workers (int, optional): Number of workers to read the data (default=1).
|
||||
|
|
@ -136,18 +136,18 @@ class Caltech101Dataset(GeneratorDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
ValueError: If target_type is not set correctly.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `target_type` is not set correctly.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -295,19 +295,20 @@ class Caltech256Dataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `target_type` is not 'category', 'annotation' or 'all'.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -429,14 +430,14 @@ class CelebADataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
ValueError: If usage is not 'train', 'valid', 'test' or 'all'.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If `usage` is not 'train', 'valid', 'test' or 'all'.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -589,20 +590,20 @@ class Cifar10Dataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
ValueError: If usage is not 'train', 'test' or 'all'.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If `usage` is not 'train', 'test' or 'all'.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -720,20 +721,20 @@ class Cifar100Dataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, 'num_samples' reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
ValueError: If usage is not 'train', 'test' or 'all'.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If `usage` is not 'train', 'test' or 'all'.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -848,23 +849,23 @@ class CityscapesDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is invalid or does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If task is invalid.
|
||||
ValueError: If quality_mode is invalid.
|
||||
ValueError: If usage is invalid.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is invalid or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `task` is invalid.
|
||||
ValueError: If `quality_mode` is invalid.
|
||||
ValueError: If `usage` is invalid.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -1015,8 +1016,8 @@ class CocoDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
extra_metadata(bool, optional): Flag to add extra meta-data to row. If True, an additional column will be
|
||||
|
|
@ -1059,17 +1060,17 @@ class CocoDataset(MappableDataset, VisionBaseDataset):
|
|||
+-------------------------+----------------------------------------------+
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
RuntimeError: If parse JSON file failed.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
ValueError: If task is not in ['Detection', 'Stuff', 'Panoptic', 'Keypoint', 'Captioning'].
|
||||
ValueError: If annotation_file is not exist.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If `task` is not in ['Detection', 'Stuff', 'Panoptic', 'Keypoint', 'Captioning'].
|
||||
ValueError: If `annotation_file` is not exist.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- Column '[_meta-filename, dtype=string]' won't be output unless an explicit rename dataset op is added
|
||||
|
|
@ -1261,25 +1262,25 @@ class DIV2KDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is invalid or does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If usage is invalid.
|
||||
ValueError: If downgrade is invalid.
|
||||
ValueError: If scale is invalid.
|
||||
ValueError: If scale equal to 8 and downgrade not equal to 'bicubic'.
|
||||
ValueError: If downgrade in ['mild', 'difficult', 'wild'] and scale not equal to 4.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is invalid or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `usage` is invalid.
|
||||
ValueError: If `downgrade` is invalid.
|
||||
ValueError: If `scale` is invalid.
|
||||
ValueError: If `scale` equal to 8 and downgrade not equal to 'bicubic'.
|
||||
ValueError: If `downgrade` in ['mild', 'difficult', 'wild'] and `scale` not equal to 4.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -1450,11 +1451,11 @@ class EMnistDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -1581,12 +1582,12 @@ class FakeImageDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. 'sampler' and 'shuffle' are mutually exclusive.
|
||||
|
|
@ -1670,13 +1671,13 @@ class FashionMnistDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -1786,21 +1787,21 @@ class FlickrDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is not valid or does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If annotation_file is not exist.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is not valid or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `annotation_file` is not exist.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -2029,17 +2030,17 @@ class Flowers102Dataset(GeneratorDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
Random accessible input is required. When this argument is specified, 'num_samples' reflects the max
|
||||
sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This argument must be specified only
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This argument must be specified only
|
||||
when num_shards is also specified. Random accessible input is required.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. 'sampler' and 'shuffle' are mutually exclusive.
|
||||
|
|
@ -2197,20 +2198,20 @@ class ImageFolderDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
RuntimeError: If class_indexing is not a dictionary.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- The shape of the image column is [image_size] if decode flag is False, or [H,W,C] otherwise.
|
||||
|
|
@ -2582,13 +2583,13 @@ class ManifestDataset(MappableDataset, VisionBaseDataset):
|
|||
|
||||
Raises:
|
||||
RuntimeError: If dataset_files are not valid or do not exist.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
RuntimeError: If class_indexing is not a dictionary.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- The shape of the image column is [image_size] if decode flag is False, or [H,W,C] otherwise.
|
||||
|
|
@ -2697,13 +2698,14 @@ class MnistDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If `usage` is not 'train'、'test' or 'all'.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but shard_id is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -2824,17 +2826,17 @@ class PhotoTourDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If usage is not in ["train", "test"].
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `usage` is not in ["train", "test"].
|
||||
ValueError: If name is not in ["notredame", "yosemite", "liberty",
|
||||
"notredame_harris", "yosemite_harris", "liberty_harris"].
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. `sampler` and `shuffle` are mutually exclusive. The table
|
||||
|
|
@ -2972,15 +2974,15 @@ class Places365Dataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If usage is not in ["train-standard", "train-challenge", "val"].
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
ValueError: If `usage` is not in ["train-standard", "train-challenge", "val"].
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. 'sampler' and 'shuffle' are mutually exclusive.
|
||||
|
|
@ -3114,13 +3116,13 @@ class QMnistDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -3230,8 +3232,8 @@ class RandomDataset(SourceDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, 'num_samples' reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
"""
|
||||
|
||||
@check_random_dataset
|
||||
|
|
@ -3335,20 +3337,20 @@ class SBDataset(GeneratorDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is not valid or does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If task is not in ['Boundaries', 'Segmentation'].
|
||||
ValueError: If usage is not in ['train', 'val', 'train_noval', 'all'].
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is not valid or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `task` is not in ['Boundaries', 'Segmentation'].
|
||||
ValueError: If `usage` is not in ['train', 'val', 'train_noval', 'all'].
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -3469,13 +3471,13 @@ class SBUDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. 'sampler' and 'shuffle' are mutually exclusive.
|
||||
|
|
@ -3577,18 +3579,18 @@ class SemeionDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
@ -3703,20 +3705,20 @@ class STL10Dataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, 'num_samples' reflects
|
||||
the max sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is not valid or does not exist or does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If usage is invalid.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is not valid or does not exist or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `usage` is invalid.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. 'sampler' and 'shuffle' are mutually exclusive.
|
||||
|
|
@ -3869,18 +3871,18 @@ class SVHNDataset(GeneratorDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided into (default=None).
|
||||
Random accessible input is required. When this argument is specified, 'num_samples' reflects the max
|
||||
sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This argument must be specified only
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This argument must be specified only
|
||||
when num_shards is also specified. Random accessible input is required.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is not valid or does not exist or does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If usage is invalid.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is not valid or does not exist or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `usage` is invalid.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a sampler. 'sampler' and 'shuffle' are mutually exclusive.
|
||||
|
|
@ -3995,14 +3997,14 @@ class USPSDataset(SourceDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir is not valid or does not exist or does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If usage is invalid.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` is not valid or does not exist or does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `usage` is invalid.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Examples:
|
||||
>>> usps_dataset_dir = "/path/to/usps_dataset_directory"
|
||||
|
|
@ -4091,27 +4093,27 @@ class VOCDataset(MappableDataset, VisionBaseDataset):
|
|||
num_shards (int, optional): Number of shards that the dataset will be divided
|
||||
into (default=None). When this argument is specified, `num_samples` reflects
|
||||
the maximum sample number of per shard.
|
||||
shard_id (int, optional): The shard ID within num_shards (default=None). This
|
||||
argument can only be specified when num_shards is also specified.
|
||||
shard_id (int, optional): The shard ID within `num_shards` (default=None). This
|
||||
argument can only be specified when `num_shards` is also specified.
|
||||
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing.
|
||||
(default=None, which means no cache is used).
|
||||
extra_metadata(bool, optional): Flag to add extra meta-data to row. If True, an additional column named
|
||||
:py:obj:`[_meta-filename, dtype=string]` will be output at the end (default=False).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
RuntimeError: If xml of Annotations is an invalid format.
|
||||
RuntimeError: If xml of Annotations loss attribution of `object`.
|
||||
RuntimeError: If xml of Annotations loss attribution of `bndbox`.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
ValueError: If task is not equal 'Segmentation' or 'Detection'.
|
||||
ValueError: If task equal 'Segmentation' but class_indexing is not None.
|
||||
ValueError: If txt related to mode is not exist.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- Column '[_meta-filename, dtype=string]' won't be output unless an explicit rename dataset op
|
||||
|
|
@ -4298,17 +4300,17 @@ class WIDERFaceDataset(MappableDataset, VisionBaseDataset):
|
|||
(default=None, which means no cache is used).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If dataset_dir does not contain data files.
|
||||
RuntimeError: If num_parallel_workers exceeds the max thread numbers.
|
||||
RuntimeError: If sampler and shuffle are specified at the same time.
|
||||
RuntimeError: If sampler and sharding are specified at the same time.
|
||||
RuntimeError: If num_shards is specified but shard_id is None.
|
||||
RuntimeError: If shard_id is specified but num_shards is None.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
ValueError: If usage is not in ['train', 'test', 'valid', 'all'].
|
||||
ValueError: If annotation_file is not exist.
|
||||
ValueError: If dataset_dir is not exist.
|
||||
ValueError: If shard_id is invalid (< 0 or >= num_shards).
|
||||
RuntimeError: If `dataset_dir` does not contain data files.
|
||||
ValueError: If `num_parallel_workers` exceeds the max thread numbers.
|
||||
RuntimeError: If `sampler` and `shuffle` are specified at the same time.
|
||||
RuntimeError: If `sampler` and `num_shards`/`shard_id` are specified at the same time.
|
||||
RuntimeError: If `num_shards` is specified but `shard_id` is None.
|
||||
RuntimeError: If `shard_id` is specified but `num_shards` is None.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
ValueError: If `usage` is not in ['train', 'test', 'valid', 'all'].
|
||||
ValueError: If `annotation_file` is not exist.
|
||||
ValueError: If `dataset_dir` is not exist.
|
||||
ValueError: If `shard_id` is invalid (< 0 or >= `num_shards`).
|
||||
|
||||
Note:
|
||||
- This dataset can take in a `sampler`. `sampler` and `shuffle` are mutually exclusive.
|
||||
|
|
|
|||
|
|
@ -366,7 +366,7 @@ class SentencePieceTokenizer(TextTensorOperation):
|
|||
|
||||
Raises:
|
||||
TypeError: If `mode` is not of type string or SentencePieceVocab.
|
||||
TypError: If `out_type` is not of type SPieceTokenizerOutType.
|
||||
TypeError: If `out_type` is not of type SPieceTokenizerOutType.
|
||||
|
||||
Supported Platforms:
|
||||
``CPU``
|
||||
|
|
@ -401,7 +401,7 @@ class SlidingWindow(TextTensorOperation):
|
|||
Raises:
|
||||
TypeError: If `width` is not of type int.
|
||||
ValueError: If value of `width` is not positive.
|
||||
TypError: If `axis` is not of type int.
|
||||
TypeError: If `axis` is not of type int.
|
||||
|
||||
Supported Platforms:
|
||||
``CPU``
|
||||
|
|
|
|||
Loading…
Reference in New Issue