class_docstr: fastNLP的数据容器。
  
  Example::
  
      from fastNLP.core.dataset import DataSet, Instance
      data = {'x': [[1, 0, 1], [0, 1, 1], 'y': [0, 1]}
      data1 = [Instance(x=[1,0,1],y=0), Instance(x=[0,1,1],y=1)]
      ds = DataSet(data)
      ds = DataSet(data1)
  
  fastNLP的 DataSet 是 key-value 存储形式， 目前支持两种初始化方式，输入 data 分别为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 和
  ``Dict[str, List[Any]]``。
  
      * 当 data 为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 时,  每个 ``Instance`` 的 field_name 需要保持一致。
        Instance 详见 :class:`~fastNLP.core.dataset.Instance` 。
      * 当 data 为 ``Dict[str, List[Any]]`` 时， 则每个 key 的 value 应该为等长的 list， 否则不同 field 的长度不一致。
  
  :param data: 初始化的内容，其只能为两种类型，分别为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 和
      ``Dict[str, List[Any]]``。
  
      * 当 data 为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 时,  每个 ``Instance`` 的 field_name 需要保持一致。
        Instance 详见 :class:`~fastNLP.core.dataset.Instance` 。
      * 当 data 为 ``Dict[str, List[Any]] 时， 则每个 key 的 value 应该为等长的 list， 否则不同 field 的长度不一致。
class_docstr_tok: ["fastNLP\u7684\u6570\u636e\u5bb9\u5668\u3002","Example",":",":","from","fastNLP.core.dataset","import","DataSet",",","Instance","data","=","{","'","x","'",":","[","[","1",",","0",",","1","]",",","[","0",",","1",",","1","]",",","'","y","'",":","[","0",",","1","]","}","data1","=","[","Instance","(","x=","[","1,0,1","]",",","y=0",")",",","Instance","(","x=","[","0,1,1","]",",","y=1",")","]","ds","=","DataSet","(","data",")","ds","=","DataSet","(","data1",")","fastNLP\u7684","DataSet","\u662f","key-value","\u5b58\u50a8\u5f62\u5f0f\uff0c","\u76ee\u524d\u652f\u6301\u4e24\u79cd\u521d\u59cb\u5316\u65b9\u5f0f\uff0c\u8f93\u5165","data","\u5206\u522b\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u548c","``","Dict","[","str",",","List","[","Any","]","]","``","\u3002","*","\u5f53","data","\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u65f6",",","\u6bcf\u4e2a","``","Instance","``","\u7684","field_name","\u9700\u8981\u4fdd\u6301\u4e00\u81f4\u3002","Instance","\u8be6\u89c1",":","class",":","`","~fastNLP.core.dataset.Instance","`","\u3002","*","\u5f53","data","\u4e3a","``","Dict","[","str",",","List","[","Any","]","]","``","\u65f6\uff0c","\u5219\u6bcf\u4e2a","key","\u7684","value","\u5e94\u8be5\u4e3a\u7b49\u957f\u7684","list\uff0c","\u5426\u5219\u4e0d\u540c","field","\u7684\u957f\u5ea6\u4e0d\u4e00\u81f4\u3002",":","param","data",":","\u521d\u59cb\u5316\u7684\u5185\u5bb9\uff0c\u5176\u53ea\u80fd\u4e3a\u4e24\u79cd\u7c7b\u578b\uff0c\u5206\u522b\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u548c","``","Dict","[","str",",","List","[","Any","]","]","``","\u3002","*","\u5f53","data","\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u65f6",",","\u6bcf\u4e2a","``","Instance","``","\u7684","field_name","\u9700\u8981\u4fdd\u6301\u4e00\u81f4\u3002","Instance","\u8be6\u89c1",":","class",":","`","~fastNLP.core.dataset.Instance","`","\u3002","*","\u5f53","data","\u4e3a","``","Dict","[","str",",","List","[","Any","]","]","\u65f6\uff0c","\u5219\u6bcf\u4e2a","key","\u7684","value","\u5e94\u8be5\u4e3a\u7b49\u957f\u7684","list\uff0c","\u5426\u5219\u4e0d\u540c","field","\u7684\u957f\u5ea6\u4e0d\u4e00\u81f4\u3002"]
docstr: 如果需要对某个 field 的内容进行特殊的调整，请使用这个函数。
  
  :param field_name: 需要调整的 field 的名称。如果 :meth:`Dataset.__getitem__` 方法返回的是字典类型，则可以直接使用对应的
      field 的 key 来表示，如果是嵌套字典，可以使用元组表示多层次的 key，例如 ``{'a': {'b': 1}}`` 中可以使用 ``('a', 'b')``;
      如果 :meth:`Dataset.__getitem__` 返回的是 Sequence 类型，则可以使用 ``'_0'``, ``'_1'`` 表示序列中第 **0** 或 **1** 个元素。
      如果该 field 在数据中没有找到，则报错；如果 :meth:`Dataset.__getitem__` 返回的是就是整体内容，请使用 "_single" 。
  :param pad_val: 这个 field 的默认 pad 值。如果设置为 ``None``，则表示该 field 不需要 pad , fastNLP 默认只会对可以 pad 的
      field 进行 pad，所以如果对应 field 本身就不是可以 pad 的形式，可以不需要主动设置为 ``None`` 。如果 ``backend`` 为 ``None``，
      该值无意义。
  :param dtype: 对于需要 pad 的 field ，该 field 数据的 ``dtype`` 。
  :param backend: 可选 ``['raw', 'numpy', 'torch', 'paddle', 'jittor', 'oneflow', 'auto']`` ，分别代表，输出为 :class:`list`, 
      :class:`numpy.ndarray`, :class:`torch.Tensor`, :class:`paddle.Tensor`, :class:`jittor.Var`, :class:`oneflow.Tensor` 类型。
      若 ``pad_val`` 为 ``None`` ，该值无意义 。
  :param pad_fn: 指定当前 field 的 pad 函数，传入该函数则 ``pad_val``, ``dtype``, ``backend`` 等参数失效。``pad_fn`` 的输入为当前 field 的
      batch 形式。 Collator 将自动 unbatch 数据，然后将各个 field 组成各自的 batch 。
  :return: 自身的 collator；
docstr_tok: ["\u5982\u679c\u9700\u8981\u5bf9\u67d0\u4e2a","field","\u7684\u5185\u5bb9\u8fdb\u884c\u7279\u6b8a\u7684\u8c03\u6574\uff0c\u8bf7\u4f7f\u7528\u8fd9\u4e2a\u51fd\u6570\u3002",":","param","field_name",":","\u9700\u8981\u8c03\u6574\u7684","field","\u7684\u540d\u79f0\u3002\u5982\u679c",":","meth",":","`","Dataset.__getitem__","`","\u65b9\u6cd5\u8fd4\u56de\u7684\u662f\u5b57\u5178\u7c7b\u578b\uff0c\u5219\u53ef\u4ee5\u76f4\u63a5\u4f7f\u7528\u5bf9\u5e94\u7684","field","\u7684","key","\u6765\u8868\u793a\uff0c\u5982\u679c\u662f\u5d4c\u5957\u5b57\u5178\uff0c\u53ef\u4ee5\u4f7f\u7528\u5143\u7ec4\u8868\u793a\u591a\u5c42\u6b21\u7684","key\uff0c\u4f8b\u5982","``","{","'","a","'",":","{","'","b","'",":","1","}","}","``","\u4e2d\u53ef\u4ee5\u4f7f\u7528","``","(","'","a","'",",","'","b","'",")","``",";","\u5982\u679c",":","meth",":","`","Dataset.__getitem__","`","\u8fd4\u56de\u7684\u662f","Sequence","\u7c7b\u578b\uff0c\u5219\u53ef\u4ee5\u4f7f\u7528","``","'_0","'","``",",","``","'_1","'","``","\u8868\u793a\u5e8f\u5217\u4e2d\u7b2c","*","*","0","*","*","\u6216","*","*","1","*","*","\u4e2a\u5143\u7d20\u3002","\u5982\u679c\u8be5","field","\u5728\u6570\u636e\u4e2d\u6ca1\u6709\u627e\u5230\uff0c\u5219\u62a5\u9519\uff1b\u5982\u679c",":","meth",":","`","Dataset.__getitem__","`","\u8fd4\u56de\u7684\u662f\u5c31\u662f\u6574\u4f53\u5185\u5bb9\uff0c\u8bf7\u4f7f\u7528","``","_single","''","\u3002",":","param","pad_val",":","\u8fd9\u4e2a","field","\u7684\u9ed8\u8ba4","pad","\u503c\u3002\u5982\u679c\u8bbe\u7f6e\u4e3a","``","None","``","\uff0c\u5219\u8868\u793a\u8be5","field","\u4e0d\u9700\u8981","pad",",","fastNLP","\u9ed8\u8ba4\u53ea\u4f1a\u5bf9\u53ef\u4ee5","pad","\u7684","field","\u8fdb\u884c","pad\uff0c\u6240\u4ee5\u5982\u679c\u5bf9\u5e94","field","\u672c\u8eab\u5c31\u4e0d\u662f\u53ef\u4ee5","pad","\u7684\u5f62\u5f0f\uff0c\u53ef\u4ee5\u4e0d\u9700\u8981\u4e3b\u52a8\u8bbe\u7f6e\u4e3a","``","None","``","\u3002\u5982\u679c","``","backend","``","\u4e3a","``","None","``","\uff0c","\u8be5\u503c\u65e0\u610f\u4e49\u3002",":","param","dtype",":","\u5bf9\u4e8e\u9700\u8981","pad","\u7684","field","\uff0c\u8be5","field","\u6570\u636e\u7684","``","dtype","``","\u3002",":","param","backend",":","\u53ef\u9009","``","[","'raw","'",",","'numpy","'",",","'torch","'",",","'paddle","'",",","'jittor","'",",","'oneflow","'",",","'auto","'","]","``","\uff0c\u5206\u522b\u4ee3\u8868\uff0c\u8f93\u51fa\u4e3a",":","class",":","`","list","`",",",":","class",":","`","numpy.ndarray","`",",",":","class",":","`","torch.Tensor","`",",",":","class",":","`","paddle.Tensor","`",",",":","class",":","`","jittor.Var","`",",",":","class",":","`","oneflow.Tensor","`","\u7c7b\u578b\u3002","\u82e5","``","pad_val","``","\u4e3a","``","None","``","\uff0c\u8be5\u503c\u65e0\u610f\u4e49","\u3002",":","param","pad_fn",":","\u6307\u5b9a\u5f53\u524d","field","\u7684","pad","\u51fd\u6570\uff0c\u4f20\u5165\u8be5\u51fd\u6570\u5219","``","pad_val","``",",","``","dtype","``",",","``","backend","``","\u7b49\u53c2\u6570\u5931\u6548\u3002","``","pad_fn","``","\u7684\u8f93\u5165\u4e3a\u5f53\u524d","field","\u7684","batch","\u5f62\u5f0f\u3002","Collator","\u5c06\u81ea\u52a8","unbatch","\u6570\u636e\uff0c\u7136\u540e\u5c06\u5404\u4e2a","field","\u7ec4\u6210\u5404\u81ea\u7684","batch","\u3002",":","return",":","\u81ea\u8eab\u7684","collator\uff1b"]
code: def set_pad(
      self,
      field_name: Union[str, tuple],
      pad_val: Union[int, float, None] = 0,
      dtype=None,
      backend=None,
      pad_fn: Callable = None,
  ) -> Collator:
      """
      如果需要对某个 field 的内容进行特殊的调整，请使用这个函数。
  
      :param field_name: 需要调整的 field 的名称。如果 :meth:`Dataset.__getitem__` 方法返回的是字典类型，则可以直接使用对应的
          field 的 key 来表示，如果是嵌套字典，可以使用元组表示多层次的 key，例如 ``{'a': {'b': 1}}`` 中可以使用 ``('a', 'b')``;
          如果 :meth:`Dataset.__getitem__` 返回的是 Sequence 类型，则可以使用 ``'_0'``, ``'_1'`` 表示序列中第 **0** 或 **1** 个元素。
          如果该 field 在数据中没有找到，则报错；如果 :meth:`Dataset.__getitem__` 返回的是就是整体内容，请使用 "_single" 。
      :param pad_val: 这个 field 的默认 pad 值。如果设置为 ``None``，则表示该 field 不需要 pad , fastNLP 默认只会对可以 pad 的
          field 进行 pad，所以如果对应 field 本身就不是可以 pad 的形式，可以不需要主动设置为 ``None`` 。如果 ``backend`` 为 ``None``，
          该值无意义。
      :param dtype: 对于需要 pad 的 field ，该 field 数据的 ``dtype`` 。
      :param backend: 可选 ``['raw', 'numpy', 'torch', 'paddle', 'jittor', 'oneflow', 'auto']`` ，分别代表，输出为 :class:`list`,
          :class:`numpy.ndarray`, :class:`torch.Tensor`, :class:`paddle.Tensor`, :class:`jittor.Var`, :class:`oneflow.Tensor` 类型。
          若 ``pad_val`` 为 ``None`` ，该值无意义 。
      :param pad_fn: 指定当前 field 的 pad 函数，传入该函数则 ``pad_val``, ``dtype``, ``backend`` 等参数失效。``pad_fn`` 的输入为当前 field 的
          batch 形式。 Collator 将自动 unbatch 数据，然后将各个 field 组成各自的 batch 。
      :return: 自身的 collator；
      """
      if isinstance(self.collator, Collator):
          self.collator.set_pad(
              field_name=field_name,
              pad_val=pad_val,
              dtype=dtype,
              pad_fn=pad_fn,
              backend=backend,
          )
          return self.collator
      else:
          raise ValueError(
              f"Only when the collate_fn is a fastNLP Collator, set_pad() is allowed."
          )
  
imports: import _pickle
  from copy import deepcopy
  from typing import Optional, List, Callable, Union, Dict, Any, Mapping
  from types import LambdaType
  import sys
  import time
  import numpy
  from .field import FieldArray
  from .instance import Instance
  from fastNLP.core.utils.utils import pretty_table_printer, deprecated
  from fastNLP.core.collators import Collator
  from fastNLP.core.utils.rich_progress import f_rich_progress, DummyFRichProgress
  from fastNLP.core.utils.tqdm_progress import f_tqdm_progress
  from ..log import logger
  from fastNLP.core.utils.dummy_class import DummyClass
  from ..utils.utils import _get_fun_msg
label_desc: Use image node_id 40 for calling the DataSet obj's underlying member method code with example usage: obj.set_pad(field_name, pad_val, dtype, backend, pad_fn) and returns: self