class_docstr: fastNLP的数据容器。
  
  Example::
  
      from fastNLP.core.dataset import DataSet, Instance
      data = {'x': [[1, 0, 1], [0, 1, 1], 'y': [0, 1]}
      data1 = [Instance(x=[1,0,1],y=0), Instance(x=[0,1,1],y=1)]
      ds = DataSet(data)
      ds = DataSet(data1)
  
  fastNLP的 DataSet 是 key-value 存储形式， 目前支持两种初始化方式，输入 data 分别为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 和
  ``Dict[str, List[Any]]``。
  
      * 当 data 为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 时,  每个 ``Instance`` 的 field_name 需要保持一致。
        Instance 详见 :class:`~fastNLP.core.dataset.Instance` 。
      * 当 data 为 ``Dict[str, List[Any]]`` 时， 则每个 key 的 value 应该为等长的 list， 否则不同 field 的长度不一致。
  
  :param data: 初始化的内容，其只能为两种类型，分别为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 和
      ``Dict[str, List[Any]]``。
  
      * 当 data 为 ``List[:class:`~fastNLP.core.dataset.Instance`]`` 时,  每个 ``Instance`` 的 field_name 需要保持一致。
        Instance 详见 :class:`~fastNLP.core.dataset.Instance` 。
      * 当 data 为 ``Dict[str, List[Any]] 时， 则每个 key 的 value 应该为等长的 list， 否则不同 field 的长度不一致。
class_docstr_tok: ["fastNLP\u7684\u6570\u636e\u5bb9\u5668\u3002","Example",":",":","from","fastNLP.core.dataset","import","DataSet",",","Instance","data","=","{","'","x","'",":","[","[","1",",","0",",","1","]",",","[","0",",","1",",","1","]",",","'","y","'",":","[","0",",","1","]","}","data1","=","[","Instance","(","x=","[","1,0,1","]",",","y=0",")",",","Instance","(","x=","[","0,1,1","]",",","y=1",")","]","ds","=","DataSet","(","data",")","ds","=","DataSet","(","data1",")","fastNLP\u7684","DataSet","\u662f","key-value","\u5b58\u50a8\u5f62\u5f0f\uff0c","\u76ee\u524d\u652f\u6301\u4e24\u79cd\u521d\u59cb\u5316\u65b9\u5f0f\uff0c\u8f93\u5165","data","\u5206\u522b\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u548c","``","Dict","[","str",",","List","[","Any","]","]","``","\u3002","*","\u5f53","data","\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u65f6",",","\u6bcf\u4e2a","``","Instance","``","\u7684","field_name","\u9700\u8981\u4fdd\u6301\u4e00\u81f4\u3002","Instance","\u8be6\u89c1",":","class",":","`","~fastNLP.core.dataset.Instance","`","\u3002","*","\u5f53","data","\u4e3a","``","Dict","[","str",",","List","[","Any","]","]","``","\u65f6\uff0c","\u5219\u6bcf\u4e2a","key","\u7684","value","\u5e94\u8be5\u4e3a\u7b49\u957f\u7684","list\uff0c","\u5426\u5219\u4e0d\u540c","field","\u7684\u957f\u5ea6\u4e0d\u4e00\u81f4\u3002",":","param","data",":","\u521d\u59cb\u5316\u7684\u5185\u5bb9\uff0c\u5176\u53ea\u80fd\u4e3a\u4e24\u79cd\u7c7b\u578b\uff0c\u5206\u522b\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u548c","``","Dict","[","str",",","List","[","Any","]","]","``","\u3002","*","\u5f53","data","\u4e3a","``","List","[",":","class",":","`","~fastNLP.core.dataset.Instance","`","]","``","\u65f6",",","\u6bcf\u4e2a","``","Instance","``","\u7684","field_name","\u9700\u8981\u4fdd\u6301\u4e00\u81f4\u3002","Instance","\u8be6\u89c1",":","class",":","`","~fastNLP.core.dataset.Instance","`","\u3002","*","\u5f53","data","\u4e3a","``","Dict","[","str",",","List","[","Any","]","]","\u65f6\uff0c","\u5219\u6bcf\u4e2a","key","\u7684","value","\u5e94\u8be5\u4e3a\u7b49\u957f\u7684","list\uff0c","\u5426\u5219\u4e0d\u540c","field","\u7684\u957f\u5ea6\u4e0d\u4e00\u81f4\u3002"]
docstr: :param num_proc: 使用进程的数量。
  
      .. note::
  
          由于 ``python`` 语言的特性，设置该参数后会导致相应倍数的内存增长，这可能会对您程序的执行带来一定的影响。另外，使用多进程时，
          ``func`` 函数中的打印将不会输出。
  
  :param func: 用户自定义处理函数，参数是 ``DataSet`` 中的 ``Instance``
  :param _apply_field: 需要传进去func的数据集的field_name
  :param progress_bar: 显示 progress_bar 的方式，支持 `["rich", "tqdm", None]`。
  :param progress_desc: 进度条的描述字符，默认为'Main
docstr_tok: [":","param","num_proc",":","\u4f7f\u7528\u8fdb\u7a0b\u7684\u6570\u91cf\u3002","..","note",":",":","\u7531\u4e8e","``","python","``","\u8bed\u8a00\u7684\u7279\u6027\uff0c\u8bbe\u7f6e\u8be5\u53c2\u6570\u540e\u4f1a\u5bfc\u81f4\u76f8\u5e94\u500d\u6570\u7684\u5185\u5b58\u589e\u957f\uff0c\u8fd9\u53ef\u80fd\u4f1a\u5bf9\u60a8\u7a0b\u5e8f\u7684\u6267\u884c\u5e26\u6765\u4e00\u5b9a\u7684\u5f71\u54cd\u3002\u53e6\u5916\uff0c\u4f7f\u7528\u591a\u8fdb\u7a0b\u65f6\uff0c","``","func","``","\u51fd\u6570\u4e2d\u7684\u6253\u5370\u5c06\u4e0d\u4f1a\u8f93\u51fa\u3002",":","param","func",":","\u7528\u6237\u81ea\u5b9a\u4e49\u5904\u7406\u51fd\u6570\uff0c\u53c2\u6570\u662f","``","DataSet","``","\u4e2d\u7684","``","Instance","``",":","param","_apply_field",":","\u9700\u8981\u4f20\u8fdb\u53bbfunc\u7684\u6570\u636e\u96c6\u7684field_name",":","param","progress_bar",":","\u663e\u793a","progress_bar","\u7684\u65b9\u5f0f\uff0c\u652f\u6301","`","[","``","rich","''",",","``","tqdm","''",",","None","]","`","\u3002",":","param","progress_desc",":","\u8fdb\u5ea6\u6761\u7684\u63cf\u8ff0\u5b57\u7b26\uff0c\u9ed8\u8ba4\u4e3a'Main"]
code: def _apply_process(
      self,
      num_proc: int = 0,
      func: Callable = None,
      progress_bar: str = "rich",
      _apply_field: str = None,
      progress_desc: str = "Main",
  ) -> list:
      """
      :param num_proc: 使用进程的数量。
  
          .. note::
  
              由于 ``python`` 语言的特性，设置该参数后会导致相应倍数的内存增长，这可能会对您程序的执行带来一定的影响。另外，使用多进程时，
              ``func`` 函数中的打印将不会输出。
  
      :param func: 用户自定义处理函数，参数是 ``DataSet`` 中的 ``Instance``
      :param _apply_field: 需要传进去func的数据集的field_name
      :param progress_bar: 显示 progress_bar 的方式，支持 `["rich", "tqdm", None]`。
      :param progress_desc: 进度条的描述字符，默认为'Main
      """
      if (
          isinstance(func, LambdaType)
          and num_proc > 1
          and func.__name__ == "<lambda>"
      ):
          raise TypeError(
              "Lambda function does not support multiple processes, please set `num_proc=0`."
          )
      if num_proc > 1 and sys.platform in ("win32", "msys", "cygwin"):
          raise RuntimeError(
              "Your platform does not support multiprocessing with fork, please set `num_proc=0`"
          )
  
      if num_proc < 2:
          results = _apply_single(
              ds=self,
              _apply_field=_apply_field,
              func=func,
              desc=progress_desc,
              progress_bar=progress_bar,
          )
      else:
          # TODO 1. desc这个需要修改一下，应该把 subprocess 的 desc 修改一下。修改成Process 1 / Process 2
          import multiprocessing as mp
  
          ctx = mp.get_context("fork")
          num_proc = min(num_proc, len(self))
          #   划分数据集
          shard_len = len(self) // num_proc
          num_left_sample = len(self) % num_proc
          start = 0
          shard_data = []
          for _i in range(num_proc):
              end = shard_len + int(_i < num_left_sample) + start
              shard_data.append(self[start:end])
              start = end
          #   配置共享参数，线程以实现 main progress 能够实时更新。
          counter = ctx.Value("i", 0, lock=True)
          pool = []
          queues = []
          results = []
          for i in range(num_proc):
              queue = ctx.SimpleQueue()
              proc = ctx.Process(
                  target=_multi_proc,
                  args=(
                      shard_data[i],
                      _apply_field,
                      func,
                      counter,
                      queue,
                  ),
              )
              proc.start()
              pool.append(proc)
              queues.append(queue)
          progress_bar = progress_bars.get(
              progress_bar, DummyFRichProgress()
          )
          total_len = len(self)
          task_id = progress_bar.add_task(
              description=progress_desc, total=total_len
          )
          last_count = -1
          while counter.value < total_len or last_count == -1:
              while counter.value == last_count:
                  time.sleep(0.1)
              advance = counter.value - last_count
              last_count = counter.value
              progress_bar.update(
                  task_id, advance=advance, refresh=True
              )
  
          for idx, proc in enumerate(pool):
              results.extend(pickle.loads(queues[idx].get()))
              proc.join()
          progress_bar.destroy_task(task_id)
      return results
  
imports: import _pickle
  from copy import deepcopy
  from typing import Optional, List, Callable, Union, Dict, Any, Mapping
  from types import LambdaType
  import sys
  import time
  import numpy
  from .field import FieldArray
  from .instance import Instance
  from fastNLP.core.utils.utils import pretty_table_printer, deprecated
  from fastNLP.core.collators import Collator
  from fastNLP.core.utils.rich_progress import f_rich_progress, DummyFRichProgress
  from fastNLP.core.utils.tqdm_progress import f_tqdm_progress
  from ..log import logger
  from fastNLP.core.utils.dummy_class import DummyClass
  from ..utils.utils import _get_fun_msg
label_desc: Use image node_id 27 for calling the DataSet obj's underlying member method code with example usage: obj._apply_process(num_proc, func, progress_bar, _apply_field, progress_desc) and returns: results