数据集上传

# 首次使用请请先安装aistudio-sdk库
pip install --upgrade aistudio-sdk

aistudio upload -h
usage: PROG upload [-h] [--repo-type {model,dataset}] [--include [INCLUDE ...]] [--exclude [EXCLUDE ...]] [--commit-message COMMIT_MESSAGE] [--token TOKEN]
                   [--max-workers MAX_WORKERS]
                   repo_id [local_path] [path_in_repo]

positional arguments:
  repo_id               The ID of the repo to upload to (e.g. `username/repo-name`)
  local_path            Optional, Local path to the file or folder to upload. Defaults to current directory.
  path_in_repo          Optional, Path of the file or folder in the repo. Defaults to the relative path of the file or folder.

options:
  -h, --help            show this help message and exit
  --repo-type {model,dataset}
                        Type of the repo to upload to (e.g. `dataset`, `model`). Defaults to be `model`.
  --include [INCLUDE ...]
                        Glob patterns to match files to upload.
  --exclude [EXCLUDE ...]
                        Glob patterns to exclude from files to upload.
  --commit-message COMMIT_MESSAGE
                        The message of commit. Default to be `None`.
  --token TOKEN         A User Access Token
  --max-workers MAX_WORKERS
                        The number of workers to use for uploading files.

# 以下示例为将本地的README.md文件上传到repo中master分支的根目录内
# 需要填写repo_id（数据集详情页中）和Access Token（在我的控制台--令牌获取）
aistudio upload {repo_id} ./path/to/local/README.md README.md --repo-type dataset --token {access_token}

# 以下示例为将本地folder文件夹中的文件上传至repo中的data目录内
# 需要填写repo_id（数据集详情页中）和Access Token（在我的控制台--令牌获取）
aistudio upload {repo_id} ./path/to/local/folder data/ --repo-type dataset --token {access_token}

# 首次使用需要安装aistudio-sdk库
pip install --upgrade aistudio-sdk

import os
# 需要填写aistudio-access-token, 在我的控制台--令牌获取
os.environ["AISTUDIO_ACCESS_TOKEN"] = "{access_token}"
from aistudio_sdk.hub import upload_file

res = upload_file(
    # 填写数据集详情页面中的repo_id
    repo_id='{repo_id}',
    # 填写要上传的文件在本地的路径，如'./path/to/local/README.md'
    path_or_fileobj='./path/to/local/README.md',
    # 填写上传至repo后的文件路径及文件名，如填写'README.md'，则会在master分支的根目录内，上传README.md
    path_in_repo='README.md',
    # 填写commit信息，非必填
    commit_message='upload dataset file to repo',
    # 填写仓库类型为dataset，上传数据集文件时为必填项
    repo_type = 'dataset'
)
print(res)

import os
# 需要填写aistudio-access-token, 在我的控制台--令牌获取
os.environ["AISTUDIO_ACCESS_TOKEN"] = "{access_token}"
from aistudio_sdk.hub import upload_folder

res = upload_folder(
    # 填写数据集详情页面中的repo_id
    repo_id='{repo_id}',
    # 填写要上传的文件在本地的路径，如'./path/to/local/dir'
    folder_path='./path/to/local/dir',
    # 填写上传至repo后的文件路径，如填写'data/'，则会将文件上传至data目录内；或不填，则默认上传至master分支的根目录内
    path_in_repo='data/',
    # 填写commit信息，非必填
    commit_message='upload dataset folder to repo',
    # 填写仓库类型为dataset，上传数据集文件时为必填项
    repo_type = 'dataset'
)
print(res)

# 请确认已经安装git-lfs (https://git-lfs.com)
git lfs install

# 首先使用git克隆远程仓库到本地，需要填写Access Token（在我的控制台--令牌获取）和repo_id（数据集详情页中）
# 示例：git clone https://6b27a165b0851ae4651fbc5fe09022cd17@git.aistudio.baidu.com/myname/reponame.git
git clone https://{access_token}@git.aistudio.baidu.com/{repo_id}.git

# 进入本地的git仓库目录，如cd dataset_name/
cd "数据集名称"/
# 添加修改过的文件到暂存区，如 git add README.md 添加指定的README.md文件，或 git add . 添加所有修改文件
git add "已修改的文件"
# 将暂存区的修改文件提交到本地仓库，并附带一条简短的提交说明，如 git commit -m "添加README.md文件"
git commit -m "修改描述"
# 推送到远程仓库
git push