двойные кавычки с пробелом через пробел из файла yaml

Question

Я хочу записать YAML-файл в influenxdb.Чтобы сделать это, мне нужно экранировать все специальные символы, как указано ниже.https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_reference/#special-characters

Я пытался использовать команду sed для экранирования, запятой и двойных кавычек.

example: sed 's/ /\\ /g' | sed 's/\"//g' | sed 's/\,//g'

После побега с помощью команды sed он не работал для меня во время записи в influenxdb.

Я хочу записать файл yaml ниже в impxdb, для этого мне нужно экранировать все специальные символы,Я не уверен, как это сделать на yaml.

Существуют ли какие-либо команды, которые автоматически экранируют все специальные символы в файле yaml ??https://raw.githubusercontent.com/starkandwayne/concourse-tutorial/master/ci/pipeline.yml

Упомянутый здесь формат Yaml отлично работал для меня.https://yaml.org/start.html

Answer 1

Вы можете заменить подпрограмму, которая выводит данные обратно в YAML, и явно установить стиль только для значений, а также заменить любой символ, который необходимо экранировать (двойные кавычки уже экранированы библиотекой):

import sys
from pathlib import Path
import ruamel.yaml

pipeline = Path('pipeline.yaml')


def my_represent_dict(self, data):
    try:
        t = data.tag.value
    except AttributeError:
        t = None
    if t:
        if t.startswith('!!'):
            tag = 'tag:yaml.org,2002:' + t[2:]
        else:
            tag = t
    else:
        tag = u'tag:yaml.org,2002:map'
    result = self.represent_mapping(tag, data)
    # the original would now just return result
    for (key_node, value_node) in result.value:
        if value_node.tag != 'tag:yaml.org,2002:str':  # not a string
            continue
        value_node.style = '"'    # hard code the style
        for ch in ',= ':          # escape  
            value_node.value = value_node.value.replace(ch, '\\' + ch)
    return result

ruamel.yaml.representer.RoundTripRepresenter.add_representer(ruamel.yaml.comments.CommentedMap, my_represent_dict)


yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
data = yaml.load(pipeline)
yaml.dump(data, sys.stdout)

, что дает:

meta:
  name: "((\\ param\\ \"Please\\ name\\ your\\ pipeline\"\\ ))"
  team: "((\\ param\\ \"Please\\ provide\\ expected\\ concourse\\ team\\ to\\ deploy\\\
    \ test\\ pipelines\"\\ ))"
  pipeline: "((\\ grab\\ meta.name\\ ))"
  target: "((\\ param\\ \"Please\\ identify\\ the\\ name\\ of\\ the\\ target\\ Concourse\\\
    \ CI\"\\ ))"
  url: "((\\ param\\ \"Please\\ specify\\ the\\ full\\ url\\ of\\ the\\ target\\ Concourse\\\
    \ CI\"\\ ))"

  image:
    name: "starkandwayne/concourse"
    tag: "latest"

  github:
    uri: "((\\ concat\\ \"git@github.com:\"\\ meta.github.owner\\ \"/\"\\ meta.github.repo\\\
      \ ))"
    owner: "((\\ param\\ \"Please\\ specify\\ the\\ name\\ of\\ the\\ user\\ /\\ organization\\\
      \ that\\ owns\\ the\\ Github\\ repository\"\\ ))"
    repo: "((\\ param\\ \"Please\\ specify\\ the\\ name\\ of\\ the\\ Github\\ repository\"\
      \\ ))"
    branch_master: "master"
    branch_develop: "develop"
    private_key: "((\\ param\\ \"Please\\ generate\\ an\\ SSH\\ Deployment\\ Key\\\
      \ for\\ this\\ repo\\ and\\ specify\\ it\\ here\"\\ ))"
    access_token: "((\\ param\\ \"Please\\ generate\\ a\\ Personal\\ Access\\ Token\\\
      \ to\\ be\\ used\\ for\\ creating\\ github\\ releases\\ (do\\ you\\ have\\ a\\\
      \ ci-bot?)\"\\ ))"

  cf:
    api: "https://api.run.pivotal.io"
    skip_cert_check: false
    username: "((\\ param\\ \"please\\ provide\\ cf.username\"\\ ))"
    password: "((\\ param\\ \"please\\ provide\\ cf.password\"\\ ))"
    organization: "((\\ param\\ \"please\\ provide\\ cf.organization\"\\ ))"
    space: "((\\ param\\ \"please\\ provide\\ cf.space-production\"\\ ))"

    master:
      appname: "((\\ grab\\ meta.name\\ ))"
    develop:
      appname: "((\\ concat\\ meta.name\\ \"-develop\"\\ ))"

  concourse:
    url: "((\\ param\\ \"please\\ provide\\ concourse.url\"\\ ))"
    username: "((\\ param\\ \"please\\ provide\\ concourse.username\"\\ ))"
    password: "((\\ param\\ \"please\\ provide\\ concourse.password\"\\ ))"
    ca_cert: "((\\ param\\ \"please\\ provide\\ concourse.ca_cert\"\\ ))"

  credhub:
    url: "((\\ param\\ \"please\\ provide\\ credhub.url\"\\ ))"
    username: "((\\ param\\ \"please\\ provide\\ credhub.username\"\\ ))"
    password: "((\\ param\\ \"please\\ provide\\ credhub.password\"\\ ))"
    ca_cert: "((\\ param\\ \"please\\ provide\\ credhub.ca_cert\"\\ ))"

groups:
- name: "((\\ grab\\ meta.pipeline\\ ))"
  jobs:
  - testflight-develop
  - testflight-master
  - website-master
  - website-develop

jobs:
- name: "testflight-develop"
  public: false
  serial_groups: [tutorial-pipeline]
  plan:
  - get: "git"
    resource: "git-develop"
    trigger: true
  - task: "testflight"
    config:
      platform: "linux"
      image_resource:
        type: "docker-image"
        source:
          repository: "((\\ grab\\ meta.image.name\\ ))"
          tag: "((\\ grab\\ meta.image.tag\\ ))"
      inputs:
      - {name: "git"}
      run:
        path: "./git/ci/scripts/testflight-with-credhub.sh"
        args: []
      params:
        REPO_ROOT: "git"
        FLY_URL: "((\\ grab\\ meta.concourse.url\\ ))"
        FLY_TEAM: "((\\ grab\\ meta.team\\ ))"
        FLY_USERNAME: "((\\ grab\\ meta.concourse.username\\ ))"
        FLY_PASSWORD: "((\\ grab\\ meta.concourse.password\\ ))"
        FLY_CACERT: "((\\ grab\\ meta.concourse.ca_cert\\ ))"
        CREDHUB_URL: "((\\ grab\\ meta.credhub.url\\ ))"
        CREDHUB_USERNAME: "((\\ grab\\ meta.credhub.username\\ ))"
        CREDHUB_PASSWORD: "((\\ grab\\ meta.credhub.password\\ ))"
        CREDHUB_CACERT: "((\\ grab\\ meta.credhub.ca_cert\\ ))"
- name: "testflight-master"
  public: false
  serial_groups: [tutorial-pipeline]
  plan:
  - get: "git"
    resource: "git-master"
    trigger: true
  - task: "testflight"
    config:
      platform: "linux"
      image_resource:
        type: "docker-image"
        source:
          repository: "((\\ grab\\ meta.image.name\\ ))"
          tag: "((\\ grab\\ meta.image.tag\\ ))"
      inputs:
      - {name: "git"}
      run:
        path: "./git/ci/scripts/testflight-with-credhub.sh"
        args: []
      params:
        REPO_ROOT: "git"
        FLY_URL: "((\\ grab\\ meta.concourse.url\\ ))"
        FLY_TEAM: "((\\ grab\\ meta.team\\ ))"
        FLY_USERNAME: "((\\ grab\\ meta.concourse.username\\ ))"
        FLY_PASSWORD: "((\\ grab\\ meta.concourse.password\\ ))"
        FLY_CACERT: "((\\ grab\\ meta.concourse.ca_cert\\ ))"
        CREDHUB_URL: "((\\ grab\\ meta.credhub.url\\ ))"
        CREDHUB_USERNAME: "((\\ grab\\ meta.credhub.username\\ ))"
        CREDHUB_PASSWORD: "((\\ grab\\ meta.credhub.password\\ ))"
        CREDHUB_CACERT: "((\\ grab\\ meta.credhub.ca_cert\\ ))"

- name: "website-master"
  public: false
  serial: true
  plan:
  - get: "git"
    resource: "git-master"
    trigger: true
  - task: "mkdocs-build"
    config:
      platform: "linux"
      image_resource:
        type: "docker-image"
        source:
          repository: "((\\ grab\\ meta.image.name\\ ))"
          tag: "((\\ grab\\ meta.image.tag\\ ))"
      inputs:
      - {name: "git"}
      outputs:
      - {name: "site"}
      run:
        path: "./git/ci/scripts/mkdocs-build.sh"
        args: []
      params:
        REPO_ROOT: "git"
        SITE_ROOT: "site"
  - put: "deploy-master"
    params:
      manifest: "site/manifest-production.yml"
      path: "site"
      current_app_name: "((\\ grab\\ meta.cf.master.appname\\ ))"

- name: "website-develop"
  public: false
  serial: true
  plan:
  - get: "git"
    resource: "git-develop"
    trigger: true
  - task: "mkdocs-build"
    config:
      platform: "linux"
      image_resource:
        type: "docker-image"
        source:
          repository: "((\\ grab\\ meta.image.name\\ ))"
          tag: "((\\ grab\\ meta.image.tag\\ ))"
      inputs:
      - {name: "git"}
      outputs:
      - {name: "site"}
      run:
        path: "./git/ci/scripts/mkdocs-build.sh"
        args: []
      params:
        REPO_ROOT: "git"
        SITE_ROOT: "site"
  - put: "deploy-develop"
    params:
      manifest: "site/manifest-develop.yml"
      path: "site"
      current_app_name: "((\\ grab\\ meta.cf.develop.appname\\ ))"


resources:
- name: "git-develop"
  type: "git"
  source:
    uri: "((\\ grab\\ meta.github.uri\\ ))"
    branch: "((\\ grab\\ meta.github.branch_develop\\ ))"
    private_key: "((\\ grab\\ meta.github.private_key\\ ))"

- name: "git-master"
  type: "git"
  source:
    uri: "((\\ grab\\ meta.github.uri\\ ))"
    branch: "((\\ grab\\ meta.github.branch_master\\ ))"
    private_key: "((\\ grab\\ meta.github.private_key\\ ))"

- name: "deploy-develop"
  type: "cf"
  source:
    api: "((\\ grab\\ meta.cf.api\\ ))"
    skip_cert_check: "((\\ grab\\ meta.cf.skip_cert_check\\ ))"
    username: "((\\ grab\\ meta.cf.username\\ ))"
    password: "((\\ grab\\ meta.cf.password\\ ))"
    organization: "((\\ grab\\ meta.cf.organization\\ ))"
    space: "((\\ grab\\ meta.cf.space\\ ))"

- name: "deploy-master"
  type: "cf"
  source:
    api: "((\\ grab\\ meta.cf.api\\ ))"
    skip_cert_check: "((\\ grab\\ meta.cf.skip_cert_check\\ ))"
    username: "((\\ grab\\ meta.cf.username\\ ))"
    password: "((\\ grab\\ meta.cf.password\\ ))"
    organization: "((\\ grab\\ meta.cf.organization\\ ))"
    space: "((\\ grab\\ meta.cf.space\\ ))"

Вы, конечно, можете указать экземпляр Path() в качестве второго параметра для .dump() для записи в файл.