尝试为gitlab.com ..gitlab ci.yaml文件向去皮 api发出curl请求,但收到错误的请求响应:{"status":400,"error":"Bad Request"}
#!/usr/bin/env bash
PAYLOAD=$( cat << JSON
{ "content":
$(<$PWD/../.gitlab-ci.yml)
JSON
)
echo "Payload is $PAYLOAD"
curl --include --show-error --request POST --header "Content-Type: application/json" --header "Accept: application/json" "https://gitlab.com/api/v4/ci/lint" --data-binary "$PAYLOAD"
有没有人通过bash脚本成功地将..gitlab ci.yml链接起来?还尝试将内容有效负载封装在大括号中,并接收相同的响应。
更新
我认为正在发生的情况是,GitLab CI端点希望将.GitLab文件的内容为POST请求转换为json。请参阅这里
修改脚本,在发送之前使用ruby将yaml转换为json,这适用于simple ..gitlab ci.yml。但是,当我的项目使用yaml文件时,它会出现一个错误:当我使用gitlab网页链接文件时,{"status":"invalid","errors":["(\u003cunknown\u003e): did not find expected ',' or ']' while parsing a flow sequence at line 1 column 221"]}%
是有效的。
{"content": "{ \"stages\": [ \"build\", \"test\", \"pages\", \"release\" ], \"variables\": { \"DOCKER_DRIVER\": \"overlay2\" }, \"services\": [ \"docker:19.03.11-dind\" ], \"build:plugin\": { \"image\": \"docker:19.03.11\", \"stage\": \"build\", \"before_script\": [ \"echo \"$CI_JOB_TOKEN\" | docker login -u gitlab-ci-token --password-stdin \"$CI_REGISTRY\"\" ].....
在上面的json摘录中,列221是\"image\": \"docker:19.03.11\"
,特别是在结束转义引号处。你认为不正确的转义引号是个问题吗?
#!/usr/bin/env bash
json=$(ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))' < .gitlab-ci.yml)
# escape quotes
json_content=$(echo $json | perl -pe 's/(?<!\\)"/\\"/g')
# Add object contect for GitLab linter
json_content='{"content": "'${json_content}'"}'
echo "${json_content}"
curl --include --show-error --request POST \
--header "Content-Type: application/json" \
--header "Accept: application/json" \
"https://gitlab.com/api/v4/ci/lint" \
--data-binary "$json_content"
第二次更新
使用上面的bash脚本,这个yaml文件:
stages:
- test
test:
stage: test
script:
- echo "test"
转换为此json:
{"content": "{ \"stages\": [ \"test\" ], \"test\": { \"stage\": \"test\", \"script\": [ \"echo \"test\"\" ] } }"}
当它被发送到api时,接收以下json错误响应:
{"status":"invalid","errors":["(\u003cunknown\u003e): did not find expected ',' or ']' while parsing a flow sequence at line 1 column 62"]}%
发布于 2020-07-06 03:33:17
最后,使用以下脚本使其工作:
#!/usr/bin/env bash
json=$(ruby -ryaml -rjson -e 'puts(YAML.load(ARGF.read).to_json)' custom_hooks/valid.yml)
# escape quotes
json_content=$(echo $json | python -c 'import json,sys; print(json.dumps(sys.stdin.read()))')
echo $json_content
# Add object contect for GitLab linter
json_content="{\"content\": ${json_content}}"
# Output escaped content to file
echo $json_content > custom_hooks/input.json
echo "Escaped json content written to file input.json"
curl --include --show-error --request POST \
--header "Content-Type: application/json" \
--header "Accept: application/json" \
"https://gitlab.com/api/v4/ci/lint" \
--data-binary "$json_content"
注:B将调整脚本,从系统args中读取文件,而不是固定的文件位置--定制_hooks/valid.yml。另外,JSON响应需要使用jq或python / ruby命令shell进行解析。包括这个脚本的偶然机会,它将帮助他人。
问题是,最初我将文件的YAML内容直接发送到api:
{ "content": { <contents of .gitlab-yml> } }
GitLab似乎接受在其API中转换为转义JSON字符串的YAML。因此使用ruby将yaml转换为JSON,然后使用python转义由ruby生成的结果JSON。最后能够使用curl将转义的GitLab字符串发送到JSON以验证.
不确定Ruby是否有相当于python的json.dumps的东西..。但是这个解决方案允许我验证gitlab-ci.下一阶段连接到git预提交钩子/服务器端预接收(如果可能的话!)若要防止无效的..gitlab ci.yml文件破坏CI管道。
新手到ruby...since发布了最初的答案,尝试创建一个可以从预提交钩子中使用的ruby脚本,现在只需要bash和ruby:
#!/usr/bin/env ruby
require 'json'
require 'net/http'
require 'optparse'
require 'yaml'
=begin
POST to GitLab api for linting ci yaml
Params:
+url+ :: Api url
+yaml+ :: Yaml payload for linting
Returns:
Json validation result from API for HTTP response Success
Aborts with HTTP Message for all other status codes
=end
def call_api(url, yaml)
uri = URI.parse(url)
req = Net::HTTP::Post.new(uri)
req.content_type='application/json'
req['Accept']='application/json'
req.body = JSON.dump({"content" => yaml.to_json})
https = Net::HTTP.new(uri.host, uri.port)
https.use_ssl = true
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
response = https.request(req)
case response
when Net::HTTPSuccess
puts "request successful"
return JSON.parse response.body
when Net::HTTPUnauthorized
abort("#{response.message}: invalid token in api request?")
when Net::HTTPServerError
abort('error' => "#{response.message}: server error, try again later?")
when Net::HTTPBadRequest
puts "Bad request..." + request.body
abort("#{response.message}: bad api request?")
when Net::HTTPNotFound
abort("#{response.message}: api request not found?")
else
puts "Failed validation\nJSON payload :: #{request.body}\nHTTP Response: #{response.message}"
abort("#{response.message}: failed api request?")
end
end
=begin
Display exit report and raise the appropriate system exit code
Params:
+status+ :: Validation status string. Legal values are valid or invalid
+errors+ :: String array storing errors if yaml was reported as invalid
Returns:
Exits with 0 when successful
Exits with 1 on validation errors or fails to parse legal status value
=end
def exit_report(status, errors)
case status
when "valid"
puts ".gitlab-ci.yml is valid"
exit(0)
when "invalid"
abort(".gitlab-ci.yml is invalid with errors:\n\n" + errors.join("\n"))
else
abort("A problem was encountered parsing status : " + status)
end
end
=begin
Load yaml file from path and return contents
Params:
+path+ :: Absolute or relative path to .gitlab-ci.yml file
=end
def load_yaml(path)
begin
YAML.load_file(path)
rescue Errno::ENOENT
abort("Failed to load .gitlab-ci.yml")
end
end
=begin
Parse command line options
Returns:
Hash containing keys: {:yaml_file,:url}
=end
def read_args()
options = {}
OptionParser.new do |opt|
opt.on('-f', '--yaml YAML-PATH', 'Path to .gitlab-ci.yml') { |o| options[:yaml_file] = o }
opt.on('-l', '--url GitLab url', 'GitLab API url') { |o| options[:url] = o }
end.parse!
options
end
=begin
Load yaml to send to GitLab API for linting
Display report of linting retrieved from api
Returns:
Exits with 0 upon success and 1 when errors encountered
=end
def main()
# try and parse the arguments
options = read_args()
unless !options.has_key?(:yaml_file) || !options.has_key?(:url)
# try and load the yaml from path
puts "Loading file #{options[:yaml_file]}"
yaml = load_yaml(options[:yaml_file])
# make lint request to api
puts "Making POST request to #{options[:url]}"
response_data=call_api(options[:url], yaml)
# display exit report and raise appropriate exit code
unless !response_data.has_key?("status") || !response_data.has_key?("errors")
exit_report response_data["status"], response_data["errors"]
else
puts "Something went wrong parsing the json response " + response_data
end
else
abort("Missing required arguments yaml_file and url, use -h for usage")
end
end
# start
main
https://stackoverflow.com/questions/62719563
复制