MinIO是一个基于Apache License v2.0开源协议的对象存储服务。它兼容亚马逊S3云存储服务接口，非常适合于存储大容量非结构化的数据，例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等，而一个对象文件可以是任意大小，从几KB到最大5T不等。

开始运行

直接下载二进制文件运行即可，当然也可以自己从源码进行编译。
这里只是简单的运行起来，不考虑额外配置，冗余纠错什么的。

Linux：
	wget https://dl.min.io/server/minio/release/linux-ppc64le/minio
	chmod +x minio
	./minio server /data
Windows：
	https://dl.min.io/server/minio/release/windows-amd64/minio.exe
	minio.exe server D:\Photos

MinIO Server带有一个嵌入的Web对象浏览器，安装后使用浏览器访问 http://127.0.0.1:9000，如果可以访问，则表示minio已经安装成功。可以通过这个页面对服务器进行操作，或者也可以通过MinIO client操作。
启动时，MinIO会将环境变量MINIO_ROOT_USER和MINIO_ROOT_PASSWORD的值作为root凭证，如不指定，均为minioadmin。

因为MinIO与Amazon S3兼容，所以两者的SDK能换着用。关于REST API，MinIO的文档没介绍，直接参考S3也是一样的。
下面尝试REST API中的GET，PUT，DELETE方法，貌似MinIO不支持POST，但S3是支持的。

单块传输的签名方法

该节内容大都摘自Signature Calculations for the Authorization Header: Transferring Payload in a Single Chunk (AWS Signature Version 4)。
the process of calculating the seed signature

FunctionDescription
Lowercase()Convert the string to lowercase.
Hex()Lowercase base 16 encoding.
SHA256Hash()Secure Hash Algorithm (SHA) cryptographic hash function.
HMAC-SHA256()Computes HMAC by using the SHA256 algorithm with the signing key provided. This is the final signature.
Trim()Remove any leading or trailing whitespace.
UriEncode()URI encode every byte.

SHA256：对于任意长度的消息，SHA256都会产生一个256bit长的哈希值，称作消息摘要（Message Digest ）。

HMAC：Hash-based Message Authentication Code，是一种使用密码散列函数，同时结合一个加密密钥，通过特别计算方式之后产生的消息认证码。它可以用来保证数据的完整性，同时可以用来作某个消息的身份验证。

UriEncode() must enforce the following rules:
• URI encode every byte except the unreserved characters: ‘A’-‘Z’, ‘a’-‘z’, ‘0’-‘9’, ‘-’, ‘.’, ‘_’, and ‘~’.
• The space character is a reserved character and must be encoded as “%20” (and not as “+”).
• Each URI encoded byte is formed by a ‘%’ and the two-digit hexadecimal value of the byte.
• Letters in the hexadecimal value must be uppercase, for example “%1A”.
• Encode the forward slash character, ‘/’, everywhere except in the object key name. For example, if the object key name is photos/Jan/sample.jpg, the forward slash in the key name is not encoded.

The standard UriEncode functions provided by your development platform may not work because of differences in mplementation and related ambiguity in the underlying RFCs. We recommend that you write your own custom UriEncode function to ensure that your encoding will work.
The following is an example UriEncode() function in Java.

public static String UriEncode(CharSequence input, boolean encodeSlash) {
	StringBuilder result = new StringBuilder();
	for (int i = 0; i < input.length(); i++) {
		char ch = input.charAt(i);
		if ((ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')
		|| ch == '_' || ch == '-' || ch == '~' || ch == '.') {
			result.append(ch);
		} else if (ch == '/') {
			result.append(encodeSlash ? "%2F" : ch);
		} else {
			result.append(toHexUTF8(ch));
		}
	}
	return result.toString();
}

关于payload：
When transferring payload in a single chunk, you can optionally choose to include the payload hash in the signature calculations, referred as signed payload (if you don’t include it, the payload is considered unsigned). The signing procedure discussed in the following section applies to both, but note the following differences:

Signed payload option – You include the payload hash when constructing the canonical request (that then becomes part of StringToSign, as explained in the signature calculation section). You also specify the same value as the x-amz-content-sha256 header value when sending the request to S3.

Unsigned payload option – You include the literal string UNSIGNED-PAYLOAD when constructing a canonical request, and set the same value as the x-amz-content-sha256 header value when sending the request to Amazon S3.

When you send your request to Amazon S3, the x-amz-content-sha256 header value informs Amazon S3 whether the payload is signed or not. Amazon S3 can then create signature accordingly for verification.

简单起见，后面都用UNSIGNED-PAYLOAD。

通过shell操作

下面的代码修改自 shell实现s3 GET PUT（authv4）。

#!/bin/bash

usage() {
	echo $0 [method] [uri] [file]
	echo $0 GET /bucket/object output_file
	echo $0 PUT /bucket/object input_file
}

if [[ $# == 0 ]]; then
	usage
	exit 0
fi

method=$1
uri=$2
file=$3

secret_id="minioadmin"
secret_key="minioadmin"
host="154.8.199.216:9000"
region=""

x_amz_date=`date --date="8 hour ago" "+%Y%m%dT%H%M%SZ"`
date_day=`date --date="8 hour ago" "+%Y%m%d"`
signed_headers="host;x-amz-date"

sha256() {
	globaldgst=$(echo -en $1 | openssl dgst -sha256 | awk '{print $2}')
}
hmacsha256() {
	local key=$1
	globaldgst=$(echo -en $2 | openssl dgst -sha256 -hmac $key | awk '{print $2}')
}
hmacsha256hexkey() {
	local key=$1
	globaldgst=$(echo -en $2 | openssl dgst -sha256 -mac HMAC -macopt hexkey:$key | awk '{print $2}')
}

create_canonical_request() {
	local canonical_headers="host:${host}\nx-amz-date:${x_amz_date}\n"
	local payload_hash="UNSIGNED-PAYLOAD"
	canonical_request="${method}\n${uri}\n\n${canonical_headers}\n${signed_headers}\n${payload_hash}"
}

create_string_to_sign() {
	create_canonical_request
	local credential_scope="${date_day}/${region}/s3/aws4_request"
	sha256 $canonical_request
	string_to_sign="AWS4-HMAC-SHA256\n${x_amz_date}\n${credential_scope}\n${globaldgst}"
}

create_signing_key() {
	hmacsha256 AWS4$secret_key $date_day
	hmacsha256hexkey $globaldgst $region
	hmacsha256hexkey $globaldgst s3
	hmacsha256hexkey $globaldgst aws4_request
	signing_key=$globaldgst
}

send_request() {
	create_string_to_sign
	create_signing_key

	echo "canonical_request:"
	echo -e $canonical_request
	echo ----------------
	echo "string_to_sign:"
	echo -e $string_to_sign
	echo ----------------
	echo "signing_key:"
	echo -e $signing_key
	echo ----------------
	echo

	hmacsha256hexkey $signing_key $string_to_sign
	local signature=$globaldgst
	local credential="$secret_id/$date_day/$region/s3/aws4_request"

	if [ "$method" == "PUT" ]; then
		local file_size=$(stat -c%s "$file")
		curl -vX PUT "http://${host}${uri}" \
		-H "Host: ${host}" \
		-H "Content-Length: ${file_size}" \
		-H "Authorization: AWS4-HMAC-SHA256 Credential=${credential}, SignedHeaders=${signed_headers}, Signature=${signature}" \
		-H "x-amz-content-sha256: UNSIGNED-PAYLOAD" \
		-H "x-amz-date: ${x_amz_date}" \
		-T "${file}" 
	elif [ "$method" == "GET" ]; then
		if [[ ! -z $file ]]; then
			outparam="-o $file"
		fi
		curl -vsX GET "http://${host}${uri}" \
		-H "Host: ${host}" \
		-H "Authorization: AWS4-HMAC-SHA256 Credential=${credential}, SignedHeaders=${signed_headers}, Signature=${signature}" \
		-H "x-amz-content-sha256: UNSIGNED-PAYLOAD" \
		-H "x-amz-date: ${x_amz_date}" \
		$outparam
	else 
		usage
	fi	
}

send_request
echo

默认情况下启动的MinIO的region为空，所以脚本中的region为空，但我试了试，region随便填什么都行。
或许因为region为空，MinIO的response header中显示服务器时间是0时区的时间，并提示请求时间和服务器时间差距过大，所以请求时把北京时间转换成了0时区时间。

通过c++操作

该节使用 aws-sigv4-c 和 cpp-httplib 实现。

extern "C" {
#include "aws_sigv4.h"
}
#include "httplib.h"
#include <iostream>

using namespace std;

int main(void)
{
	time_t t_time;
	struct tm *ptm;
	time(&t_time);
	ptm = gmtime(&t_time);
	char curtime[64];
	sprintf(curtime, "%04d%02d%02dT%02d%02d%02dZ", ptm->tm_year+1900,
			ptm->tm_mon+1, ptm->tm_mday, ptm->tm_hour, ptm->tm_min, ptm->tm_sec);

	unsigned char *auth;
	int signcode = aws_sigv4_sign_with_cstr(
			(unsigned char*)"minioadmin", 
			(unsigned char*)"minioadmin", 
			(unsigned char*)"GET", 
			(unsigned char*)"/bt1/build.sh", 
			(unsigned char*)"",
			(unsigned char*)"154.8.199.216:9000", 
			(unsigned char*)curtime,
			(unsigned char*)"UNSIGNED-PAYLOAD", 
			(unsigned char*)"s3", 
			(unsigned char*)"",
			&auth);

	if(signcode != AWS_SIGV4_OK)
	{
		cout << "sign error code is " << signcode << endl;
		return 0;
	}

	cout << "signature is " << auth << endl;

	httplib::Headers headers = {
		{ "Host", "154.8.199.216:9000" },
		{ "Authorization", (char*)auth },
		{ "x-amz-content-sha256", "UNSIGNED-PAYLOAD" },
		{ "x-amz-date", curtime}
	};

	free(auth);

	httplib::Client cli("http://154.8.199.216:9000");
	cli.set_connection_timeout(0, 300000); // 300 milliseconds
	cli.set_read_timeout(5, 0); // 5 seconds
	cli.set_write_timeout(5, 0); // 5 seconds

	//res = cli.Put("/resource/foo", "text", "text/plain");
	//res = cli.Delete("/resource/foo");

	if (auto res = cli.Get("/bt1/build.sh", headers))
	{
		if (res->status == 200)
		{
			std::cout << res->body << std::endl;
		}
		else
		{
			cout << "status is " << res->status << endl;
			std::cout << res->body << std::endl;
		}
	}
	else
	{
		auto err = res.error();
		cout << "error " << err << endl;
	}
	return 0;
}

需要对awssigv4做一些修改：

--- a/aws_sigv4/lib/aws_sigv4.c
+++ b/aws_sigv4/lib/aws_sigv4.c
@@ -190,9 +190,18 @@ void get_canonical_request(aws_sigv4_params_t* sigv4_params,
   str += signed_headers.len;
   *(str++) = '\n';

+  unsigned int uplen = strlen("UNSIGNED-PAYLOAD");
+  if(sigv4_params->payload.len == uplen && strncmp(sigv4_params->payload.data, "UNSIGNED-PAYLOAD", uplen) == 0)
+  {
+  strcpy(str, "UNSIGNED-PAYLOAD");
+  str += uplen;
+  }
+  else
+  {
   aws_sigv4_str_t hex_sha256 = { .data = str };
   get_hex_sha256(&sigv4_params->payload, &hex_sha256);
   str += hex_sha256.len;
+  }

   canonical_request->len = str - canonical_request->data;
 }
@@ -224,7 +233,7 @@ int aws_sigv4_sign(aws_sigv4_params_t* sigv4_params, aws_sigv4_header_t* auth_he
       || aws_sigv4_empty_str(&sigv4_params->uri)
       || aws_sigv4_empty_str(&sigv4_params->host)
       || aws_sigv4_empty_str(&sigv4_params->x_amz_date)
-      || aws_sigv4_empty_str(&sigv4_params->region)
+      //|| aws_sigv4_empty_str(&sigv4_params->region)
       || aws_sigv4_empty_str(&sigv4_params->service))
   {
     rc = AWS_SIGV4_INVALID_INPUT_ERROR;
diff --git a/aws_sigv4/lib/aws_sigv4_common.c b/aws_sigv4/lib/aws_sigv4_common.c
index da28af7..5ccca43 100644
--- a/aws_sigv4/lib/aws_sigv4_common.c
+++ b/aws_sigv4/lib/aws_sigv4_common.c
@@ -41,7 +41,7 @@ static unsigned char* aws_sigv4_vslprintf(unsigned char* buf, unsigned char* las
         str = va_arg(args, aws_sigv4_str_t *);
         if (aws_sigv4_empty_str(str))
         {
-          goto finished;
+          //goto finished;
         }
         size_t cp_len = n_max >= str->len ? str->len : n_max;
         strncpy((char*) c_ptr, (char*) str->data, cp_len);

第一处修改是为了让其支持UNSIGNED-PAYLOAD，第二三处修改是为了让其支持空region。

awssigv4库是c的，用g++编译很多错误，只好先将aws_sigv4_common.c，aws_sigv4.c编成静态库：
gcc -c aws_sigv4_common.c aws_sigv4.c
ar -r libawssigv4.a aws_sigv4_common.o aws_sigv4.o
再用g++编译：g++ client.cpp libawssigv4.a -lssl -lcrypto

哦对了，这个http库需要正则库的支持，还得把gcc升级到5.0以上。
用c++写就贼费劲，MinIO就没提供c++的SDK，Amazon提供了，但安装很麻烦。

顺便了解到一种结构体初始化的方式（摘自C++ 结构体初始化与赋值）：
指定初始化（Designated Initializer）实现上有两种方式，一种是通过点号加赋值符号实现，即“.fieldname=value”，另外一种是通过冒号实现，即“fieldname:value”，其中 fieldname 为结构体成员名称 。前者是 C99 标准引入的初始化方式，后者是 GCC 的扩展。遗憾的是有些编译器并不支持指定初始化，比如 Visual C++。
struct A {
int b;
int c;
};
// 点号+赋值符号
A a = {.b = 1, .c = 2};
//冒号
A a = {b:1, c:2};

匿名访问存储桶和对象

后来发现，可以为buckets和objects设定匿名访问策略，简单测试的话挺好。
允许的策略有none, download, upload, public。默认是none，download表示可以匿名下载，upload表示可以匿名上传，public表示可以匿名上传下载。

设置一个别名，方便使用
# mc alias set minio http://154.8.199.216:9000 minioadmin minioadmin

创建images桶
# mc mb minio/images
Bucket created successfully `minio/images.

设置images桶可匿名上传下载
# mc policy set public minio/images/
Access permission for `minio/images/` is set to `public`

使用Webhook发布MinIO事件

可以使用存储桶事件通知来监视存储桶中对象上发生的事件。 存储桶事件可以发布到多种目标，这里使用Webhook方式监听文件上传事件。

查看当前notify_webhook状态：
# mc admin config get minio notify_webhook
notify_webhook enable=off endpoint= auth_token= queue_limit=0 queue_dir= client_cert= client_key=

更新配置，然后重启MinIO服务让配配置生效. 配置和重启MinIO时，endpoint必须是可访问的状态
# mc admin config set minio notify_webhook:1 endpoint="http://127.0.0.1:8090"
# mc admin service restart minio

在一个叫images的存储桶上开启事件通知，一旦上有文件上传到存储桶中或从桶中删除，事件将被触发
# mc event add minio/images arn:minio:sqs::1:webhook --event put,delete

查看配置
# mc event list minio/images
arn:minio:sqs::1:webhook   s3:ObjectCreated:*,s3:ObjectRemoved:*   Filter:

本来找到一个提供webhook服务的开源项目 webhookit，但安装时需要联网下载一些依赖，在内网测试就没办法了，所以用wsgiref库写了个简单的：

#!/usr/bin/env python
# -*- coding: utf-8 -*-
def app(environ, start_response):
	try:
		request_body_size = int(environ.get('CONTENT_LENGTH', 0))
	except(ValueError):
		request_body_size = 0

	if(request_body_size > 0):
		request_body = environ['wsgi.input'].read(request_body_size)
		print(request_body)

	start_response('200 OK', [('Content-Type', 'text/html')])
	return ['']

from wsgiref.simple_server import make_server
httpd = make_server('', 8090, app)
print('serving on 8090...')
httpd.serve_forever()

参考

Amazon S3 REST API
Android端AWS s3 V4签名算法
AWS V4签名实现(C版本)
MinIO中国镜像
MinIO存储桶通知指南
WSGI简介

Logo
华为开发者空间

华为开发者空间，是为全球开发者打造的专属开发空间，汇聚了华为优质开发资源及工具，致力于让每一位开发者拥有一台云主机，基于华为根生态开发、创新。

更多推荐

cover

一文解读GaussDB(DWS)监控运维诊断优化能力

avatar 华为开发者空间
cover

GaussDB技术解读——GaussDB架构介绍（四）

avatar 华为开发者空间
cover

华为云数据仓库专业级开发者认证重磅发布

avatar 华为开发者空间