SpringBoot整合MinIO分布式文件存储
Minio 是个基于 Golang 编写的开源对象存储套件,虽然轻量,却拥有着不错的性能。中文文档对应的是上个版本,新版本中有些内容已发生了变化,所以最好是看英文文档。何为对象存储?对象存储服务( Object Storage Service,OSS )是一种海量、安全、低成本、高可靠的云存储服务,适合存放任意类型的文件。容量和处理能力弹性扩展,多种存储类型供选择,全面优化存储成本。对于中小型企业
MinIO
1.MinIO安装
Minio 是个基于 Golang 编写的开源对象存储套件,虽然轻量,却拥有着不错的性能。
-
官网地址:MinIO | High Performance, Kubernetes Native Object Storageopen in new window
-
官网文档( 中文 )地址:官网中文网址open in new window 中文文档对应的是上个版本,新版本中有些内容已发生了变化,所以最好是看英文文档。
-
JAVA SDK API:minio java sdk api 文档
何为对象存储?
对象存储服务( Object Storage Service,OSS )是一种海量、安全、低成本、高可靠的云存储服务,适合存放任意类型的文件。容量和处理能力弹性扩展,多种存储类型供选择,全面优化存储成本。
对于中小型企业,如果不选择存储上云,那么 Minio 是个不错的选择,麻雀虽小,五脏俱全
1.1 CentOS7安装
1.下载MinIO,并创建目录
mkdir /usr/local/minio && cd /usr/local/minio && mkdir bin data wget https://dl.min.io/server/minio/release/linux-amd64/minio bin
在usr/local下创建minio文件夹,并在minio文件里面创建bin和data目录,把下载好的minio文件拷贝到bin目录里面
2.赋予它可执行权限
chmod +x bin/minio
3.运行
./bin/minio server ./data
4.将 minio 添加成 Linux 的服务
cat > /etc/systemd/system/minio.service << EOF
[Unit]
Description=Minio
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/minio/bin/minio
[Service]
WorkingDirectory=/usr/local/minio/
PermissionsStartOnly=true
ExecStart=/usr/local/minio/bin/minio server /usr/local/minio/data
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
[Install]
WantedBy=multi-user.target
EOF
这样就可以使用 systemctl 启停 minio
systemctl start minio # 启动 systemctl stop minio # 停止
MinIO Server 成功启动后访问 [http://192.168.128.128:9000],你会看到类似如下界面:
正在上传…重新上传取消
输入用户名/密码 minioadmin/minioadmin
可以进入 web 管理系统:
正在上传…重新上传取消
刚打开的时候,是没有bucket桶,可以手动或者通过java代码来创建一个桶。
1.1.1 .权限修改
可以上传,配置代理地址,设置公有访问权限以后,直接打开地址访问不了
预览不了
直接在浏览器打开显示无权限
This XML file does not appear to have any style information associated with it. The document tree is shown below.
创建的桶默认的权限时private私有的,外部不能访问,你可以修改桶的属性,点击manage,找到Access Policy,修改权限为public即可。
1.2 Docker安装
一.docker拉取镜像 docker pull minio 二.创建 docker容器,启动服务容器 docker run -id --name=minio -p 9090:9000 -e "MINIO_PROMETHEUS_AUTH_TYPE=public" \ -e "MINIO_ROOT_USER=minioadmin" -e "MINIO_ROOT_PASSWORD=minioadmin" \ -v /mnt/minio/data:/data -v /mnt/minio/config:/root/.minio minio/minio server /data \ --console-address ":9000" --address ":9090" 单行模式 docker run -p 9000:9000 -p 9090:9090 --net=host --name minio -d --restart=always -e "MINIO_ACCESS_KEY=minioadmin" -e "MINIO_SECRET_KEY=minioadmin" -v /home/minio/data:/data -v /home/minio/config:/root/.minio minio/minio server /data --console-address ":9090" -address ":9000"
--console-address ":9000" --address ":9090" docker使用静态的固定端口,以避免启动时使用随机端口
浏览器访问:192.168.128.128:9090
2.MinIO Java SDK
1.引入相关依赖
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.8.1</version> <!-- minio 依赖于 okhttp 且版本较高。注意,spring-boot-dependencies 中的不够高 -->
</dependency>
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.3.9</version>
</dependency>
2.测试
@SpringBootTest
class DemoApplicationTests {
@Test
public void demo() throws Exception {
// 使用 MinIO 服务的 URL 和端口,Access key 和 Secret key 创建一个 MinioClient 对象。
MinioClient minioClient = MinioClient.builder()
.endpoint("http://127.0.0.1:9000")
.credentials("minioadmin", "minioadmin")
.build();
// 检查存储桶是否已经存在
boolean isExist = minioClient.bucketExists(BucketExistsArgs.builder().bucket("abc").build());
if (isExist) {
System.out.println("Bucket already exists.");
} else {
// 创建一个名为 asiatrip 的存储桶,用于存储文件。
minioClient.makeBucket(MakeBucketArgs.builder().bucket("abc").build());
}
// 使用 putObject 上传一个文件到存储桶中。
File file = new File("e:/BluceLee/1.jpg");
InputStream inputStream = new FileInputStream(file);
PutObjectArgs args = PutObjectArgs.builder()
.bucket("abc")
.object("xiaolong.jpg")
.contentType("image/jpg")
.stream(inputStream, inputStream.available(), -1)
.build();
minioClient.putObject(args);
}
}
3.springboot整合MinIO
1.引入依赖
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.8.1</version> <!-- minio 依赖于 okhttp 且版本较高。注意,spring-boot-dependencies 中的不够高 -->
</dependency>
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.3.9</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.70</version>
</dependency>
2.相关配置
server: port: 8081 spring: application: name: demo servlet: multipart: max-file-size: 200MB #设置单个文件的大小 因为springboot内置tomact的的文件传输默认为10MB max-request-size: 500MB #设置单次请求的文件总大小 enabled: true #千万注意要设置该参数,否则不生效 # minio 文件存储配置信息 minio: endpoint: http://192.168.128.128:9000 accesskey: minioadmin secretKey: minioadmin
3.minio配置类和配置属性
@Data
@Component
@ConfigurationProperties(prefix = "minio")
public class MinioProp {
private String endpoint;
private String accesskey;
private String secretKey;
}
=============================================
@Configuration
@EnableConfigurationProperties(MinioProp.class)
public class MinioConfig {
@Autowired
private MinioProp minioProp;
@Bean
public MinioClient minioClient() throws Exception {
return MinioClient.builder().endpoint(minioProp.getEndpoint())
.credentials(minioProp.getAccesskey(), minioProp.getSecretKey()).build();
}
}
4.编写简单文件上传工具类
@Slf4j
@Component
public class MinioUtils {
@Autowired
private MinioClient client;
@Autowired
private MinioProp minioProp;
/**
* 创建bucket
*/
public void createBucket(String bucketName) {
BucketExistsArgs bucketExistsArgs = BucketExistsArgs.builder().bucket(bucketName).build();
MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket(bucketName).build();
try {
if (client.bucketExists(bucketExistsArgs))
return;
client.makeBucket(makeBucketArgs);
} catch (Exception e) {
log.error("创建桶失败:{}", e.getMessage());
throw new RuntimeException(e);
}
}
/**
* @param file 文件
* @param bucketName 存储桶
* @return
*/
public JSONObject uploadFile(MultipartFile file, String bucketName) throws Exception {
JSONObject res = new JSONObject();
res.put("code", 0);
// 判断上传文件是否为空
if (null == file || 0 == file.getSize()) {
res.put("msg", "上传文件不能为空");
return res;
}
// 判断存储桶是否存在
createBucket(bucketName);
// 文件名
String originalFilename = file.getOriginalFilename();
// 新的文件名 = 存储桶名称_时间戳.后缀名
String fileName = bucketName + "_" + System.currentTimeMillis() + originalFilename.substring(originalFilename.lastIndexOf("."));
// 开始上传
InputStream inputStream = file.getInputStream();
PutObjectArgs args = PutObjectArgs.builder().bucket(bucketName).object(fileName)
.stream(inputStream,inputStream.available(),-1).build();
client.putObject(args);
res.put("code", 1);
res.put("msg", minioProp.getEndpoint() + "/" + bucketName + "/" + fileName);
return res;
}
}
5.controller层测试
@PostMapping("/uploadimg")
@ResponseBody
public String uploadimg(@RequestParam(name = "file", required = false) MultipartFile file,
HttpServletRequest request) {
JSONObject res = null;
try {
res = minioUtils.uploadFile(file, "abc");
} catch (Exception e) {
e.printStackTrace();
res.put("code", 0);
res.put("msg", "上传失败");
}
return res.toJSONString();
}
@PostMapping("/uploadvideo")
@ResponseBody
public String uploadvideo(@RequestParam(name = "file", required = false) MultipartFile file,
HttpServletRequest request) {
JSONObject res = null;
try {
res = minioUtils.uploadFile(file, "abc");
} catch (Exception e) {
e.printStackTrace();
res.put("code", 0);
res.put("msg", "上传失败");
}
return res.toJSONString();
}
6.页面编写,上传视频和图片皆可
<!DOCTYPE html>
<html lang="en" xmlns:th="http://www.thymeleaf.org">
<head>
<meta charset="UTF-8">
<title>首页</title>
</head>
<body>
<center>
<form accept-charset="UTF-8" th:action="@{uploadimg}" method="post"
enctype="multipart/form-data" target="_blank">
文件:<input type="file" name="file"/>
<input type="submit" value="上传"/>
</form>
</center>
</body>
</html>
=================================================================
<form accept-charset="UTF-8" th:action="@{uploadvideo}" method="post"
enctype="multipart/form-data" target="_blank">
文件:<input type="file" name="file"/>
<input type="submit" value="上传"/>
</form>
注意:上传完成后,返回的url地址为:http://服务ip:端口/桶名称/文件名
7.测试,在页面其它地方放入标签显示
<img src = "http://192.168.128.128:9000/abc/abc_1652184296820.jpg" />
<br/>
<video width="1120" height="540" controls="controls" id="video" preload="auto" >
<source src="http://192.168.128.128:9000/abc/abc_1652234547469.mp4" type="video/mp4">
</video>
4.附录完整工具类
import io.minio.*;
import io.minio.errors.*;
import io.minio.http.Method;
import io.minio.messages.*;
// import net.minidev.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.multipart.MultipartFile;
import java.io.*;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.List;
import java.util.concurrent.TimeUnit;
public class MinioUtils1 {
private static final Logger log = LoggerFactory.getLogger(MinioUtils1.class);
private final String endpoint;
private final String accessKey;
private final String secretKey;
private MinioClient minioClient;
public MinioUtils1(String endpoint, String accessKey, String secretKey) {
this.endpoint = endpoint;
this.accessKey = accessKey;
this.secretKey = secretKey;
this.minioClient = MinioClient.builder().endpoint(endpoint).credentials(accessKey, secretKey).build();
}
/*
@PostConstruct
private MinioClient client() {
}
*/
public boolean doesBucketExists(String bucketName) {
BucketExistsArgs args = BucketExistsArgs.builder()
.bucket(bucketName)
.build();
try {
return minioClient.bucketExists(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
throw new RuntimeException(e);
}
}
/**
* 创建 bucket
*
* @param bucketName 桶名
*/
public void createBucket(String bucketName) {
BucketExistsArgs bucketExistsArgs = BucketExistsArgs.builder().bucket(bucketName).build();
MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket(bucketName).build();
try {
if (minioClient.bucketExists(bucketExistsArgs))
return;
minioClient.makeBucket(makeBucketArgs);
} catch (Exception e) {
log.error("创建桶失败:{}", e.getMessage());
throw new RuntimeException(e);
}
}
/**
* 判断文件是否存在
*
* @param bucketName 存储桶
* @param objectName 对象
* @return true:存在
*/
public boolean doesObjectExist(String bucketName, String objectName) {
StatObjectArgs args = StatObjectArgs.builder().bucket(bucketName).object(objectName).build();
try {
minioClient.statObject(args);
} catch (Exception e) {
return false;
}
return true;
}
/**
* 判断文件夹是否存在
*
* @param bucketName 存储桶
* @param objectName 文件夹名称(去掉/)
* @return true:存在
*/
public boolean doesFolderExist(String bucketName, String objectName) {
ListObjectsArgs args = ListObjectsArgs.builder()
.bucket(bucketName)
.prefix(objectName)
.recursive(false)
.build();
boolean exist = false;
try {
Iterable<Result<Item>> results = minioClient.listObjects(args);
for (Result<Item> result : results) {
Item item = result.get();
if (!item.isDir())
continue;
if (objectName.equals(item.objectName())) {
exist = true;
}
}
} catch (Exception e) {
exist = false;
}
return exist;
}
/**
* 通过 MultipartFile ,上传文件
*
* @param bucketName 存储桶
* @param file 文件
* @param objectName 对象名
*/
public ObjectWriteResponse putObject(String bucketName, MultipartFile file, String objectName, String contentType) {
try {
InputStream inputStream = file.getInputStream();
PutObjectArgs args = PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.contentType(contentType)
.stream(inputStream, inputStream.available(), -1)
.build();
return minioClient.putObject(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
throw new RuntimeException(e);
}
}
/**
* 上传本地文件
*
* @param bucketName 存储桶
* @param objectName 对象名称
* @param fileName 本地文件路径
*/
public ObjectWriteResponse putObject(String bucketName, String objectName, String fileName) {
try {
UploadObjectArgs args = UploadObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.filename(fileName)
.build();
return minioClient.uploadObject(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
throw new RuntimeException(e);
}
}
/**
* 通过流上传文件
*
* @param bucketName 存储桶
* @param objectName 文件对象
* @param inputStream 文件流
*/
public ObjectWriteResponse putObjectByStream(String bucketName, String objectName, InputStream inputStream) {
try {
PutObjectArgs args = PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.stream(inputStream, inputStream.available(), -1)
.build();
return minioClient.putObject(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
throw new RuntimeException(e);
}
}
/**
* 创建文件夹或目录
*
* @param bucketName 存储桶
* @param objectName 目录路径
*/
public ObjectWriteResponse putDirObject(String bucketName, String objectName) {
PutObjectArgs args = PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.stream(new ByteArrayInputStream(new byte[]{}), 0, -1)
.build();
try {
return minioClient.putObject(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
throw new RuntimeException(e);
}
}
/**
* 获取全部 bucket
*/
public List<Bucket> getAllBuckets() throws Exception {
return minioClient.listBuckets();
}
/**
* 根据 bucketName 删除信息
*
* @param bucketName 桶名
*/
public void removeBucket(String bucketName) {
try {
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
e.printStackTrace();
}
}
/**
* 获取⽂件外链
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param expires 过期时间 <=7
*/
public String getObjectUrl(String bucketName, String objectName, Integer expires) {
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder()
.method(Method.GET)
.bucket(bucketName)
.object(objectName)
.expiry(expires) // 单位:秒
.build();
try {
return minioClient.getPresignedObjectUrl(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidResponseException | InvalidKeyException | NoSuchAlgorithmException | IOException | XmlParserException | ServerException e) {
throw new RuntimeException(e);
}
}
/**
* 获取⽂件外链
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param duration 过期时间
* @param unit 过期时间的单位
*/
public String getObjectUrl(String bucketName, String objectName, int duration, TimeUnit unit) {
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder()
.method(Method.GET)
.bucket(bucketName)
.object(objectName)
.expiry(duration, unit)
.build();
try {
return minioClient.getPresignedObjectUrl(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidResponseException | InvalidKeyException | NoSuchAlgorithmException | IOException | XmlParserException | ServerException e) {
throw new RuntimeException(e);
}
}
/**
* 获取文件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @return ⼆进制流
*/
public InputStream getObject(String bucketName, String objectName) throws Exception {
GetObjectArgs args = GetObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build();
return minioClient.getObject(args);
}
/**
* 上传文件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param stream ⽂件流
* @throws Exception https://docs.minio.io/cn/java-client-api-reference.html#putObject
*/
public void putObject(String bucketName, String objectName, InputStream stream) {
putObjectByStream(bucketName, objectName, stream);
}
/**
* 文件流上传文件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param stream ⽂件流
* @param size ⼤⼩
* @param contextType 类型
*/
public void putObject(String bucketName, String objectName, InputStream stream, long size, String contextType) {
putObjectByStream(bucketName, objectName, stream);
}
/**
* 获取文件信息
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @throws Exception https://docs.minio.io/cn/java-client-api-reference.html#statObject
*/
public StatObjectResponse getObjectInfo(String bucketName, String objectName) {
StatObjectArgs args = StatObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build();
try {
return minioClient.statObject(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
throw new RuntimeException(e);
}
}
/**
* 删除文件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @throws Exception https://docs.minio.io/cn/java-client-apireference.html#removeObject
*/
public void removeObject(String bucketName, String objectName) {
RemoveObjectArgs args = RemoveObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build();
try {
minioClient.removeObject(args);
} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {
e.printStackTrace();
}
}
}
更多推荐
所有评论(0)