SpringCloud微服務實戰——搭建企業級開發框架（二十九）：集成對象存儲服務MinIO+七牛雲+阿里雲+騰訊雲详情 - 對象存儲,oss,七牛雲存儲,騰訊雲,minio 全棧程序猿日志

微服務應用中圖片、文件等存儲區別於單體應用，單體應用可以放到本地讀寫磁盤文件，微服務應用必需用到分佈式存儲，將圖片、文件等存儲到服務穩定的分佈式存儲服務器。目前，很多雲服務商提供了存儲的雲服務，比如阿里雲OSS、騰訊雲COS、七牛雲對象存儲Kodo、百度雲對象存儲BOS等等、還有開源對象存儲服務器，比如FastDFS、MinIO等。
如果我們的框架只支持一種存儲服務，那麼在後期擴展或者修改時會有侷限性，所以，這裏希望能夠定義一個抽象接口，想使用哪種服務就實現哪種服務，在配置多個服務時，調用的存儲時可以進行選擇。在這裏雲服務選擇七牛雲，開源服務選擇MinIO進行集成，如果需要其他服務可以自行擴展。
首先，在框架搭建前，我們先準備環境，這裏以MinIO和七牛云為例，MinIO的安裝十分簡單，我們這裏選擇Linux安裝包的方式來安裝，具體方式參考：http://docs.minio.org.cn/docs/，七牛雲只需要到官網註冊並實名認證即可獲得10G免費存儲容量https://www.qiniu.com/。

一、基礎底層庫實現

1、在GitEgg-Platform中新建gitegg-platform-dfs (dfs: Distributed File System分佈式文件系統)子工程用於定義對象存儲服務的抽象接口，新建IDfsBaseService用於定義文件上傳下載常用接口

/**
 * 分佈式文件存儲操作接口定義
 * 為了保留系統操作記錄，原則上不允許上傳文件物理刪除，修改等操作。
 * 業務操作的修改刪除文件，只是關聯關係的修改，重新上傳文件後並與業務關聯即可。
 */
public interface IDfsBaseService {

    /**
     * 獲取簡單上傳憑證
     * @param bucket
     * @return
     */
    String uploadToken(String bucket);


    /**
     * 獲取覆蓋上傳憑證
     * @param bucket
     * @return
     */
    String uploadToken(String bucket, String key);

    /**
     * 創建 bucket
     * @param bucket
     */
    void createBucket(String bucket);

    /**
     * 通過流上傳文件，指定文件名
     * @param inputStream
     * @param fileName
     * @return
     */
    GitEggDfsFile uploadFile(InputStream inputStream, String fileName);

    /**
     * 通過流上傳文件，指定文件名和bucket
     * @param inputStream
     * @param bucket
     * @param fileName
     * @return
     */
    GitEggDfsFile uploadFile(InputStream inputStream, String bucket, String fileName);


    /**
     * 通過文件名獲取文件訪問鏈接
     * @param fileName
     * @return
     */
    String getFileUrl(String fileName);

    /**
     * 通過文件名和bucket獲取文件訪問鏈接
     * @param fileName
     * @param bucket
     * @return
     */
    String getFileUrl(String bucket, String fileName);

    /**
     * 通過文件名和bucket獲取文件訪問鏈接,設置有效期
     * @param bucket
     * @param fileName
     * @param duration
     * @param unit
     * @return
     */
    String getFileUrl(String bucket, String fileName, int duration, TimeUnit unit);

    /**
     * 通過文件名以流的形式下載一個對象
     * @param fileName
     * @return
     */
    OutputStream getFileObject(String fileName, OutputStream outputStream);

    /**
     * 通過文件名和bucket以流的形式下載一個對象
     * @param fileName
     * @param bucket
     * @return
     */
    OutputStream getFileObject(String bucket, String fileName, OutputStream outputStream);

    /**
     * 根據文件名刪除文件
     * @param fileName
     */
    String removeFile(String fileName);

    /**
     * 根據文件名刪除指定bucket下的文件
     * @param bucket
     * @param fileName
     */
    String removeFile(String bucket, String fileName);


    /**
     * 根據文件名列表批量刪除文件
     * @param fileNames
     */
    String removeFiles(List<String> fileNames);

    /**
     * 根據文件名列表批量刪除bucket下的文件
     * @param bucket
     * @param fileNames
     */
    String removeFiles(String bucket, List<String> fileNames);
}

2、在GitEgg-Platform中新建gitegg-platform-dfs-minio子工程，新建MinioDfsServiceImpl和MinioDfsProperties用於實現IDfsBaseService文件上傳下載接口

@Data
@Component
@ConfigurationProperties(prefix = "dfs.minio")
public class MinioDfsProperties {

    /**
     * AccessKey
     */
    private String accessKey;

    /**
     * SecretKey
     */
    private String secretKey;

    /**
     * 區域，需要在MinIO配置服務器的物理位置，默認是us-east-1（美國東區1）,這也是亞馬遜S3的默認區域。
     */
    private String region;

    /**
     * Bucket
     */
    private String bucket;

    /**
     * 公開還是私有
     */
    private Integer accessControl;

    /**
     * 上傳服務器域名地址
     */
    private String uploadUrl;

    /**
     * 文件請求地址前綴
     */
    private String accessUrlPrefix;

    /**
     *  上傳文件夾前綴
     */
    private String uploadDirPrefix;
}

@Slf4j
@AllArgsConstructor
public class MinioDfsServiceImpl implements IDfsBaseService {

    private final MinioClient minioClient;

    private final MinioDfsProperties minioDfsProperties;

    @Override
    public String uploadToken(String bucket) {
        return null;
    }

    @Override
    public String uploadToken(String bucket, String key) {
        return null;
    }

    @Override
    public void createBucket(String bucket) {
        BucketExistsArgs bea = BucketExistsArgs.builder().bucket(bucket).build();
        try {
            if (!minioClient.bucketExists(bea)) {
                MakeBucketArgs mba = MakeBucketArgs.builder().bucket(bucket).build();
                minioClient.makeBucket(mba);
            }
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        }
    }

    @Override
    public GitEggDfsFile uploadFile(InputStream inputStream, String fileName) {
        return this.uploadFile(inputStream, minioDfsProperties.getBucket(), fileName);
    }

    @Override
    public GitEggDfsFile uploadFile(InputStream inputStream, String bucket, String fileName) {
        GitEggDfsFile dfsFile = new GitEggDfsFile();
        try {

            dfsFile.setBucket(bucket);
            dfsFile.setBucketDomain(minioDfsProperties.getUploadUrl());
            dfsFile.setFileUrl(minioDfsProperties.getAccessUrlPrefix());
            dfsFile.setEncodedFileName(fileName);

            minioClient.putObject(PutObjectArgs.builder()
                    .bucket(bucket)
                    .stream(inputStream, -1, 5*1024*1024)
                    .object(fileName)
                    .build());

        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        }
        return dfsFile;
    }

    @Override
    public String getFileUrl(String fileName) {
        return this.getFileUrl(minioDfsProperties.getBucket(), fileName);
    }

    @Override
    public String getFileUrl(String bucket, String fileName) {
        return this.getFileUrl(bucket, fileName, DfsConstants.DFS_FILE_DURATION, DfsConstants.DFS_FILE_DURATION_UNIT);
    }

    @Override
    public String getFileUrl(String bucket, String fileName, int duration, TimeUnit unit) {
        String url = null;
        try {
            url = minioClient.getPresignedObjectUrl(
                    GetPresignedObjectUrlArgs.builder()
                            .method(Method.GET)
                            .bucket(bucket)
                            .object(fileName)
                            .expiry(duration, unit)
                            .build());
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        }
        return url;
    }

    @Override
    public OutputStream getFileObject(String fileName, OutputStream outputStream) {
        return this.getFileObject(minioDfsProperties.getBucket(), fileName, outputStream);
    }

    @Override
    public OutputStream getFileObject(String bucket, String fileName, OutputStream outputStream) {
        BufferedInputStream bis = null;
        InputStream stream = null;
        try {
            stream = minioClient.getObject(
                    GetObjectArgs.builder()
                            .bucket(bucket)
                            .object(fileName)
                            .build());
            bis = new BufferedInputStream(stream);
            IOUtils.copy(bis, outputStream);
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        } finally {
            if (stream != null) {
                try {
                    stream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (bis != null) {
                try {
                    bis.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        return outputStream;
    }

    @Override
    public String removeFile(String fileName) {
        return this.removeFile(minioDfsProperties.getBucket(), fileName);
    }

    @Override
    public String removeFile(String bucket, String fileName) {
        return this.removeFiles(bucket, Collections.singletonList(fileName));
    }

    @Override
    public String removeFiles(List<String> fileNames) {
        return this.removeFiles(minioDfsProperties.getBucket(), fileNames);
    }

    @Override
    public String removeFiles(String bucket, List<String> fileNames) {
        List<DeleteObject> deleteObject = new ArrayList<>();
        if (!CollectionUtils.isEmpty(fileNames))
        {
            fileNames.stream().forEach(item -> {
                deleteObject.add(new DeleteObject(item));
            });
        }
        Iterable<Result<DeleteError>> result = minioClient.removeObjects(RemoveObjectsArgs.builder()
                .bucket(bucket)
                .objects(deleteObject)
                .build());
        try {
            return JsonUtils.objToJsonIgnoreNull(result);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

3、在GitEgg-Platform中新建gitegg-platform-dfs-qiniu子工程，新建QiNiuDfsServiceImpl和QiNiuDfsProperties用於實現IDfsBaseService文件上傳下載接口

@Data
@Component
@ConfigurationProperties(prefix = "dfs.qiniu")
public class QiNiuDfsProperties {


    /**
     * AccessKey
     */
    private String accessKey;

    /**
     * SecretKey
     */
    private String secretKey;

    /**
     * 七牛雲機房
     */
    private String region;

    /**
     * Bucket 存儲塊
     */
    private String bucket;

    /**
     * 公開還是私有
     */
    private Integer accessControl;

    /**
     * 上傳服務器域名地址
     */
    private String uploadUrl;

    /**
     * 文件請求地址前綴
     */
    private String accessUrlPrefix;

    /**
     * 上傳文件夾前綴
     */
    private String uploadDirPrefix;
}

@Slf4j
@AllArgsConstructor
public class QiNiuDfsServiceImpl implements IDfsBaseService {

    private final Auth auth;

    private final UploadManager uploadManager;

    private final BucketManager bucketManager;

    private final QiNiuDfsProperties qiNiuDfsProperties;

    /**
     *
     * @param bucket
     * @return
     */
    @Override
    public String uploadToken(String bucket) {
        Auth auth = Auth.create(qiNiuDfsProperties.getAccessKey(), qiNiuDfsProperties.getSecretKey());
        String upToken = auth.uploadToken(bucket);
        return upToken;
    }

    /**
     *
     * @param bucket
     * @param key
     * @return
     */
    @Override
    public String uploadToken(String bucket, String key) {
        Auth auth = Auth.create(qiNiuDfsProperties.getAccessKey(), qiNiuDfsProperties.getSecretKey());
        String upToken = auth.uploadToken(bucket, key);
        return upToken;
    }

    @Override
    public void createBucket(String bucket) {
        try {
            String[] buckets = bucketManager.buckets();
            if (!ArrayUtil.contains(buckets, bucket)) {
                bucketManager.createBucket(bucket, qiNiuDfsProperties.getRegion());
            }
        } catch (QiniuException e) {
            e.printStackTrace();
        }

    }

    /**
     *
     * @param inputStream
     * @param fileName
     * @return
     */
    @Override
    public GitEggDfsFile uploadFile(InputStream inputStream, String fileName) {
        return this.uploadFile(inputStream, qiNiuDfsProperties.getBucket(), fileName);
    }

    /**
     *
     * @param inputStream
     * @param bucket
     * @param fileName
     * @return
     */
    @Override
    public GitEggDfsFile uploadFile(InputStream inputStream, String bucket, String fileName) {
        GitEggDfsFile dfsFile = null;
        //默認不指定key的情況下，以文件內容的hash值作為文件名
        String key = null;
        if (!StringUtils.isEmpty(fileName))
        {
            key = fileName;
        }
        try {
            String upToken = auth.uploadToken(bucket);
            Response response = uploadManager.put(inputStream, key, upToken,null, null);
            //解析上傳成功的結果
            dfsFile = JsonUtils.jsonToPojo(response.bodyString(), GitEggDfsFile.class);
            if (dfsFile != null) {
                dfsFile.setBucket(bucket);
                dfsFile.setBucketDomain(qiNiuDfsProperties.getUploadUrl());
                dfsFile.setFileUrl(qiNiuDfsProperties.getAccessUrlPrefix());
                dfsFile.setEncodedFileName(fileName);
            }
        } catch (QiniuException ex) {
            Response r = ex.response;
            log.error(r.toString());
            try {
                log.error(r.bodyString());
            } catch (QiniuException ex2) {
                log.error(ex2.toString());
            }
        } catch (Exception e) {
            log.error(e.toString());
        }
        return dfsFile;
    }

    @Override
    public String getFileUrl(String fileName) {
        return this.getFileUrl(qiNiuDfsProperties.getBucket(), fileName);
    }

    @Override
    public String getFileUrl(String bucket, String fileName) {
        return this.getFileUrl(bucket, fileName, DfsConstants.DFS_FILE_DURATION, DfsConstants.DFS_FILE_DURATION_UNIT);
    }

    @Override
    public String getFileUrl(String bucket, String fileName, int duration, TimeUnit unit) {
        String finalUrl = null;
        try {
            Integer accessControl = qiNiuDfsProperties.getAccessControl();
            if (accessControl != null && DfsConstants.DFS_FILE_PRIVATE == accessControl.intValue()) {
                String encodedFileName = URLEncoder.encode(fileName, "utf-8").replace("+", "%20");
                String publicUrl = String.format("%s/%s", qiNiuDfsProperties.getAccessUrlPrefix(), encodedFileName);
                String accessKey = qiNiuDfsProperties.getAccessKey();
                String secretKey = qiNiuDfsProperties.getSecretKey();
                Auth auth = Auth.create(accessKey, secretKey);
                long expireInSeconds = unit.toSeconds(duration);
                finalUrl = auth.privateDownloadUrl(publicUrl, expireInSeconds);
            }
            else {
                finalUrl = String.format("%s/%s", qiNiuDfsProperties.getAccessUrlPrefix(), fileName);
            }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return finalUrl;
    }

    @Override
    public OutputStream getFileObject(String fileName, OutputStream outputStream) {
        return this.getFileObject(qiNiuDfsProperties.getBucket(), fileName, outputStream);
    }

    @Override
    public OutputStream getFileObject(String bucket, String fileName, OutputStream outputStream) {
        URL url = null;
        HttpURLConnection conn = null;
        BufferedInputStream bis = null;
        try {
            String path =  this.getFileUrl(bucket, fileName, DfsConstants.DFS_FILE_DURATION, DfsConstants.DFS_FILE_DURATION_UNIT);
            url = new URL(path);
            conn = (HttpURLConnection)url.openConnection();
            //設置超時間
            conn.setConnectTimeout(DfsConstants.DOWNLOAD_TIMEOUT);
            //防止屏蔽程序抓取而返回403錯誤
            conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");
            conn.connect();
            //得到輸入流
            bis = new BufferedInputStream(conn.getInputStream());
            IOUtils.copy(bis, outputStream);
        } catch (Exception e) {
            log.error("讀取網絡文件異常:" + fileName);
        }
        finally {
            conn.disconnect();
            if (bis != null) {
                try {
                    bis.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        return outputStream;
    }

    /**
     *
     * @param fileName
     * @return
     */
    @Override
    public String removeFile(String fileName) {
        return this.removeFile( qiNiuDfsProperties.getBucket(), fileName);
    }

    /**
     *
     * @param bucket
     * @param fileName
     * @return
     */
    @Override
    public String removeFile(String bucket, String fileName) {
        String resultStr = null;
        try {
            Response response = bucketManager.delete(bucket, fileName);
            resultStr = JsonUtils.objToJson(response);
        } catch (QiniuException e) {
            Response r = e.response;
            log.error(r.toString());
            try {
                log.error(r.bodyString());
            } catch (QiniuException ex2) {
                log.error(ex2.toString());
            }
        } catch (Exception e) {
            log.error(e.toString());
        }
        return resultStr;
    }

    /**
     *
     * @param fileNames
     * @return
     */
    @Override
    public String removeFiles(List<String> fileNames) {
        return this.removeFiles(qiNiuDfsProperties.getBucket(), fileNames);
    }

    /**
     *
     * @param bucket
     * @param fileNames
     * @return
     */
    @Override
    public String removeFiles(String bucket, List<String> fileNames) {
        String resultStr = null;
        try {
            if (!CollectionUtils.isEmpty(fileNames) && fileNames.size() > GitEggConstant.Number.THOUSAND)
            {
                throw new BusinessException("單次批量請求的文件數量不得超過1000");
            }
            BucketManager.BatchOperations batchOperations = new BucketManager.BatchOperations();
            batchOperations.addDeleteOp(bucket, (String [])fileNames.toArray());
            Response response = bucketManager.batch(batchOperations);
            BatchStatus[] batchStatusList = response.jsonToObject(BatchStatus[].class);
            resultStr = JsonUtils.objToJson(batchStatusList);

        } catch (QiniuException ex) {
            log.error(ex.response.toString());
        } catch (Exception e) {
            log.error(e.toString());
        }
        return resultStr;
    }
}

4、在GitEgg-Platform中新建gitegg-platform-dfs-starter子工程，用於集成所有文件上傳下載子工程，方便業務統一引入所有實現

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>GitEgg-Platform</artifactId>
        <groupId>com.gitegg.platform</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>gitegg-platform-dfs-starter</artifactId>
    <name>${project.artifactId}</name>
    <packaging>jar</packaging>

    <dependencies>
        <!-- gitegg 分佈式文件自定義擴展-minio -->
        <dependency>
            <groupId>com.gitegg.platform</groupId>
            <artifactId>gitegg-platform-dfs-minio</artifactId>
        </dependency>
        <!-- gitegg 分佈式文件自定義擴展-七牛雲 -->
        <dependency>
            <groupId>com.gitegg.platform</groupId>
            <artifactId>gitegg-platform-dfs-qiniu</artifactId>
        </dependency>
    </dependencies>
</project>

5、gitegg-platform-bom中添加文件存儲相關依賴

            <!-- gitegg 分佈式文件自定義擴展 -->
            <dependency>
                <groupId>com.gitegg.platform</groupId>
                <artifactId>gitegg-platform-dfs</artifactId>
                <version>${gitegg.project.version}</version>
            </dependency>
            <!-- gitegg 分佈式文件自定義擴展-minio -->
            <dependency>
                <groupId>com.gitegg.platform</groupId>
                <artifactId>gitegg-platform-dfs-minio</artifactId>
                <version>${gitegg.project.version}</version>
            </dependency>
            <!-- gitegg 分佈式文件自定義擴展-七牛雲 -->
            <dependency>
                <groupId>com.gitegg.platform</groupId>
                <artifactId>gitegg-platform-dfs-qiniu</artifactId>
                <version>${gitegg.project.version}</version>
            </dependency>
            <!-- gitegg 分佈式文件自定義擴展-starter -->
            <dependency>
                <groupId>com.gitegg.platform</groupId>
                <artifactId>gitegg-platform-dfs-starter</artifactId>
                <version>${gitegg.project.version}</version>
            </dependency>

           <!-- minio文件存儲服務 https://mvnrepository.com/artifact/io.minio/minio -->
            <dependency>
                <groupId>io.minio</groupId>
                <artifactId>minio</artifactId>
                <version>${dfs.minio.version}</version>
            </dependency>
            <!--七牛雲文件存儲服務-->
            <dependency>
                <groupId>com.qiniu</groupId>
                <artifactId>qiniu-java-sdk</artifactId>
                <version>${dfs.qiniu.version}</version>
            </dependency>

二、業務功能實現

分佈式文件存儲功能作為系統擴展功能放在gitegg-service-extension工程中，首先需要分為幾個模塊：

文件服務器的基本配置模塊
文件的上傳、下載記錄模塊(下載只記錄私有文件，對於公共可訪問的文件不需要記錄)

前端訪問下載實現
1、新建文件服務器配置表，用於存放文件服務器相關配置，定義好表結構，使用代碼生成工具生成增刪改查代碼。

CREATE TABLE `t_sys_dfs`  (
`id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '主鍵',
`tenant_id` bigint(20) NOT NULL DEFAULT 0 COMMENT '租户id',
`dfs_type` bigint(20) NULL DEFAULT NULL COMMENT '分佈式存儲分類',
`dfs_code` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '分佈式存儲編號',
`access_url_prefix` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '文件訪問地址前綴',
`upload_url` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '分佈式存儲上傳地址',
`bucket` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '空間名稱',
`app_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '應用ID',
`region` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '區域',
`access_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT 'accessKey',
`secret_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT 'secretKey',
`dfs_default` tinyint(2) NOT NULL DEFAULT 0 COMMENT '是否默認存儲 0否，1是',
`dfs_status` tinyint(2) NOT NULL DEFAULT 1 COMMENT '狀態 0禁用，1 啓用',
`access_control` tinyint(2) NOT NULL DEFAULT 0 COMMENT '訪問控制 0私有，1公開',
`comments` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '備註',
`create_time` datetime(0) NULL DEFAULT NULL COMMENT '創建時間',
`creator` bigint(20) NULL DEFAULT NULL COMMENT '創建者',
`update_time` datetime(0) NULL DEFAULT NULL COMMENT '更新時間',
`operator` bigint(20) NULL DEFAULT NULL COMMENT '更新者',
`del_flag` tinyint(2) NULL DEFAULT 0 COMMENT '1:刪除 0:不刪除',
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = '分佈式存儲配置表' ROW_FORMAT = DYNAMIC;

2、新建DfsQiniuFactory和DfsMinioFactory接口實現工廠類，用於根據當前用户的選擇，實例化需要的接口實現類

/**
 * 七牛雲上傳服務接口工廠類
 */
public class DfsQiniuFactory {

  public static IDfsBaseService getDfsBaseService(DfsDTO dfsDTO) {
      Auth auth = Auth.create(dfsDTO.getAccessKey(), dfsDTO.getSecretKey());
      Configuration cfg = new Configuration(Region.autoRegion());
      UploadManager uploadManager = new UploadManager(cfg);
      BucketManager bucketManager = new BucketManager(auth, cfg);
      QiNiuDfsProperties qiNiuDfsProperties = new QiNiuDfsProperties();
      qiNiuDfsProperties.setAccessKey(dfsDTO.getAccessKey());
      qiNiuDfsProperties.setSecretKey(dfsDTO.getSecretKey());
      qiNiuDfsProperties.setRegion(dfsDTO.getRegion());
      qiNiuDfsProperties.setBucket(dfsDTO.getBucket());
      qiNiuDfsProperties.setUploadUrl(dfsDTO.getUploadUrl());
      qiNiuDfsProperties.setAccessUrlPrefix(dfsDTO.getAccessUrlPrefix());
      qiNiuDfsProperties.setAccessControl(dfsDTO.getAccessControl());
      return new QiNiuDfsServiceImpl(auth, uploadManager, bucketManager, qiNiuDfsProperties);
  }
}

/**
 * MINIO上傳服務接口工廠類
 */
public class DfsMinioFactory {

  public static IDfsBaseService getDfsBaseService(DfsDTO dfsDTO) {
      MinioClient minioClient =
              MinioClient.builder()
                      .endpoint(dfsDTO.getUploadUrl())
                      .credentials(dfsDTO.getAccessKey(), dfsDTO.getSecretKey()).build();;
      MinioDfsProperties minioDfsProperties = new MinioDfsProperties();
      minioDfsProperties.setAccessKey(dfsDTO.getAccessKey());
      minioDfsProperties.setSecretKey(dfsDTO.getSecretKey());
      minioDfsProperties.setRegion(dfsDTO.getRegion());
      minioDfsProperties.setBucket(dfsDTO.getBucket());
      minioDfsProperties.setUploadUrl(dfsDTO.getUploadUrl());
      minioDfsProperties.setAccessUrlPrefix(dfsDTO.getAccessUrlPrefix());
      minioDfsProperties.setAccessControl(dfsDTO.getAccessControl());

      return new MinioDfsServiceImpl(minioClient, minioDfsProperties);
  }
}

3、新建DfsFactory工廠類，添加@Component使用容器管理該類（默認單例），用於根據系統用户配置，生成及緩存對應的上傳下載接口實現

/**
 * DfsFactory工廠類,根據系統用户配置，生成及緩存對應的上傳下載接口實現
 */
@Component
public class DfsFactory {

  /**
   * DfsService 緩存
   */
  private final static Map<Long, IDfsBaseService> dfsBaseServiceMap = new ConcurrentHashMap<>();

  /**
   * 獲取 DfsService
   *
   * @param dfsDTO 分佈式存儲配置
   * @return dfsService
   */
  public IDfsBaseService getDfsBaseService(DfsDTO dfsDTO) {
      //根據dfsId獲取對應的分佈式存儲服務接口，dfsId是唯一的，每個租户有其自有的dfsId
      Long dfsId = dfsDTO.getId();
      IDfsBaseService dfsBaseService = dfsBaseServiceMap.get(dfsId);
      if (null == dfsBaseService) {
          Class cls = null;
          try {
              cls = Class.forName(DfsFactoryClassEnum.getValue(String.valueOf(dfsDTO.getDfsType())));
              Method staticMethod = cls.getDeclaredMethod(DfsConstants.DFS_SERVICE_FUNCTION, DfsDTO.class);
              dfsBaseService = (IDfsBaseService) staticMethod.invoke(cls, dfsDTO);
              dfsBaseServiceMap.put(dfsId, dfsBaseService);
          } catch (ClassNotFoundException | NoSuchMethodException e) {
              e.printStackTrace();
          } catch (IllegalAccessException e) {
              e.printStackTrace();
          } catch (InvocationTargetException e) {
              e.printStackTrace();
          }
      }
      return dfsBaseService;
  }
}

4、新建枚舉類DfsFactoryClassEnum，用於DfsFactory 工廠類通過反射實例化對應文件服務器的接口實現類

/**
 * @ClassName: DfsFactoryClassEnum
 * @Description: 分佈式存儲工廠類枚舉 ，因dfs表存的是數據字典表的id，這裏省一次數據庫查詢，所以就用數據字典的id
 * @author GitEgg
 * @date 2020年09月19日 下午11:49:45
 */
public enum DfsFactoryClassEnum {

  /**
   * MINIO MINIO
   */
  MINIO("2", "com.gitegg.service.extension.dfs.factory.DfsMinioFactory"),

  /**
   * 七牛雲Kodo QINIUYUN_KODO
   */
  QI_NIU("3", "com.gitegg.service.extension.dfs.factory.DfsQiniuFactory"),

  /**
   * 阿里雲OSS ALIYUN_OSS
   */
  ALI_YUN("4", "com.gitegg.service.extension.dfs.factory.DfsAliyunFactory"),

  /**
   * 騰訊雲COS TENCENT_COS
   */
  TENCENT("5", "com.gitegg.service.extension.dfs.factory.DfsTencentFactory");

  public String code;

  public String value;

  DfsFactoryClassEnum(String code, String value) {
      this.code = code;
      this.value = value;
  }

  public static String getValue(String code) {
      DfsFactoryClassEnum[] smsFactoryClassEnums = values();
      for (DfsFactoryClassEnum smsFactoryClassEnum : smsFactoryClassEnums) {
          if (smsFactoryClassEnum.getCode().equals(code)) {
              return smsFactoryClassEnum.getValue();
          }
      }
      return null;
  }

  public String getCode() {
      return code;
  }

  public void setCode(String code) {
      this.code = code;
  }

  public String getValue() {
      return value;
  }

  public void setValue(String value) {
      this.value = value;
  }
}

5、新建IGitEggDfsService接口，用於定義業務需要的文件上傳下載接口

/**
 * 業務文件上傳下載接口實現
 *
 */
public interface IGitEggDfsService {


  /**
   * 獲取文件上傳的 token
   * @param dfsCode
   * @return
   */
  String uploadToken(String dfsCode);


  /**
   * 上傳文件
   *
   * @param dfsCode
   * @param file
   * @return
   */
  GitEggDfsFile uploadFile(String dfsCode, MultipartFile file);

  /**
   * 獲取文件訪問鏈接
   * @param dfsCode
   * @param fileName
   * @return
   */
  String getFileUrl(String dfsCode, String fileName);


  /**
   * 下載文件
   * @param dfsCode
   * @param fileName
   * @return
   */
  OutputStream downloadFile(String dfsCode, String fileName, OutputStream outputStream);
}

6、新建IGitEggDfsService接口實現類GitEggDfsServiceImpl，用於實現業務需要的文件上傳下載接口

@Slf4j
@Service
@RequiredArgsConstructor(onConstructor_ = @Autowired)
public class GitEggDfsServiceImpl implements IGitEggDfsService {

  private final DfsFactory dfsFactory;

  private final IDfsService dfsService;

  private final IDfsFileService dfsFileService;

  @Override
  public String uploadToken(String dfsCode) {
      QueryDfsDTO queryDfsDTO = new QueryDfsDTO();
      queryDfsDTO.setDfsCode(dfsCode);
      DfsDTO dfsDTO = dfsService.queryDfs(queryDfsDTO);
      IDfsBaseService dfsBaseService = dfsFactory.getDfsBaseService(dfsDTO);
      String token = dfsBaseService.uploadToken(dfsDTO.getBucket());
      return token;
  }

  @Override
  public GitEggDfsFile uploadFile(String dfsCode, MultipartFile file) {
      QueryDfsDTO queryDfsDTO = new QueryDfsDTO();
      DfsDTO dfsDTO = null;

      // 如果上傳時沒有選擇存儲方式，那麼取默認存儲方式
      if(StringUtils.isEmpty(dfsCode)) {
          queryDfsDTO.setDfsDefault(GitEggConstant.ENABLE);
      }
      else {
          queryDfsDTO.setDfsCode(dfsCode);
      }

      GitEggDfsFile gitEggDfsFile = null;
      DfsFile dfsFile = new DfsFile();
      try {

          dfsDTO = dfsService.queryDfs(queryDfsDTO);

          IDfsBaseService dfsFileService = dfsFactory.getDfsBaseService(dfsDTO);

          //獲取文件名
          String originalName = file.getOriginalFilename();
          //獲取文件後綴
          String extension = FilenameUtils.getExtension(originalName);
          String hash = Etag.stream(file.getInputStream(), file.getSize());
          String fileName = hash + "." + extension;

          // 保存文件上傳記錄
          dfsFile.setDfsId(dfsDTO.getId());
          dfsFile.setOriginalName(originalName);
          dfsFile.setFileName(fileName);
          dfsFile.setFileExtension(extension);
          dfsFile.setFileSize(file.getSize());
          dfsFile.setFileStatus(GitEggConstant.ENABLE);

          //執行文件上傳操作
          gitEggDfsFile = dfsFileService.uploadFile(file.getInputStream(), fileName);

          if (gitEggDfsFile != null)
          {
              gitEggDfsFile.setFileName(originalName);
              gitEggDfsFile.setKey(hash);
              gitEggDfsFile.setHash(hash);
              gitEggDfsFile.setFileSize(file.getSize());
          }

          dfsFile.setAccessUrl(gitEggDfsFile.getFileUrl());
      } catch (IOException e) {
          log.error("文件上傳失敗：{}", e);
          dfsFile.setFileStatus(GitEggConstant.DISABLE);
          dfsFile.setComments(String.valueOf(e));
      } finally {
          dfsFileService.save(dfsFile);
      }

      return gitEggDfsFile;
  }

  @Override
  public String getFileUrl(String dfsCode, String fileName) {
      String fileUrl = null;

      QueryDfsDTO queryDfsDTO = new QueryDfsDTO();
      DfsDTO dfsDTO = null;
      // 如果上傳時沒有選擇存儲方式，那麼取默認存儲方式
      if(StringUtils.isEmpty(dfsCode)) {
          queryDfsDTO.setDfsDefault(GitEggConstant.ENABLE);
      }
      else {
          queryDfsDTO.setDfsCode(dfsCode);
      }

      try {
          dfsDTO = dfsService.queryDfs(queryDfsDTO);
          IDfsBaseService dfsFileService = dfsFactory.getDfsBaseService(dfsDTO);
          fileUrl = dfsFileService.getFileUrl(fileName);
      }
      catch (Exception e)
      {
          log.error("文件上傳失敗：{}", e);
      }
      return fileUrl;
  }

  @Override
  public OutputStream downloadFile(String dfsCode, String fileName, OutputStream outputStream) {
      QueryDfsDTO queryDfsDTO = new QueryDfsDTO();
      DfsDTO dfsDTO = null;
      // 如果上傳時沒有選擇存儲方式，那麼取默認存儲方式
      if(StringUtils.isEmpty(dfsCode)) {
          queryDfsDTO.setDfsDefault(GitEggConstant.ENABLE);
      }
      else {
          queryDfsDTO.setDfsCode(dfsCode);
      }

      try {
          dfsDTO = dfsService.queryDfs(queryDfsDTO);
          IDfsBaseService dfsFileService = dfsFactory.getDfsBaseService(dfsDTO);
          outputStream = dfsFileService.getFileObject(fileName, outputStream);
      }
      catch (Exception e)
      {
          log.error("文件上傳失敗：{}", e);
      }
      return outputStream;
  }
}

7、新建GitEggDfsController用於文件上傳下載通用訪問控制器

@RestController
@RequestMapping("/extension")
@RequiredArgsConstructor(onConstructor_ = @Autowired)
@Api(value = "GitEggDfsController|文件上傳前端控制器")
@RefreshScope
public class GitEggDfsController {

  private final IGitEggDfsService gitEggDfsService;

  /**
   * 上傳文件
   * @param uploadFile
   * @param dfsCode
   * @return
   */
  @PostMapping("/upload/file")
  public Result<?> uploadFile(@RequestParam("uploadFile") MultipartFile[] uploadFile, String dfsCode) {
      GitEggDfsFile gitEggDfsFile = null;
      if (ArrayUtils.isNotEmpty(uploadFile))
      {
          for (MultipartFile file : uploadFile) {
              gitEggDfsFile = gitEggDfsService.uploadFile(dfsCode, file);
          }
      }
      return Result.data(gitEggDfsFile);
  }


  /**
   * 通過文件名獲取文件訪問鏈接
   */
  @GetMapping("/get/file/url")
  @ApiOperation(value = "查詢分佈式存儲配置表詳情")
  public Result<?> query(String dfsCode, String fileName) {
      String fileUrl = gitEggDfsService.getFileUrl(dfsCode, fileName);
      return Result.data(fileUrl);
  }

  /**
   * 通過文件名以文件流的方式下載文件
   */
  @GetMapping("/get/file/download")
  public void downloadFile(HttpServletResponse response,HttpServletRequest request,String dfsCode, String fileName) {
      if (fileName != null) {
          response.setCharacterEncoding(request.getCharacterEncoding());
          response.setContentType("application/octet-stream");
          response.addHeader("Content-Disposition", "attachment;fileName=" + fileName);
          OutputStream os = null;
          try {
              os = response.getOutputStream();
              os = gitEggDfsService.downloadFile(dfsCode, fileName, os);
              os.flush();
              os.close();
          } catch (Exception e) {
              e.printStackTrace();
          } finally {
              if (os != null) {
                  try {
                      os.close();
                  } catch (IOException e) {
                      e.printStackTrace();
                  }
              }
          }
      }
  }
}

8、前端上傳下載實現，注意的是：axios請求下載文件流時，需要設置 responseType: 'blob'

上傳

          handleUploadTest (row) {
              this.fileList = []
              this.uploading = false
              this.uploadForm.dfsType = row.dfsType
              this.uploadForm.dfsCode = row.dfsCode
              this.uploadForm.uploadFile = null
              this.dialogTestUploadVisible = true
          },
          handleRemove (file) {
              const index = this.fileList.indexOf(file)
              const newFileList = this.fileList.slice()
              newFileList.splice(index, 1)
              this.fileList = newFileList
          },
          beforeUpload (file) {
              this.fileList = [...this.fileList, file]
              return false
          },
          handleUpload () {
              this.uploadedFileName = ''
              const { fileList } = this
              const formData = new FormData()
              formData.append('dfsCode', this.uploadForm.dfsCode)
              fileList.forEach(file => {
                formData.append('uploadFile', file)
              })
              this.uploading = true
              dfsUpload(formData).then(() => {
                  this.fileList = []
                  this.uploading = false
                  this.$message.success('上傳成功')
              }).catch(err => {
                console.log('uploading', err)
                this.$message.error('上傳失敗')
              })
          }

下載

          getFileUrl (row) {
              this.listLoading = true
              this.fileDownload.dfsCode = row.dfsCode
              this.fileDownload.fileName = row.fileName
              dfsGetFileUrl(this.fileDownload).then(response => {
                  window.open(response.data)
                  this.listLoading = false
              })
          },
          downLoadFile (row) {
              this.listLoading = true
              this.fileDownload.dfsCode = row.dfsCode
              this.fileDownload.fileName = row.fileName
              this.fileDownload.responseType = 'blob'
              dfsDownloadFileUrl(this.fileDownload).then(response => {
                  const blob = new Blob([response.data])
                  var fileName = row.originalName
                  const elink = document.createElement('a')
                  elink.download = fileName
                  elink.style.display = 'none'
                  elink.href = URL.createObjectURL(blob)
                  document.body.appendChild(elink)
                  elink.click()
                  URL.revokeObjectURL(elink.href)
                  document.body.removeChild(elink)
                  this.listLoading = false
              })
          }

前端接口

import request from '@/utils/request'

export function dfsUpload (formData) {
  return request({
    url: '/gitegg-service-extension/extension/upload/file',
    method: 'post',
    data: formData
  })
}
export function dfsGetFileUrl (query) {
  return request({
    url: '/gitegg-service-extension/extension/get/file/url',
    method: 'get',
    params: query
  })
}

export function dfsDownloadFileUrl (query) {
return request({
  url: '/gitegg-service-extension/extension/get/file/download',
  method: 'get',
  responseType: 'blob',
  params: query
})
}

三、功能測試界面

1、批量上傳
上傳界面
2、文件流下載及獲取文件地址

備註

1、防止文件名重複，這裏文件名統一採用七牛雲的hash算法，可以防止文件重複，在界面需要展示的文件名，則存儲到數據庫一個文件名字段進行展示。所有的上傳文件都留有記錄。

GitEgg-Cloud是一款基於SpringCloud整合搭建的企業級微服務應用開發框架，開源項目地址:

Gitee: https://gitee.com/wmz1930/GitEgg

GitHub: https://github.com/wmz1930/GitEgg

全棧程序猿日志

@gitegg

标签

騰訊雲 (12)

oss (9)

minio (6)

七牛雲存儲 (5)

對象存儲 (4)

动态

SpringCloud微服務實戰——搭建企業級開發框架（二十九）：集成對象存儲服務MinIO+七牛雲+阿里雲+騰訊雲 - 动态详情

一、基礎底層庫實現

二、業務功能實現

三、功能測試界面

備註

GitEgg-Cloud是一款基於SpringCloud整合搭建的企業級微服務應用開發框架，開源項目地址:

Add a new 评论

全棧程序猿 日志

@gitegg

标签

騰訊雲 (12)

oss (9)

minio (6)

七牛雲存儲 (5)

對象存儲 (4)

动态

SpringCloud微服務實戰——搭建企業級開發框架（二十九）：集成對象存儲服務MinIO+七牛雲+阿里雲+騰訊雲 - 动态 详情

一、基礎底層庫實現

二、業務功能實現

三、功能測試界面

備註

GitEgg-Cloud是一款基於SpringCloud整合搭建的企業級微服務應用開發框架，開源項目地址:

Add a new 评论

全棧程序猿日志

SpringCloud微服務實戰——搭建企業級開發框架（二十九）：集成對象存儲服務MinIO+七牛雲+阿里雲+騰訊雲 - 动态详情