Spring Framework 集成FTP

二叶草
二叶草
二叶草
1214
文章
0
评论
2020年3月20日20:53:00 评论 543

Spring Framework 集成FTP文件上传下载

1.实现背景及现实意义

在系统晚间批量的时候需要从核心系统同步借据详情、还款流水、五级分类等相关信息。如果所有的信息同步使用rpc调用,功能固然可以实现,但是从效率和核心系统的压力来看是不合算的,所以此间设计批量文件的传输。主要目的是提高系统交互的效率,减少系统压力,增加数据的一致性。

2.什么是ftp文件传输

FTP(File Transfer Protocol)是文件传输协议的简称。正如其名所示:FTP的主要作用,就是让用户连接上一个远程计算机(这些计算机上运行着FTP服务器程序)察看远程计算机有哪些文件,然后把文件从远程计算机上拷到本地计算机,或把本地计算机的文件送到远程计算机去。

3.ftp文件传输在java中的实现

3.1Maven依赖

<dependency>
    <groupId>commons-net</groupId>
    <artifactId>commons-net</artifactId>
    <version>3.5</version>
</dependency>

3.2ftp相关config封装

/**
 * @ClassName: FtpConfig 
 * @Description: ftp参数集合封装
 * @Author: 尚先生
 * @CreateDate: 2019/4/22 8:30
 * @Version: 1.0
 */
public class FtpConfig {

    private String server;
    private int port;
    private String username;
    private String password;

    // 客户端模式:
    private int clientMode;

    // 文件传输类型
    private int fileType = FTP.BINARY_FILE_TYPE;

    // 缓存大小
    private int bufferSize = 2048;

    private String path;

    public String getServer() {
        return server;
    }

    public void setServer(String server) {
        this.server = server;
    }

    public int getPort() {
        return port;
    }

    public void setPort(int port) {
        this.port = port;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public String getPath() {
        return path;
    }

    public void setPath(String path) {
        this.path = path;
    }

    public int getClientMode() {
        return clientMode;
    }

    public void setClientMode(int clientMode) {
        this.clientMode = clientMode;
    }

    public int getFileType() {
        return fileType;
    }

    public void setFileType(int fileType) {
        this.fileType = fileType;
    }

    public int getBufferSize() {
        return bufferSize;
    }

    public void setBufferSize(int bufferSize) {
        this.bufferSize = bufferSize;
    }
}

3.3ftp工具类实现

/**
 * @ClassName: FtpClientUtils 
 * @Description: ftp工具类
 * @Author: 尚先生
 * @CreateDate: 2019/4/23 9:10
 * @Version: 1.0
 */
public class FtpClientUtils {

    private Logger LOGGER = LoggerFactory.getLogger(FtpClientUtils.class);

    private FtpConfig ftpConfig;

    private String controlEncoding = "UTF-8";

    private String localFileEncoding = "UTF-8";

    private String sendFtpFileNameEncoding = "iso-8859-1";

    /**
     * 连接ftp服务器
     * @throws SocketException
     * @throws IOException
     */
    private boolean connectServer(FTPClient ftpClient) throws SocketException,
            IOException {
        boolean isConnected = false;
        String server = ftpConfig.getServer();
        int port = ftpConfig.getPort();
        String user = ftpConfig.getUsername();
        String password = ftpConfig.getPassword();
        String path = ftpConfig.getPath();
        ftpClient.connect(server, port);
        LOGGER.info("连接发图片文件服务器 " + server + ".");
        if (FTPReply.isPositiveCompletion(ftpClient.getReplyCode())) {
            if (ftpClient.login(user, password)) {
                //如果服务器支持就用UTF-8编码,否则就使用本地编码
                if (FTPReply.isPositiveCompletion(ftpClient.sendCommand("OPTS UTF8", "ON"))) {
                    localFileEncoding = "UTF-8";
                }
                if (StringUtils.isNotBlank(path)) {
                    ftpClient.changeWorkingDirectory(path);
                }
                isConnected = true;
            }
        }
        return isConnected;
    }

    /**
     * 断开与远程服务器的连接
     * @throws IOException
     */
    private void disconnect(FTPClient ftpClient) throws IOException {
        if (ftpClient.isConnected()) {
            ftpClient.disconnect();
        }
    }

   

    /**
     * 得到某个目录下的文件名列表
     * @param path
     * @return
     * @throws IOException
     */
    @SuppressWarnings("unused")
    private List<String> getFileList(String path, FTPClient ftpClient) throws IOException {
        FTPFile[] ftpFiles = ftpClient.listFiles(path);
        List<String> retList = new ArrayList<String>();
        if (ftpFiles == null || ftpFiles.length == 0) {
            return retList;
        }

        for (FTPFile ftpFile : ftpFiles) {
            if (ftpFile.isFile()) {
                retList.add(ftpFile.getName());
            }
        }
        return retList;
    }

    /**
     * 功能: 删除文件
     * @param pathName
     * @return
     * @throws IOException
     */
    private boolean deleteFile(String pathName, FTPClient ftpClient) throws IOException {
        return ftpClient.deleteFile(pathName);
    }


    /**
     * 功能: 下载文件
     * @param sourceFileName
     * @return
     * @throws IOException
     */
    private InputStream downFile(String sourceFileName, FTPClient ftpClient) throws IOException {
        return ftpClient.retrieveFileStream(sourceFileName);
    }

    /**
     * 两种支持的模式
     */
    private void updateClientMode(FTPClient client) {
        int clientMode = ftpConfig.getClientMode();
        switch (clientMode) {
            case FTPClient.ACTIVE_LOCAL_DATA_CONNECTION_MODE:
                // 主动模式
                client.enterLocalActiveMode();
                break;
            case FTPClient.PASSIVE_LOCAL_DATA_CONNECTION_MODE:
                // 被动模式
                client.enterLocalPassiveMode();
                break;
            default:
                break;
        }
    }


    /**
     * 从FTP服务器上下载文件,支持断点续传,下载百分比汇报
     * @param remote 远程文件路径及名称
     * @param local  本地文件完整绝对路径
     * @return 下载的状态
     * @throws IOException
     */
    public DownloadStatus download(String remote, String local) throws IOException {
        DownloadStatus result = DownloadStatus.serverConntionFail;
        FTPClient ftpClient = new FTPClient();
        try {
            if (connectServer(ftpClient)) {
                updateClientMode(ftpClient);
                // 设置以二进制方式传输
                ftpClient.setFileType(ftpConfig.getFileType());
                ftpClient.setBufferSize(ftpClient.getBufferSize());
                // 检查远程文件是否存在
                FTPFile[] files = ftpClient.listFiles(new String(remote.getBytes(localFileEncoding), sendFtpFileNameEncoding));
                if (files.length != 1) {
                    LOGGER.info("远程文件[{}]不存在", new Object[]{remote});
                    return DownloadStatus.RemoteFileNotExist;
                }
                long lRemoteSize = files[0].getSize();
                File localFile = new File(local);
                //  先删除本地文件,不需要断点续传功能
                localFile.delete();
                OutputStream out = new FileOutputStream(localFile);
                InputStream in = ftpClient.retrieveFileStream(new String(remote.getBytes(localFileEncoding), sendFtpFileNameEncoding));
                byte[] bytes = new byte[1024];
                long step = lRemoteSize / 100;
                // 文件过小,step可能为0
                step = step == 0 ? 1 : step;
                long process = 0;
                long localSize = 0L;
                int c;
                while ((c = in.read(bytes)) != -1) {
                    out.write(bytes, 0, c);
                    localSize += c;
                    long nowProcess = localSize / step;
                    if (nowProcess > process) {
                        process = nowProcess;
                        if (process % 10 == 0) {
                            LOGGER.info("下载进度:" + process);
                        }
                    }
                }
                in.close();
                out.close();
                boolean upNewStatus = ftpClient.completePendingCommand();
                if (upNewStatus) {
                    result = DownloadStatus.DownloadNewSuccess;
                } else {
                    result = DownloadStatus.DownloadNewFailed;
                }
            }
            return result;
        } finally {
            disconnect(ftpClient);
        }

    }

    /**
     * 上传文件到FTP服务器,支持断点续传
     * @param local  本地文件名称,绝对路径
     * @param remote 远程文件路径,按照Linux上的路径指定方式,支持多级目录嵌套,支持递归创建不存在的目录结构
     * @return 上传结果
     * @throws IOException
     */
    public UploadStatus upload(String local, String remote) throws IOException {
        UploadStatus result = UploadStatus.serverConntionFail;
        FTPClient ftpClient = new FTPClient();
        try {
            if (connectServer(ftpClient)) {
                // 设置PassiveMode传输
                ftpClient.enterLocalPassiveMode();
                // 设置以二进制流的方式传输
                ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
                ftpClient.setControlEncoding(controlEncoding);
                // 对远程目录的处理
                String remoteFileName = remote;
                if (remote.contains("/")) {
                    remoteFileName = remote.substring(remote.lastIndexOf("/") + 1);
                    // 创建服务器远程目录结构,创建失败直接返回
                    if (createDirecroty(remote, ftpClient) == UploadStatus.CreateDirectoryFail) {
                        return UploadStatus.CreateDirectoryFail;
                    }
                }
                // 检查远程是否存在文件
                FTPFile[] files = ftpClient.listFiles(new String(remoteFileName.getBytes(localFileEncoding), sendFtpFileNameEncoding));
                if (files.length == 1) {
                    ftpClient.deleteFile(remoteFileName);
                    File localFile = new File(local);
                    // 尝试移动文件内读取指针,实现断点续传
                    result = uploadFile(remoteFileName, localFile, ftpClient, 0);
                    // 如果断点续传没有成功,则删除服务器上文件,重新上传
                    if (result == UploadStatus.UploadFromBreakFailed) {
                        if (!ftpClient.deleteFile(remoteFileName)) {
                            return UploadStatus.DeleteRemoteFaild;
                        }

                        result = uploadFile(remoteFileName, localFile, ftpClient, 0);
                    }
                } else {
                    result = uploadFile(remoteFileName, new File(local), ftpClient, 0);
                }

            }
            return result;
        } finally {
            disconnect(ftpClient);
        }
    }

    /**
     * 递归创建远程服务器目录
     * @param remote     远程服务器文件绝对路径
     * @param _ftpClient FTPClient对象
     * @return 目录创建是否成功
     * @throws IOException
     */
    private UploadStatus createDirecroty(String remote, FTPClient _ftpClient) throws IOException {
        UploadStatus status = UploadStatus.CreateDirectorySuccess;
        String directory = remote.substring(0, remote.lastIndexOf("/") + 1);
        if (!directory.equals("/") &&
                !_ftpClient.changeWorkingDirectory(new String(directory.getBytes(localFileEncoding), sendFtpFileNameEncoding))) {
            // 如果远程目录不存在,则递归创建远程服务器目录
            int start = 0;
            int end = 0;
            if (directory.startsWith("/")) {
                start = 1;
            } else {
                start = 0;
            }

            end = directory.indexOf("/", start);
            while (true) {
                String subDirectory = new String(remote.substring(start, end).getBytes(localFileEncoding), sendFtpFileNameEncoding);
                if (!_ftpClient.changeWorkingDirectory(subDirectory)) {
                    if (_ftpClient.makeDirectory(subDirectory)) {
                        _ftpClient.changeWorkingDirectory(subDirectory);
                    } else {
                        LOGGER.info("创建目录失败");
                        return UploadStatus.CreateDirectoryFail;
                    }
                }
                start = end + 1;
                end = directory.indexOf("/", start);
                // 检查所有目录是否创建完毕
                if (end <= start) {
                    break;
                }
            }
        }

        return status;
    }

    /**
     * 上传文件到服务器,新上传和断点续传
     * @param remoteFile 远程文件名,在上传之前已经将服务器工作目录做了改变
     * @param localFile  本地文件File句柄,绝对路径
     * @param _ftpClient FTPClient引用
     * @return remoteSize 远程大小
     * @throws IOException
     */
    private UploadStatus uploadFile(String remoteFile, File localFile, FTPClient _ftpClient, long remoteSize) throws IOException {
        // 显示进度的上传
        UploadStatus status = null;
        LOGGER.info("localFile.length():" + localFile.length());
        long step = localFile.length() / 100;
        // 文件过小,step可能为0
        step = step == 0 ? 1 : step;
        long process = 0;
        long localreadbytes = 0L;
        RandomAccessFile raf = new RandomAccessFile(localFile, "r");
        OutputStream out = _ftpClient.appendFileStream(new String(remoteFile.getBytes(localFileEncoding), sendFtpFileNameEncoding));
        // 断点续传
        if (remoteSize > 0) {
            _ftpClient.setRestartOffset(remoteSize);
            process = remoteSize / step;
            raf.seek(remoteSize);
            localreadbytes = remoteSize;
        }

        byte[] bytes = new byte[1024];
        int c;
        while ((c = raf.read(bytes)) != -1) {
            out.write(bytes, 0, c);
            localreadbytes += c;
            if (localreadbytes / step != process) {
                process = localreadbytes / step;
                if (process % 10 == 0) {
                    LOGGER.info("文件上传进度:" + process);
                }
            }
        }

        out.flush();
        raf.close();
        out.close();
        boolean result = _ftpClient.completePendingCommand();
        if (remoteSize > 0) {
            status = result ? UploadStatus.UploadFromBreakSuccess : UploadStatus.UploadFromBreakFailed;
        } else {
            status = result ? UploadStatus.UploadNewFileSuccess : UploadStatus.UploadNewFileFailed;
        }
        return status;
    }

    public enum UploadStatus {
        //服务器连接失败
        serverConntionFail,
        // 远程服务器相应目录创建失败
        CreateDirectoryFail,
        // 远程服务器创建目录成功
        CreateDirectorySuccess,
        // 上传新文件成功
        UploadNewFileSuccess,
        // 上传新文件失败
        UploadNewFileFailed,
        // 文件已经存在
        FileExits,
        // 远程文件大于本地文件
        RemoteFileBiggerThanLocalFile,
        // 断点续传成功
        UploadFromBreakSuccess,
        // 断点续传失败
        UploadFromBreakFailed,
        // 删除远程文件失败
        DeleteRemoteFaild;
    }

    public enum DownloadStatus {
        //服务器连接失败
        serverConntionFail,
        // 远程文件不存在
        RemoteFileNotExist,
        // 下载文件成功
        DownloadNewSuccess,
        // 下载文件失败
        DownloadNewFailed,
        // 本地文件大于远程文件
        LocalFileBiggerThanRemoteFile,
        // 断点续传成功
        DownloadFromBreakSuccess,
        // 断点续传失败
        DownloadFromBreakFailed;
    }

    public void setSendFtpFileNameEncoding(String sendFtpFileNameEncoding) {
        this.sendFtpFileNameEncoding = sendFtpFileNameEncoding;
    }

    public void setLocalFileEncoding(String localFileEncoding) {
        this.localFileEncoding = localFileEncoding;
    }

    public void setFtpConfig(FtpConfig ftpConfig) {
        this.ftpConfig = ftpConfig;
    }

    public void setControlEncoding(String controlEncoding) {
        this.controlEncoding = controlEncoding;
    }
}

3.4测试类实现

/**
 * @ClassName: FtpClientUtilsTest 
 * @Description: ftp测试工具类
 * @Author: 尚先生
 * @CreateDate: 2019/4/25 9:18
 * @Version: 1.0
 */

public class FtpClientUtilsTest {
    public static void main(String[] args) {
        // 核心文件描述:remoteRootDirPath+"/"+trandate+"/"+coreFileName
        // 本地文件描述:localRootDirPath+"/"+trandate+"/"+coreFileName
        // 设置交易日期
        String trandate = "2019-04-25";
        // 配置参数 路径+交易日期
        String localRootDirPath = "/home/loan/dev/sysCode";
        String remoteRootDirPath = "D:/core";    
        String coreFileNames = "duebillInfo_%s.dat,repayInfo_%s.dat";
        trandate = trandate.replace("-", "");
        String localDirPath = localRootDirPath.concat("/").concat(trandate).concat("/");
        String remoteDirPath = remoteRootDirPath.concat("/").concat(trandate).concat("/");
        for (String coreFileName : coreFileNames.split(",")) {
            // 文件名称
            coreFileName = String.format(coreFileName.trim(), trandate);
            // 本地文件路径
            String localCoreFilePath = localDirPath.concat(coreFileName);
            String remoteCoreFilePath = remoteDirPath.concat(coreFileName);
            // 从spring容器中获得ftp连接
            AbstractXmlApplicationContext applicationContext = new ClassPathXmlApplicationContext("application.xml");
        applicationContext.refresh();
        FtpClientUtils ftpClientUtils = applicationContext.getBean("ftpClientUtils",FtpClientUtils.class);
            FtpClientTemple.DownloadStatus downloadStatus = ftpClientUtils.download(remoteCoreFilePath, localCoreFilePath);
            if (!FtpClientTemple.DownloadStatus.DownloadFromBreakSuccess.equals(downloadStatus) && !FtpClientTemple.DownloadStatus.DownloadNewSuccess.equals(downloadStatus)) {
                return RepeatStatus.CONTINUABLE;
            }
        }
        return RepeatStatus.FINISHED;
    }
}

3.5配置文件

<bean id="coreFtpConfig" class="com.sxs.com.ftp.FtpConfig">
        <property name="server" value="${ftp.host}"/>
        <property name="port" value="${ftp.port}"/>
        <property name="username" value="${ftp.username}"/>
        <property name="password" value="${ftp.password}"/>
        <property name="fileType" value="#{T(org.apache.commons.net.ftp.FTP).BINARY_FILE_TYPE}"/>
        <property name="clientMode" value="#{T(org.apache.commons.net.ftp.FTPClient).PASSIVE_LOCAL_DATA_CONNECTION_MODE}"/>
    </bean>

    <bean id="ftpClientUtils" class="com.sxs.com.ftp.FtpClientUtils">
        <property name="ftpConfig" ref="coreFtpConfig"/>
    </bean>

3.6测试结果

执行测试类,测试下载方法
服务调用结果为成功

打开 Git Bash Here
cd D:core
ll
    20190425
cd 20190425
ll -als
    ./
    ../
    duebillInfo_20190425.dat
    repayInfo_20190425.dat

本文来源于:Spring Framework 集成FTP-变化吧门户
特别声明:以上文章内容仅代表作者本人观点,不代表变化吧门户观点或立场。如有关于作品内容、版权或其它问题请于作品发表后的30日内与变化吧联系。

转载请注明:{{title}}-变化吧
  • 赞助本站
  • 微信扫一扫
  • weinxin
  • 赞助本站
  • 支付宝扫一扫
  • weinxin
二叶草
FTP-主动模式和被动模式的分析 ftp工具

FTP-主动模式和被动模式的分析

FTP,很多人都非常熟悉了。很多FTP实现的软件使用主动模式来传输数据,那主动模式和被动模式的区别在哪呢? 为啥有了主动模式还有被动模式呢? 借助wireshark来分析FTP主动模式和被动模式的区别...