filezilla多线程下载设置,简述hdfs上传文件工作流程

设置hosts相关信息1 .设置hosts

root@node130:~# vim /etc/hosts 10.172.200.130 node 13010.172.200.131 node 13110.172.200.132 node 13210.200.133 node 13310.172.2000

root @ node 1303360~# hostname CTL-- staticset-hostname node 130 root @ node 1303360~# hostname node 130 root @ node 1303360

设置二密自由登录的相关信息

1 .客户端生成公钥

# root @ node 130:~# ssh-keygengeneratingpublic/privatersakeypair.enterfileinwhichtosavethekey (3360 enter pass phrase (emptyfornopassphrase ) 3360 entersamepassphraseagain 3330 edin/root/. SSS id _ RSA.yourpublickeyhasbeensavedin/root/. ssh/id _ RSA.pub.thekeyfingerprintis : sha 25633607 e 50vj xjc9ixqvezir0ry 00 k a5 uibzrk 7yj qh0yk 7w root @ node 130 the key ' srandom aaaaat

2 .将公钥复制到相应的服务器

root @ node 1303360~# CD~~/. ssh/root @ node 1303360~/.ssh # ssh-copy-id-I~/.ssh/id _ RSA.pu pu root @ 1324 root/. ssh/id _ RSA.pub ' theauthenticityofhost ' ' can ' tbe established.ecdsakeyfingerprintissha 2563: GD/xarnlj 990 08 R3 rhomrhzvkuvrqrl0ynxbvw.areyousureyouwantto yes/usr /温柔的戒指/ssh-copy-id : info : attemptingtologinwithenewkeeker tofilteroutanythatarealreadyinstalled/usr /慈爱的戒指/ssh-copy-id : info :1 key (s ) ) ) ) )。 remaintobeinstalled-- ifyouarepromptednowitistoinstallthenewkeysroot @ 10.172.200.139 ' s password : number of key (s asa with : ' ssh ' root @ 10.172.200.131 ' andchecktomakesurethatonlythekey (

3 .显示相应服务器的许可信息

root @ node 131:~# CD~~/. ssh/root @ node 131:~/.ssh # ls authorized _ keys id _ r said _ RSA.pu bknown _

4 .测试登录

root @ node 130:~# ssh root @ 10.172.200.131 welcometouos 20g nu/Linux * hom

epage:https://www.chinauos.com/​   * Bugreport:https://bbs.chinauos.com/​​Last login: Sat Jun 26 11:00:00 2021 from 10.173.134.66root@node131:~# ​

每台机器上都执行以上操作

三 rsync相关操作

1. 各个服务器上分别安装 rsync

root@node130:~# apt -y install rsync正在读取软件包列表... 完成正在分析软件包的依赖关系树       正在读取状态信息... 完成       下列软件包是自动安装的并且现在不需要了： fbterm imageworsener libc-ares2 libde265-0 libheif1 liblqr-1-0 libmaxminddb0 libqtermwidget5-0 libsbc1 libsmi2ldbl libspandsp2 libutf8proc2 libwireshark-data libwireshark11 libwiretap8 libwscodecs2 libwsutil9 qtermwidget5-data squashfs-tools使用'apt autoremove'来卸载它(它们)。下列【新】软件包将被安装： rsync升级了 0 个软件包，新安装了 1 个软件包，要卸载 0 个软件包，有 232 个软件包未被升级。需要下载 391 kB 的归档。解压缩后会消耗 848 kB 的额外空间。获取:1 https://enterprise-packages.chinauos.com/server-enterprise fou/sp3/main mips64el rsync mips64el 3.1.3-6 [391 kB]已下载 391 kB，耗时 1秒 (643 kB/s)正在选中未选择的软件包 rsync。(正在读取数据库 ... 系统当前共安装有 180489 个文件和目录。)准备解压 .../rsync_3.1.3-6_mips64el.deb ...正在解压 rsync (3.1.3-6) ...正在设置 rsync (3.1.3-6) ...Created symlink /etc/systemd/system/multi-user.target.wants/rsync.service → /lib/systemd/system/rsync.service.正在处理用于 man-db (2.8.5-2) 的触发器 ...正在处理用于 systemd (241.8-1+dde) 的触发器 ...​

每台机器上都执行以上操作

2. 编写脚本，并把它放置在node130 上

#!/慈祥的戒指/bash# $#：表示传递给脚本或函数的参数个数。#1 获取输入参数个数，如果没有参数，直接退出pcount=$#if((pcount==0)); thenecho no args;exit;fi #2 获取文件名称p1=$1fname=`basename $p1`echo fname=$fname #3 获取上级目录到绝对路径pdir=`cd -P $(dirname $p1); pwd`echo pdir=$pdir #4 获取当前用户名称user=`whoami`echo user=$user​#5 循环for((host=1; host<9; host++)); do       #echo $pdir/$fname $user@node13$host:$pdir        echo --------------- node13$host ----------------       rsync -rvl $pdir/$fname $user@node13$host:$pdirdone

保存文件并命名为 xsync.sh

3. 授权

root@node130:~# chmod 777 xsync.sh root@node130:~# ls -l-rw-r--r--  1 root root   49063732 6月  25 11:35 seeyon.zip-rwxrwxrwx  1 root root        951 6月  26 14:00 xsync.sh​

4. 使用并测试

root@node130:~# ./xsync.sh seeyon.zip fname=seeyon.zippdir=/data/seeyon/G6N5/ApacheJetspeed/webappsuser=root--------------- node131 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node132 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node133 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node134 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node135 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node136 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  30,892.00 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node137 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.29--------------- node138 ----------------sending incremental file listseeyon.zip​sent 28,125 bytes received 49,105 bytes  51,486.67 bytes/sectotal size is 49,063,732 speedup is 635.295.查看各个节点文件是否分发成功root@node131:/data/seeyon/G6N5/ApacheJetspeed/webapps# ls -l总用量 47928drwxr-xr-x  3 root root     4096 2月  27 11:40 orgdrwxr-xr-x  4 root root     4096 2月  27 11:40 ROOTdrwxr-xr-x 26 root root     4096 6月  18 18:25 seeyon-rw-r--r--  1 root root 49063732 6月  26 14:03 seeyon.zip​