.05.25
初始seaweedFS
安装
1、先去GitHub下载所需版本
linux_amd64.tar.gz
2、然后放到服务器的文档中
/opt/software
3、解压安装
tar -zxvf ./linux_amd64.tar.gz -C /opt/install/seaweed
4、在seaweed安装包下安装seaweedfs文件夹及其下属两个文件夹
mkdir -p ./seaweedfs/data ./seaweedfs/volume
5、用以下命令查看master
./weed master -h
Example: weed master -port=9333Default Usage:-cpuprofile stringcpu profile output file-defaultReplication stringDefault replication type if not specified. (default "000")-disableHttpdisable http requests, only gRPC operations are allowed.-garbageThreshold floatthreshold to vacuum and reclaim spaces (default 0.3)-ip stringmaster <ip>|<server> address, also used as identifier (default "172.19.0.16")-ip.bind stringip address to bind to-mdir stringdata directory to store meta data (default "/tmp")-memprofile stringmemory profile output file-metrics.address stringPrometheus gateway address <host>:<port>-metrics.intervalSeconds intPrometheus push interval in seconds (default 15)-options stringa file of command line options, each line in optionName=optionValue format-peers stringall master nodes in comma separated ip:port list, example: 127.0.0.1:9093,127.0.0.1:9094,127.0.0.1:9095-port inthttp listen port (default 9333)-resumeStateresume previous state on start master server-volumePreallocatePreallocate disk space for volumes.-volumeSizeLimitMB uintMaster stops directing writes to oversized volumes. (default 30000)-whiteList stringcomma separated Ip addresses having write permission. No limit if empty.Description:start a master server to provide volume=>location mapping service and sequence number of file idsThe configuration file "security.toml" is read from ".", "$HOME/.seaweedfs/", "/usr/local/etc/seaweedfs/", or "/etc/seaweedfs/", in that order.The example security.toml configuration file can be generated by "weed scaffold -config=security"
6、用以下命令查看volume
./weed volume -h
Example: weed volume -port=8080 -dir=/tmp -max=5 -ip=server_name -mserver=localhost:9333Default Usage:-compactionMBps intlimit background compaction or copying speed in mega bytes per second-concurrentUploadLimitMB intlimit total concurrent upload size (default 128)-cpuprofile stringcpu profile output file-dataCenter stringcurrent volume server's data center name-dir stringdirectories to store data files. dir[,dir]... (default "/tmp")-dir.idx stringdirectory to store .idx files-disk string[hdd|ssd|<tag>] hard drive or solid state drive or any tag-fileSizeLimitMB intlimit file size to avoid out of memory (default 256)-idleTimeout intconnection idle seconds (default 30)-images.fix.orientationAdjust jpg orientation when uploading.-index stringChoose [memory|leveldb|leveldbMedium|leveldbLarge] mode for memory~performance balance. (default "memory")-ip stringip or server name, also used as identifier (default "172.19.0.16")-ip.bind stringip address to bind to-max stringmaximum numbers of volumes, count[,count]... If set to zero, the limit will be auto configured. (default "8")-memprofile stringmemory profile output file-metricsPort intPrometheus metrics listen port-minFreeSpace stringmin free disk space (value<=100 as percentage like 1, other as human readable bytes, like 10GiB). Low disk space will mark all volumes as ReadOnly.-minFreeSpacePercent stringminimum free disk space (default to 1%). Low disk space will mark all volumes as ReadOnly (deprecated, use minFreeSpace instead). (default "1")-mserver stringcomma-separated master servers (default "localhost:9333")-options stringa file of command line options, each line in optionName=optionValue format-port inthttp listen port (default 8080)-port.public intport opened to public-pprofenable pprof http handlers. precludes --memprofile and --cpuprofile-preStopSeconds intnumber of seconds between stop send heartbeats and stop volume server (default 10)-publicUrl stringPublicly accessible address-rack stringcurrent volume server's rack name-read.redirectRedirect moved or non-local volumes. (default true)-tcp<exprimental> enable tcp port-whiteList stringcomma separated Ip addresses having write permission. No limit if empty.Description:start a volume server to provide storage spaces
.05.26
认识SeaweedFS命令
seaweedFS是根据Haystack来实现的，它有两大特性：
1、存储数以亿计的文件
2、访问文件非常快
之所以它能达到这两个特性，是因为所有文件的元数据metadata不存储在master节点，而是分散存储在volume(逻辑卷，存储数据的逻辑结构)中，master节点只保存volume信息
3、查看master服务的参数：
4、查看volume服务参数
.05.27
在集群上启动seaweedFS
1、先检查集群上是否已经运行
ps aux | grep weed
结果：
root 17420 0.0 0.0 112728 972 pts/4 S+ 09:40 0:00 grep --color=auto weed
再检查下端口号
netstat -anp |grep 9333
2、分别在三台服务器上建立文件夹
mkdir -p ./seaweedfs/data ./seaweedfs/volume
此处学习一些Linux的命令，放于Linux的笔记中如：nohup、<、&等
2、启动master服务，分别在三台机器上启动
nohup ./weed master -ip=集群ip01 -port=9333 -mdir=./seaweedfs/data -peers=集群ip01:9333,集群ip02:9333,集群ip03:9333 > ./seaweedfs/data/master.log &
nohup ./weed master -ip=集群ip02 -port=9333 -mdir=./seaweedfs/data -peers=集群ip02:9333,集群ip01:9333,集群ip03:9333 > ./seaweedfs/data/master.log &
nohup ./weed master -ip=集群ip03 -port=9333 -mdir=./seaweedfs/data -peers=集群ip03:9333,集群ip01:9333,集群ip02:9333 > ./seaweedfs/data/master.log &
3、启动volume，分别在三台机器上启动
nohup ./weed volume -dataCenter dc1 -rack rack1 -dir ./seaweedfs/volume -ip=集群ip01 -port 9222 -ip.bind 集群ip01 -max 20 -mserver 集群ip01:9333,集群ip02:9333,集群ip03:9333 -publicUrl 集群ip01:9222 > ./seaweedfs/volume/volume.log &nohup ./weed volume -dataCenter dc1 -rack rack1 -dir ./seaweedfs/volume -ip=集群ip02 -port 9222 -ip.bind 集群ip02 -max 20 -mserver 集群ip01:9333,集群ip02:9333,集群ip03:9333 -publicUrl 集群ip02:9222 > ./seaweedfs/volume/volume.log &nohup ./weed volume -dataCenter dc1 -rack rack1 -dir ./seaweedfs/volume -ip=集群ip03 -port 9222 -ip.bind 集群ip03 -max 20 -mserver 集群ip01:9333,集群ip02:9333,集群ip03:9333 -publicUrl 集群ip03:9222 > ./seaweedfs/volume/volume.log &
4、打开http://集群ip01:9333/
文件操作
1、文件上传：先请求master,分配一个fid
curl http://集群ip01:9333/dir/assign
返回结果：
{"fid":"1,01da229ee5","url":"172.17.100.30:9222","publicUrl":"172.17.100.30:9222","count":1}
然后把test文件夹里的text文件上传上去，按照分配的fid上传
curl -F file=@./seaweedfs/test/test.txt 集群ip01:9222/1,01da229ee5
得到结果：
{"name":"test.txt","size":71,"eTag":"321d88cf6431a578fc8d586889275ff4","mime":"text/plain"}
2、也可以来操作删除
curl -X DELETE 集群ip01:9222/1,01da229ee5
3、解释说明：文件fid:1,01da229ee5
1代表volume id,volume id 是一个32位无符号整型
01代表file key，file key是一个64位无符号整型
da229ee5代表file cookie，file cookie是一个32位无符号整型，用来防止文件url被解
每个数字都以16进制保存，因此如果要保存成字符串的话，需要8+1+16+8一共33个字节
4、查询，下载都要先查到id
wget 集群ip01:9222/1,01da229ee5
filer
filer是一个在SeaweedFS之上的服务，它保存路径与文件id的映射关系，最终还是使用文件id来访问文件。filer服务和master服务保持一个长连接，获取并保存volume的信息。这样就不用每次请求都去查询volume的信息。
读：
1.从数据库中查询路径的元数据
2.从volume服务中读取文件内容并发送给请求
写：
1.客户端将文件传输给filer
2.filer将文件上传到volume服务，大文件被分成多个块
3.filer将元数据和块的元数据信息写到数据库中
1、先查看filer配置文件
./weed scaffold -config=filer
2、配置文件有一些如存储方式的选择，先选一个，根据红框里的配置式样配置一个,先创建filer文件
cd seaweedfs/touch filer.tomlll
创建好后，开始配置
vi filer.toml
[leveldb2]enabled = truedir = "."
3、保存退出后，执行启动命令
./weed filer -master="集群ip01:9333" -ip=集群ip01
已经启动
4、试一下上传文件，不需要再去找分配的fid上传
curl -F file=@/opt/finished/seaweedfs/test/test2.txt http://集群ip01:9333/text/
结果：{"error":"strconv.ParseUint: parsing \"text\": invalid syntax"}无效的语法
查看启动文件
修改：
curl -F file=@/opt/finished/seaweedfs/test/test2.txt http://172.17.100.30:8888/text/
{"name":"test2.txt","size":71}
以新的名称重命名上传后的文件
curl -F file=@/opt/finished/seaweedfs/test/test2.txt http://172.17.100.30:8888/text/new.txt
查看一下内容
curl "http://172.17.100.30:8888/text/new.txt"
文件内容正确，是我瞎写的一些字母：abcdefgaskldhgkasdjgalsd;gshkdgasd;hgwe;aoityawdgjs;adjgkjsd;ugtaoieutg
挂载
1、先建立一个挂载的文件夹
mkdir -p ./seaweedfs/data/mount
2、指定SeaweedFS挂载到新建的文件夹
./weed mount -filer=172.17.100.30:8888 -dir=./seaweedfs/data/mount -filer.path=/
3、上传文件
curl -F file=@/opt/finished/seaweedfs/test/abc.txt http://172.17.100.30:8888/text/
4、查看文件夹里
