k8s-Harbor 高可用部署-01-Harbor 安装
一、Harbor基础
Harbor是一个用于存储和分发Docker镜像的企业级Registry服务器。
Harbor和Registry都是Docker的镜像仓库，但是Harbor作为更多企业的选择，是因为相比较于Regisrty来说，它具有很多的优势。
1. docker registry的缺点：
- 缺少认证机制，任何人都可以随意拉取及上传镜像，安全性缺失
- 缺乏镜像清理机制，镜像可以push却不能删除，日积月累，占用空间会越来越大
- 缺乏相应的扩展机制
2. Harbor优点
a.提供分层传输机制，优化网络传输
Docker镜像是是分层的，而如果每次传输都使用全量文件(所以用FTP的方式并不适合)，显然不经济。必须提供识别分层传输的机制，以层的UUID为标识，确定传输的对象。
b.提供WEB界面，优化用户体验
只用镜像的名字来进行上传下载显然很不方便，需要有一个用户界面可以支持登陆、搜索功能，包括区分公有、私有镜像。
c.支持水平扩展集群
当有用户对镜像的上传下载操作集中在某服务器，需要对相应的访问压力作分解。
d.良好的安全机制
企业中的开发团队有很多不同的职位，对于不同的职位人员，分配不同的权限，具有更好的安全性。
e.Harbor提供了基于角色的访问控制机制，并通过项目来对镜像进行组织和访问权限的控制。
kubernetes中通过namespace来对资源进行隔离，在企业级应用场景中，通过将两者进行结合可以有效将kubernetes使用的镜像资源进行管理和访问控制，增强镜像使用的安全性。尤其是在多租户场景下，可以通过租户、namespace和项目相结合的方式来实现对多租户镜像资源的管理和访问控制。
3.环境
- 系统版本：CentOS Linux release 7.6.1810 (Core)
- docker版本：20.10.12
- docker-compose版本：v2.3.2
二、高可用部署
课程演示环境节点规划：
高可用：
节点规划：在node-2、node-3 安装 Harbor，在 node-1 上安装 Nginx；
1、安装docker-compose
需要在两个worker 节点安装docker-compose
把在百度网盘下载的docker-compose文件到服务器，百度网盘下载地址：
链接：https://pan.baidu.com/s/16BaqfQsPPgpgP5BmOKAI0A
提取码：5z1b[root@zabbix 14:57:42~]# chmod +x docker-compose-linux-x86_64
[root@zabbix 14:57:48~]# mv docker-compose-linux-x86_64 /usr/local/sbin/docker-compose验证：
[root@homay04 harbor]# docker-compose --version
Docker Compose version v2.3.22、下载Harbor
在 GitHub 进行下载：https://github.com/goharbor/harbor/releases?page=5
wget https://github.com/goharbor/harbor/releases/download/v2.2.2/harbor-offline-installer-v2.2.2.tgz分发到 worker 节点上：
[root@homaybd03 softwards]# scp harbor-offline-installer-v2.2.2.tgz homaybd04:/opt/softwares/
harbor-offline-installer-v2.2.2.tgz 100% 481MB 34.4MB/s 00:14
[root@homaybd03 softwards]# scp harbor-offline-installer-v2.2.2.tgz homaybd05:/opt/softwares/
harbor-offline-installer-v2.2.2.tgz 100% 481MB 96.3MB/s 00:05
[root@homaybd03 softwards]#
解压缩：
tar -xvf harbor-offline-installer-v2.2.2.tgz 移动到安装/部署目录
[root@homaybd04 softwares]# ls -l
total 639740
-rw-r--r--. 1 root root 150242039 Apr 16 10:49 apache-dolphinscheduler-2.0.5-bin.tar.gz
drwxr-xr-x. 5 root root 90 Jun 4 15:23 cri-containerd
drwxr-xr-x. 2 root root 122 Jun 5 02:25 harbor
-rw-r--r--. 1 root root 504847710 Jun 5 02:22 harbor-offline-installer-v2.2.2.tgz
[root@homaybd04 softwares]# mv harbor /opt/modules/
查看目录：
[root@homaybd04 softwares]# cd /opt/modules/harbor/
[root@homaybd04 harbor]# ls -l
total 494976
-rw-r--r--. 1 root root 3361 May 15 2021 common.sh
-rw-r--r--. 1 root root 506818941 May 15 2021 harbor.v2.2.2.tar.gz
-rw-r--r--. 1 root root 7840 May 15 2021 harbor.yml.tmpl
-rwxr-xr-x. 1 root root 2500 May 15 2021 install.sh
-rw-r--r--. 1 root root 11347 May 15 2021 LICENSE
-rwxr-xr-x. 1 root root 1881 May 15 2021 prepare
[root@homaybd04 harbor]# 4、修改配置文件
[root@zabbix 15:00:06/data/software/harbor]# cp harbor.yml.tmpl harbor.ymlvim harbor.yml
数据存储位置改为/data/harbor/data ：
# 创建目录
[root@homaybd04 harbor]# mkdir -p /data/harbor/data需要挂载到一个大一点的盘上，一般为/根目录：
5、执行部署脚本
[root@hombd04 harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
✖ Need to install docker(17.06.0+) first and run this script again.
[root@homaybd04 harbor]#
5.1 没有安装docker，则需要进行安装：
全栈-02 Linux 安装 Docker 及配置
重新升级版本后，再进行安装：
[root@homaybd04 harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 20.10.16
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 2.3.2
[Step 2]: loading Harbor images ...
[Step 5]: starting Harbor ...
[+] Running 10/10
⠿ Network harbor_harbor Created 0.0s
⠿ Container harbor-log Started 0.7s
⠿ Container harbor-portal Started 1.5s
⠿ Container registry Started 1.6s
⠿ Container redis Started 1.3s
⠿ Container harbor-db Started 1.5s
⠿ Container registryctl Started 1.5s
⠿ Container harbor-core Started 1.9s
⠿ Container harbor-jobservice Started 2.4s
⠿ Container nginx Started 2.4s
✔ ----Harbor has been installed and started successfully.----
可以看到，harbor 已经安装成功了，默认账号密码为：admin/Harbor12345
以上安装完成后，再另一个worker节点安装一次。
两个节点的访问路径：
http://192.168.1.124:5000/harbor/
http://192.168.1.125:5000/harbor/
6、配置开机自启：
/usr/local/sbin/docker-compose -f /opt/modules/harbor/docker-compose.yml up -dharbor.yml 配置文件：
[root@homaybd05 harbor]# cat harbor.yml
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: 192.168.1.125
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 5000
# https related config
#https:
# https port for harbor, default is 443
# port: 443
# The path of cert and key files for nginx
#certificate: /your/certificate/path
#private_key: /your/private/key/path
# # Uncomment following will enable tls communication between all harbor components
# internal_tls:
# # set enabled to true means internal tls is enabled
# enabled: true
# # put your cert and key files on dir
# dir: /etc/harbor/tls/internal
# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433
# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: Harbor12345
# Harbor DB configuration
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: root123
# The maximum number of connections in the idle connection pool. If it <=0, no idle connections are retained.
max_idle_conns: 50
# The maximum number of open connections to the database. If it <= 0, then there is no limit on the number of open connections.
# Note: the default number of connections is 1024 for postgres of harbor.
max_open_conns: 1000
# The default data volume
data_volume: /data/harbor/data
# Harbor Storage settings by default is using /data dir on local filesystem
# Uncomment storage_service setting If you want to using external storage
# storage_service:
# # ca_bundle is the path to the custom root ca certificate, which will be injected into the truststore
# # of registry's and chart repository's containers. This is usually needed when the user hosts a internal storage with self signed certificate.
# ca_bundle:
# # storage backend, default is filesystem, options include filesystem, azure, gcs, s3, swift and oss
# # for more info about this configuration please refer https://docs.docker.com/registry/configuration/
# filesystem:
# maxthreads: 100
# # set disable to true when you want to disable registry redirect
# redirect:
# disabled: false
# Trivy configuration
#
# Trivy DB contains vulnerability information from NVD, Red Hat, and many other upstream vulnerability databases.
# It is downloaded by Trivy from the GitHub release page https://github.com/aquasecurity/trivy-db/releases and cached
# in the local file system. In addition, the database contains the update timestamp so Trivy can detect whether it
# should download a newer version from the Internet or use the cached one. Currently, the database is updated every
# 12 hours and published as a new release to GitHub.
trivy:
# ignoreUnfixed The flag to display only fixed vulnerabilities
ignore_unfixed: false
# skipUpdate The flag to enable or disable Trivy DB downloads from GitHub
#
# You might want to enable this flag in test or CI/CD environments to avoid GitHub rate limiting issues.
# If the flag is enabled you have to download the `trivy-offline.tar.gz` archive manually, extract `trivy.db` and
# `metadata.json` files and mount them in the `/home/scanner/.cache/trivy/db` path.
skip_update: false
#
# insecure The flag to skip verifying registry certificate
insecure: false
# github_token The GitHub access token to download Trivy DB
#
# Anonymous downloads from GitHub are subject to the limit of 60 requests per hour. Normally such rate limit is enough
# for production operations. If, for any reason, it's not enough, you could increase the rate limit to 5000
# requests per hour by specifying the GitHub access token. For more details on GitHub rate limiting please consult
# https://developer.github.com/v3/#rate-limiting
#
# You can create a GitHub token by following the instructions in
# https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line
#
# github_token: xxx
jobservice:
# Maximum number of job workers in job service
max_job_workers: 10
notification:
# Maximum retry count for webhook job
webhook_job_max_retry: 10
chart:
# Change the value of absolute_url to enabled can enable absolute url in chart
absolute_url: disabled
# Log configurations
log:
# options are debug, info, warning, error, fatal
level: info
# configs for logs in local storage
local:
# Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
rotate_count: 50
# Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes.
# If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G
# are all valid.
rotate_size: 200M
# The directory on your host that store log
location: /var/log/harbor
# Uncomment following lines to enable external syslog endpoint.
# external_endpoint:
# # protocol used to transmit log to external endpoint, options is tcp or udp
# protocol: tcp
# # The host of external endpoint
# host: localhost
# # Port of external endpoint
# port: 5140
#This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version: 2.2.0
# Uncomment external_database if using external database.
# external_database:
# harbor:
# host: harbor_db_host
# port: harbor_db_port
# db_name: harbor_db_name
# username: harbor_db_username
# password: harbor_db_password
# ssl_mode: disable
# max_idle_conns: 2
# max_open_conns: 0
# notary_signer:
# host: notary_signer_db_host
# port: notary_signer_db_port
# db_name: notary_signer_db_name
# username: notary_signer_db_username
# password: notary_signer_db_password
# ssl_mode: disable
# notary_server:
# host: notary_server_db_host
# port: notary_server_db_port
# db_name: notary_server_db_name
# username: notary_server_db_username
# password: notary_server_db_password
# ssl_mode: disable
# Uncomment external_redis if using external Redis server
# external_redis:
# # support redis, redis+sentinel
# # host for redis: <host_redis>:<port_redis>
# # host for redis+sentinel:
# # <host_sentinel1>:<port_sentinel1>,<host_sentinel2>:<port_sentinel2>,<host_sentinel3>:<port_sentinel3>
# host: redis:6379
# password:
# # sentinel_master_set must be set to support redis+sentinel
# #sentinel_master_set:
# # db_index 0 is for core, it's unchangeable
# registry_db_index: 1
# jobservice_db_index: 2
# chartmuseum_db_index: 3
# trivy_db_index: 5
# idle_timeout_seconds: 30
# Uncomment uaa for trusting the certificate of uaa instance that is hosted via self-signed cert.
# uaa:
# ca_file: /path/to/ca
# Global proxy
# Config http proxy for components, e.g. http://my.proxy.com:3128
# Components doesn't need to connect to each others via http proxy.
# Remove component from `components` array if want disable proxy
# for it. If you want use proxy for replication, MUST enable proxy
# for core and jobservice, and set `http_proxy` and `https_proxy`.
# Add domain to the `no_proxy` field, when you want disable proxy
# for some special registry.
proxy:
http_proxy:
https_proxy:
no_proxy:
components:
- core
- jobservice
- trivy
# metric:
# enabled: false
# port: 9090
# path: /metrics
知识补充：
1、如果以后修改了harbor.yml文件，那么先停止使用docker-compose down，再去修改harbor.yml文件，重新启动部署
./install.sh2、正常启动harbor启动：
# hombd04
# docker-compose -f /opt/modules/harbor/docker-compose.yml up -d
# hombd05
docker-compose -f /opt/module/harbor/docker-compose.yml up -d3、未修改配置文件，重启Harbor命令：
docker-compose start | stop | restart相关文章：
harbor安装
全栈-02 Linux 安装 Docker 及配置
为者常成，行者常至
自由转载-非商用-非衍生-保持署名（创意共享3.0许可证）
