Gitlab升级之路10.8.7 -> 11.11.8 -> 12.0.12 -> 12.1.17 -> 12.10.14 -> 13.0.14 -> 13.1.11 -> 13.5.4 -> 13.10.3
2021-04-21 作者 moneyslow

Gitlab因存在漏洞，必须进行升级。

https://help.aliyun.com/noticelist/articleid/1060824788.html
https://help.aliyun.com/noticelist/articleid/1060819310.html

本文以Gitlab的Omnibus方便版本来说明从低版本10.8.7 升级到13.10.3 的步骤

官方升级文档：https://docs.gitlab.com/omnibus/update/
重点在这里，版本升级要按部就班，不能步子迈的太大，扯到蛋：
https://docs.gitlab.com/ee/policy/maintenance.html#upgrade-recommendations
于是最终决定的升级路径：
10.8.7 -> 11.11.8 -> 12.0.12 -> 12.1.17 -> 12.10.14 -> 13.0.14 -> 13.1.11 -> 13.5.4 -> 13.10.3

从下载地址把上面所有的安装包下载下来：
https://packages.gitlab.com/gitlab/gitlab-ce

例如我的下载列表：
gitlab-ce-10.8.7-ce.0.el7.x86_64.rpm
gitlab-ce-12.10.14-ce.0.el7.x86_64.rpm
gitlab-ce-13.10.3-ce.0.el7.x86_64.rpm
gitlab-ce-11.11.8-ce.0.el7.x86_64.rpm
gitlab-ce-12.1.17-ce.0.el7.x86_64.rpm
gitlab-ce-13.1.11-ce.0.el7.x86_64.rpm
gitlab-ce-12.0.12-ce.0.el7.x86_64.rpm
gitlab-ce-13.0.14-ce.0.el7.x86_64.rpm
gitlab-ce-13.5.4-ce.0.el7.x86_64.rpm

在你有能力升级之前，请确定你先有能力备份恢复，并且亲自测试！
备份恢复的文档： https://docs.gitlab.com/ee/raketasks/backup_restore.html
备份恢复的重点是这些配置文件：
For Omnibus:
/etc/gitlab/gitlab-secrets.json
/etc/gitlab/gitlab.rb

####### 备份命令需要注意！，不同版本的命令不一样！ #########
在使用 Omnibus package的情况下：
GitLab 12.2 or later:
gitlab-backup create
GitLab 12.1 and earlier:
gitlab-rake gitlab:backup:create
##############################################
如果你现在版本是10.8之前的，需要升级PostgreSQL数据库：
10.0 GitLab requires the version of PostgreSQL to be 9.6 or higher.
执行操作：gitlab-ctl pg-upgrade
确定升级后的PostgreSQL版本是9.6：
/opt/gitlab/embedded/bin/psql --version

#################### 升级到10.8.7 ##########################
现在开始升级，先关几个服务：
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq

开始升级：
#yum update gitlab-ce-10.8.7-ce.0.el7.x86_64.rpm
可能有些参数报错，是因为有些参数更改或者不需要了，根据提示修改 /etc/gitlab/gitlab.rb
重新生成配置文件：

gitlab-ctl reconfigure

再次执行升级命令：
#yum update gitlab-ce-10.8.7-ce.0.el7.x86_64.rpm
升级成功后，查看最新版本：
cat /opt/gitlab/embedded/service/gitlab-rails/VERSION

################## 从10.8.7 升级到 11.11.8 #######################
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq

根据：https://docs.gitlab.com/omnibus/update/gitlab_11_changes.html
11.11.8大版本需要修改 gitlab.rb的2个地方：
1、增加 nginx[‘ssl_protocols’] = “TLSv1.2”
2、旧格式去掉 git_data_dir “/data/git-data” 增加新格式 git_data_dirs({ “default” => { “path” => “/data/git-data” } })
然后执行：
gitlab-ctl reconfigure
开始升级：大概2分钟
rpm -Uvh gitlab-ce-11.11.8-ce.0.el7.x86_64.rpm

升级完成，有提示：
Warnings:
The version of the running postgresql service is different than what is installed.
Please restart postgresql to start the new version.
sudo gitlab-ctl restart postgresql

按提示执行：gitlab-ctl restart postgresql

gitlab-ctl restart
gitlab-ctl status 查看应该正常了。

查看版本号：
[root@gitlab soft]# rpm -qa|grep gitlab
gitlab-ce-11.11.8-ce.0.el7.x86_64
[root@gitlab soft]# cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
11.11.8

############ 11.11.8 升级到 12.0.12 ##############
Sidekiq 是Ruby 和Rails 项目中常用的后台任务处理系统，查看方法：https://你的gitlab域名.com/admin/sidekiq
记录一下PostgreSQL的版本为9.6:

/opt/gitlab/embedded/bin/psql --version

psql (PostgreSQL) 9.6.11

开始升级，照例停几个服务：
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq

这个版本需要升级prometheus，大概15分钟

[root@gitlab ~]# gitlab-ctl prometheus-upgrade
Converting existing data to new format is a time consuming process and can take hours.
If you prefer not to migrate existing data, press Ctrl-C now and re-run the command with --skip-data-migration flag.
Waiting for 30 seconds for input.

Please hit Ctrl-C now if you want to cancel the operation.
…
会人性化显示大概的时间倒计时：(120G 数据量，预估1个半小时)
326 / 1440 22.64% 1h11m39s
。。。
Running handlers:
Running handlers complete
Chef Client finished, 11/663 resources updated in 10 seconds
Starting prometheus
ok: run: prometheus: (pid 32372) 0s
Prometheus upgrade completed. You are now running Prometheus version 2
Old data directory has been backed up to /var/opt/gitlab/prometheus/data_tmp.
prometheus升级完成，看到已经运行版本2.

执行：
gitlab-ctl reconfigure

正式开始升级：
[root@gitlab soft]# rpm -Uvh gitlab-ce-12.0.12-ce.0.el7.x86_64.rpm
。。。

==== Upgrade has completed ====
Please verify everything is working and run the following if so
rm -rf /var/opt/gitlab/postgresql/data.9.6
。。。
完成。
查看版本号：
[root@gitlab soft]# rpm -qa|grep gitlab
gitlab-ce-12.0.12-ce.0.el7.x86_64
[root@gitlab soft]# cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
12.0.12

Gitlab-ctl restart 如果遇到报错 FATAL: terminating connection due to administrator command
重启 gitlab-ctl restart unicorn
这个版本注意，要关闭runner的自动运行，否则任何项目都会使用runner：
settings CI/CD auto devops，关闭《Default to Auto DevOps pipeline》和《Enable shared runners for new projects》。

如果使用runner，必须升级相应的gitlab-runner

############################## 12.0.12 升级到 12.1.17 测试 #####################
查看PostgreSQL的版本：
[root@gitlab soft]# /opt/gitlab/embedded/bin/psql --version
psql (PostgreSQL) 10.7

开始升级：
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-ctl stop nginx
rpm -Uvh gitlab-ce-12.1.17-ce.0.el7.x86_64.rpm
大概2分钟后完成。
查看版本：
[root@gitlab soft]# rpm -qa|grep gitlab
gitlab-ce-12.1.17-ce.0.el7.x86_64
[root@gitlab soft]# cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
12.1.17

########################## 12.1.17 升级到 12.10.14 测试 ###############
当前PostgreSQL的版本：
[root@gitlab soft]# /opt/gitlab/embedded/bin/psql --version
psql (PostgreSQL) 10.7
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-ctl stop nginx
这个版本的PostgreSQL会自动升级：PostgreSQL will automatically be upgraded to 11.x
开始升级：
rpm -Uvh gitlab-ce-12.10.14-ce.0.el7.x86_64.rpm
gitlab-ctl restart
升级后的PostgreSQL 的版本：
[root@gitlab soft]# /opt/gitlab/embedded/bin/psql --version
psql (PostgreSQL) 11.7

同时注意！！！！ 12.2版本以后备份的命令变为 gitlab-backup create

######################### 12.10.14 升级到 13.0.14 #########################

此版本变动如下：
1、Puma becoming default web server instead of Unicorn
2、PostgreSQL 11 becoming minimum required version，这个我们搞定了已经。

开始升级：
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-ctl stop nginx

rpm -Uvh gitlab-ce-13.0.14-ce.0.el7.x86_64.rpm
gitlab-ctl restart

查看版本：
[root@gitlab soft]# cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
13.0.14
[root@gitlab ~]# rpm -qa|grep gitlab
gitlab-ce-13.0.14-ce.0.el7.x86_64

升级后运行check脚本：
gitlab-rake gitlab:check SANITIZE=true
。。。。。
818/4072 … yes
819/4073 … yes
484/4074 … yes
153/4075 … yes
Redis version >= 4.0.0? … yes
Ruby version >= 2.5.3 ? … yes (2.6.6)
Git version >= 2.22.0 ? … yes (2.26.2)
Git user has default SSH configuration? … yes
Active users: … 561
Is authorized keys file accessible? … yes
GitLab configured to store new projects in hashed storage? … yes
All projects are in hashed storage? … no
Try fixing it:
Please migrate all projects to hashed storage
as legacy storage is deprecated in 13.0 and support will be removed in 14.0.
For more information see:
doc/administration/repository_storage_types.md

Checking GitLab App … Finished
Checking GitLab subtasks … Finished

一切正常，Git version显示为2.26.2

########################## 13.0.14 升级到 13.1.11 #################
关闭服务
gitlab-ctl stop unicorn
gitlab-ctl stop puma
gitlab-ctl stop sidekiq

升级：
rpm -Uvh gitlab-ce-13.1.11-ce.0.el7.x86_64.rpm

执行check：
gitlab-rake gitlab:check SANITIZE=true
。。。。
825/4100 … yes
825/4101 … yes
521/4102 … yes
Redis version >= 4.0.0? … yes
Ruby version >= 2.5.3 ? … yes (2.6.6)
Git version >= 2.22.0 ? … yes (2.27.0)
Git user has default SSH configuration? … yes
Active users: … 560
Is authorized keys file accessible? … yes
GitLab configured to store new projects in hashed storage? … yes
All projects are in hashed storage? … no
Try fixing it:
Please migrate all projects to hashed storage
as legacy storage is deprecated in 13.0 and support will be removed in 14.0.
For more information see:
doc/administration/repository_storage_types.md

Checking GitLab App … Finished
Checking GitLab subtasks … Finished

一切正常，Git version显示为2.27.0

################## 13.1.11 升级到 13.5.4 #############
gitlab-ctl stop puma
gitlab-ctl stop sidekiq

rpm -Uvh gitlab-ce-13.5.4-ce.0.el7.x86_64.rpm
安装完成后
gitlab-rake gitlab:check SANITIZE=true

153/4075 … yes
Redis version >= 4.0.0? … yes
Ruby version >= 2.5.3 ? … yes (2.6.6)
Git version >= 2.24.0 ? … yes (2.28.0)
Git user has default SSH configuration? … yes
Active users: … 561
Is authorized keys file accessible? … yes
GitLab configured to store new projects in hashed storage? … yes
All projects are in hashed storage? … no
Try fixing it:
Please migrate all projects to hashed storage
as legacy storage is deprecated in 13.0 and support will be removed in 14.0.
For more information see:
doc/administration/repository_storage_types.md

Checking GitLab App … Finished
Checking GitLab subtasks … Finished

一切正常，Git version显示为2.27.0

####### 13.5.4 升级到 13.10.3 #########
gitlab-ctl stop puma
gitlab-ctl stop sidekiq

rpm -Uvh gitlab-ce-13.10.3-ce.0.el7.x86_64.rpm

[root@gitlab soft]# /opt/gitlab/embedded/bin/psql --version
psql (PostgreSQL) 12.6
[root@gitlab soft]# cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
13.10.3

执行检查：
[root@gitlab soft]# gitlab-rake gitlab:check SANITIZE=true

819/4073 … yes
484/4074 … yes
153/4075 … yes
Redis version >= 4.0.0? … yes
Ruby version >= 2.7.2 ? … yes (2.7.2)
Git version >= 2.29.0 ? … yes (2.29.0)
Git user has default SSH configuration? … yes
Active users: … 561
Is authorized keys file accessible? … yes
GitLab configured to store new projects in hashed storage? … yes
All projects are in hashed storage? … yes

Checking GitLab App … Finished
Checking GitLab subtasks … Finished

一切正常，Git version显示为2.28.0 ，同时存储变为 hashed storage