家庭工作站打造之旅(三)–仓库服务器设置

原理简介

DevOps(英文Development和Operations的组合)是一组过程、方法与系统的统称,用于促进开发(应用程序/软件工程)、技术运营和质量保障(QA)部门之间的沟通、协作与整合。它的出现是由于软件行业日益清晰地认识到:为了按时交付软件产品和服务,开发和运营工作必须紧密合作。

家庭工作站的目的其实就是是构造一个DevOps环境,提升我们开发、测试与交付的能力,而仓库服务器即作为整个工作站的核心,存放了自己及他人的全部劳动成果。

软件安装

为了完成DevOps的实践,我们整个仓库服务器的搭建采用以下软件的安装与配合:

java及maven 安装与配置

java8的安装

参考文档:https://linux.cn/article-3792-1.html
在Ubuntu上安装Oracle JDK 使用下面的命令安装,确保你的网络环境良好,它会下载许多的文件,需要花费一些时间:

安装步骤:

1
2
3
4
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt-get install oracle-java8-set-default`</pre>

安装检查:

1
2
3
4
tiger@repo:~# java -version
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)`</pre>

maven的安装:

参考文档:https://maven.apache.org/index.html
注意:这里我们直接采用aliyun的仓库版本,当前为3.3.9,而不是最新的3.5.2。

安装步骤:

1
sudo apt-get install maven

安装检查:

1
2
3
4
5
6
7
root@repo:~# mvn -v
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_151, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-102-generic", arch: "amd64", family: "unix"

安装配置

  • 编辑/etc/maven/settings.xml配置文件:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    <?xml version="1.0" encoding="UTF-8"?>

    <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
    <server>
    <id>repo</id>
    <username>deployment</username>
    <password>******</password>
    </server>
    </servers>
    <profiles>
    <profile>
    <id>repo</id>
    <repositories>
    <repository>
    <id>repo</id>
    <name></name>
    <url>http://repo.krproject.org/nexus/repository/maven-public/</url>
    <layout>default</layout>
    </repository>
    </repositories>
    <pluginRepositories>
    <pluginRepository>
    <id>repo</id>
    <name></name>
    <url>http://repo.krproject.org/nexus/repository/maven-public/</url>
    <layout>default</layout>
    </pluginRepository>
    </pluginRepositories>
    </profile>
    </profiles>

    <activeProfiles>
    <activeProfile>repo</activeProfile>
    </activeProfiles>

    </settings>

注意:为配合maven release插件使用,需要在代码仓库gitlab和jar包仓库nexus中,新建管理员发布用户deployment,用户名密码同maven的settings.xml中保持一致!

node及npm 安装与配置

官方安装参考文档:https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

安装步骤

1
2
3
4
sudo apt-get install -y build-essential

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs`

安装检查

  • 版本检查:
    1
    2
    3
    4
    5
    tiger@repo:~$ node -v
    v8.9.3

    tiger@repo:~$ npm -v
    5.6.0`

安装配置

编辑npm全局配置文件:/usr/etc/npmrc,将仓库设置为自有npm仓库:

1
2
3
registry=http://repo.krproject.org/nexus/repository/npm-public/
//registry=https://r.cnpmjs.org/
strict-ssl=false

docker-ce 安装与配置

官方安装参考文档:https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-using-the-repository

安装步骤

这里我们采用aliyun的镜像:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# step 1: 安装必要的一些系统工具
sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common

# step 2: 安装GPG证书
curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -

# Step 3: 写入软件源信息
sudo add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"

# Step 4: 更新并安装 Docker-CE
sudo apt-get -y update
sudo apt-get -y install docker-ce

# Step 5: 系统重启
sudo shutdown -r now

安装检查

  • 版本检查:

    1
    2
    tiger@repo:~$ docker -v
    Docker version 17.09.0-ce, build afdb6d4
  • 服务状态检查:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    tiger@repo:~$ systemctl status docker
    ● docker.service - Docker Application Container Engine
    Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
    Active: active (running) since 三 2017-12-27 05:50:45 CST; 1 day 9h ago
    Docs: https://docs.docker.com
    Main PID: 1191 (dockerd)
    Tasks: 33
    Memory: 71.8M
    CPU: 3min 3.362s
    CGroup: /system.slice/docker.service
    ├─1191 /usr/bin/dockerd -H fd://
    └─1578 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeo

安装配置

  • 用户加入docker组:

    1
    2
    3
    sudo usermod -aG docker tiger
    sudo usermod -aG docker nexus
    sudo usermod -aG docker gitlab-runner
  • 修改为国内官方镜像,参考文档:https://www.docker-cn.com/registry-mirror

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    sudo mkdir -p /etc/docker

    sudo tee /etc/docker/daemon.json <<-'EOF'
    {
    "registry-mirrors": [
    "https://registry.docker-cn.com"
    ],
    "insecure-registries": [
    "repo.krproject.org:7070",
    "repo.krproject.org:7071"
    ],
    "disable-legacy-registry": true
    }
    EOF

    sudo systemctl daemon-reload
    sudo systemctl restart docker

gitlab-ce及gitlab-runner 安装与配置

gitlab-ce的安装与配置:

采用Omnibus包的方式安装,采用清华源安装:https://mirrors.tuna.tsinghua.edu.cn/help/gitlab-ce/

安装步骤

  • 首先信任 GitLab 的 GPG 公钥:

    1
    curl https://packages.gitlab.com/gpg.key 2&gt; /dev/null | sudo apt-key add - &amp;&gt;/dev/null
  • 再添加清华gitlab-ce源

    1
    2
    3
    vi /etc/apt/sources.list.d/gitlab-ce.list

    deb https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/ubuntu xenial main
  • 安装 gitlab-ce:

    1
    2
    sudo apt-get update
    sudo apt-get install gitlab-ce

安装检查

gitlab的构造还蛮复杂的,会创建很多的用户,启动后也会有很多的进程

安装配置

官方参考文档:https://docs.gitlab.com/omnibus/settings/configuration.html

  • 编辑配置文件,主要修改访问地址和邮箱相关配置:
    1
    2
    3
    vi /etc/gitlab/gitlab.rb

    external_url 'http://repo.krproject.org/gitlab'

gitlab-runner的安装与配置:

忘掉jenkins吧,gitlab-runner采用go语言编写,比起java的jenkins有着更小的内存占用,yml的配置也让持续集成更简单直观~
我们采用包的方式安装,目前尚未找到国内的镜像(之前名叫gitlab-ci-multi-runner,10.0.0版本改名为gitlab-runner后,清华源到目前尚未做同步。。。),采用官方源安装,速度感人,参考文档:https://docs.gitlab.com/runner/install/linux-repository.html

安装步骤

1
2
3
4
5
# For Debian/Ubuntu/Mint
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

# For Debian/Ubuntu/Mint
sudo apt-get install gitlab-runner

安装检查

  • 系统服务状态:
1
2
3
4
5
6
7
8
9
10
tiger@repo:~$ systemctl status gitlab-runner
● gitlab-runner.service - GitLab Runner
Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled; vendor preset: enabled)
Active: active (running) since 二 2017-12-26 21:45:46 CST; 1 day 19h ago
Main PID: 1182 (gitlab-runner)
Tasks: 13
Memory: 27.8M
CPU: 3min 13.888s
CGroup: /system.slice/gitlab-runner.service
└─1182 /usr/bin/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --servic

安装配置

注册本runner,参考文档:https://docs.gitlab.com/runner/register/index.html

nexus 安装与配置

官方参考文档:https://help.sonatype.com/display/NXRM3/Installation
需要注意的是需要oracle jdk, openjdk不被支持,参见:https://help.sonatype.com/display/NXRM3/Java+Runtime+Environment

安装步骤

  • 创建nexus用户:

    1
    sudo useradd -m -d /home/nexus nexus
  • 将下载好的nexus-3.6.1-02-unix.tar.gz/opt/目录下解压:

    1
    2
    cd /opt
    tar -xzvf nexus-3.6.1-02-unix.tar.gz
  • Apache反向代理配置:
    参考文档:https://help.sonatype.com/display/NXRM3/Run+Behind+a+Reverse+Proxy

    1
    2
    3
    4
    5
    vi /opt/nexus/etc/nexus-default.properties

    application-port=8081
    application-host=repo.krproject.org
    nexus-context-path=/nexus
  • 配置为系统服务:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    vi /usr/lib/systemd/system/nexus.service

    [Unit]
    Description=nexus service
    After=network.target
    [Service]
    Type=forking
    LimitNOFILE=65536
    ExecStart=/opt/nexus/bin/nexus start
    ExecStop=/opt/nexus/bin/nexus stop
    User=nexus
    Restart=on-abort
    [Install]
    WantedBy=multi-user.target
  • 启动并激活:

    1
    2
    3
    sudo systemctl daemon-reload
    sudo systemctl enable nexus.service
    sudo systemctl start nexus.service

安装检查

1
2
3
4
5
6
7
8
9
10
11
tiger@repo:~$ systemctl status nexus
● nexus.service - nexus service
Loaded: loaded (/usr/lib/systemd/system/nexus.service; enabled; vendor preset: enabled)
Active: active (running) since 三 2017-12-27 05:50:27 CST; 1 day 10h ago
Process: 1178 ExecStart=/opt/nexus/bin/nexus start (code=exited, status=0/SUCCESS)
Main PID: 1485 (java)
Tasks: 133
Memory: 2.1G
CPU: 13min 19.867s
CGroup: /system.slice/nexus.service
└─1485 /usr/lib/jvm/java-8-oracle/jre/bin/java -server -Dinstall4j.jvmDir=/usr/lib/jvm/java-8-oracle/jre -Dexe4j.moduleNam

安装配置

配置maven仓库

官方参考文档:http://www.sonatype.org/nexus/2017/02/08/using-nexus-3-as-your-repository-part-1-maven-artifacts/

  • 新建用于maven发布的角色nx-maven和用户deployment
  • 按需新建proxyhosted, group的repository

配置npm仓库

官方参考文档:http://www.sonatype.org/nexus/2017/02/14/using-nexus-3-as-your-repository-part-2-npm-packages/

  • 新建用于maven发布的角色nx-npm和用户deployjs
  • 按需新建proxyhosted, group的repository
    这里proxy的地址为:https://registry.npm.taobao.org

配置docker仓库

官方参考文档:http://www.sonatype.org/nexus/2017/02/16/using-nexus-3-as-your-repository-part-3-docker-images/

  • 新建用于maven发布的角色nx-docker和用户deploydocker
  • 按需新建proxyhosted, group的repository
    这里proxy的地址为阿里云的镜像地址,需要自己申请:https://**.mirror.aliyuncs.com
  • 登录方法:
    1
    2
    3
    4
    5
    # login docker public
    docker login repo.krproject.org:7070

    # login docker hosted
    docker login repo.krproject.org:7071

apache2 安装与配置

安装步骤

1
sudo apt-get install apache2

安装检查

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
tiger@repo:~$ systemctl status apache2
● apache2.service - LSB: Apache2 web server
Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
Drop-In: /lib/systemd/system/apache2.service.d
└─apache2-systemd.conf
Active: active (running) since 三 2017-12-27 05:50:34 CST; 1 day 10h ago
Docs: man:systemd-sysv-generator(8)
Process: 24728 ExecReload=/etc/init.d/apache2 reload (code=exited, status=0/SUCCESS)
Process: 1390 ExecStart=/etc/init.d/apache2 start (code=exited, status=0/SUCCESS)
Tasks: 11
Memory: 40.5M
CPU: 21.610s
CGroup: /system.slice/apache2.service
├─ 1546 /usr/sbin/apache2 -k start
├─ 6398 /usr/sbin/apache2 -k start
├─ 6407 /usr/sbin/apache2 -k start
├─ 6543 /usr/sbin/apache2 -k start
├─ 6544 /usr/sbin/apache2 -k start
├─ 6928 /usr/sbin/apache2 -k start
├─25142 /usr/sbin/apache2 -k start
├─25149 /usr/sbin/apache2 -k start
├─25906 /usr/sbin/apache2 -k start
├─25912 /usr/sbin/apache2 -k start
└─25925 /usr/sbin/apache2 -k start

安装配置

配置到后端gitlab及nexus的反向代理:

配置虚拟主机

为了配合npm scope的包使用,参考:https://stackoverflow.com/questions/43063659/scoped-npm-projects-via-proxy-with-nexus-3-2

1
2
3
4
vi /etc/apache2/sites-available/000-default.conf

# 增加这句指令
AllowEncodedSlashes On

配置gitlab访问

1
2
3
4
5
6
7
8
9
# 配置site
vi /etc/apache2/sites-available/gitlab.conf

ProxyPass /gitlab http://127.0.0.1:8181/gitlab
ProxyPassReverse /gitlab http://127.0.0.1:8181/gitlab
ProxyPreserveHost On

# 启用site
a2ensite gitlab.conf

配置nexus访问

1
2
3
4
5
6
7
8
9
# 配置site
vi /etc/apache2/sites-available/nexus.conf

ProxyPass /nexus http://repo.krproject.org:8081/nexus
ProxyPassReverse /nexus http://repo.krproject.org:8081/nexus
ProxyPreserveHost On

# 启用site
a2ensite nexus.conf

ansible 安装与配置

用于自动化运维,官方参考文档:http://docs.ansible.com/ansible/latest/intro_installation.html#latest-releases-via-apt-ubuntu

安装步骤

  • 通过ppa源安装:
    1
    2
    3
    4
    5
    sudo apt-get update
    sudo apt-get install software-properties-common
    sudo apt-add-repository ppa:ansible/ansible
    sudo apt-get update
    sudo apt-get install ansible

安装检查

1
2
3
4
5
6
7
tiger@repo:~$ ansible --version
ansible 2.4.1.0
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/tiger/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609]

日常运维

软件版本不建议过度追新,在软件没有重大bug,影响正常使用的前提下,一般更新频率可在三个月或更长,看自己心情和新版本的功能诱惑了~
这样我们可以将/etc/apt/sources.list.d/里的文件全部下放到监狱jail中:

1
2
mkdir /etc/apt/sources.list.d/jail/
mv /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/jail/

在需要更新时,将对应源文件挪出来即可~