Chemmy's Blog

chengming0916@gmail.com

0%

发表于 Valine：

基于gitea+K3s实现DevOps/CI/CD - 掘金

Excerpt

gitea+k3s实现持续部署，持续集成，滚动发布，devops. 对于需要自建Git服务实现CI/CD及DevOps的，gitea无疑是个非常好的选择。而对于小型运维团队来说，使用K3s能减少大量的

对于需要自建Git服务实现CI/CD及DevOps的，gitea无疑是个非常好的选择。而对于小型运维团队来说，使用K3s能减少大量的维护成本。

gitea和K3s都是非常轻量和易于安装的。

gitea文档：docs.gitea.com/zh-cn/

K3s文档：docs.k3s.io/zh/

安装gitea

参考文档：docs.gitea.com/zh-cn/insta…

1
cd /usr/local mkdir gitea vim docker-compose.yml
1
version: "3" networks: gitea: external: false services: server: image: gitea/gitea:1.20.4 container_name: gitea environment: - USER_UID=1000 - USER_GID=1000 restart: always networks: - gitea volumes: - ./gitea:/data - /etc/timezone:/etc/timezone:ro - /etc/localtime:/etc/localtime:ro ports: - "3000:3000" - "222:22"
1
# 启动gitea docker compose up -d

启用action

1
vim gitea/gitea/conf/app.ini ------ #末尾添加 [actions] ENABLED=true ------ ## 重启gitea docker restart gitea

访问http://your-ip:3000 ,配置管理员账号密码和其他相应内容，点击安装即可。安装完成后如下：

image-20230915150958396.png

建议将服务暴漏为https域名，更能节省时间。以 gitea.mydomain.com 为例。

安装act runner

参考文档：**docs.gitea.com/zh-cn/usage…**

获取token

到管理后台-actions-Runner中，点击创建runner，复制token，如下

image-20230915151107307

1
# 例如如下token AaJkPUFeQs37hpGm6SHUIqHx3Tw8kmm0xX5gJv6f

注册runner

1
mkdir runner cd runner vim docker-compose.yml
1
version: "3.8" services: runner: image: gitea/act_runner:nightly container_name: gitea_runner restart: always environment: CONFIG_FILE: /config.yaml GITEA_INSTANCE_URL: "https://your-ip:3000" GITEA_RUNNER_REGISTRATION_TOKEN: "AaJkPUFeQs37hpGm6SHUIqHx3Tw8kmm0xX5gJv6f" GITEA_RUNNER_NAME: "my-runner" GITEA_RUNNER_LABELS: "${RUNNER_LABELS}"#需要特殊环境部署的，可以修改该标签，或在web控制台修改 volumes: - ./config.yaml:/config.yaml - ./data:/data - /var/run/docker.sock:/var/run/docker.sock - /etc/timezone:/etc/timezone:ro - /etc/localtime:/etc/localtime:ro ports: - "8088:8088"

为加快构建速度，一般来说我们需要允许缓存：创建config.yaml，允许缓存，请注意修改your-ip为runner所在服务器的内网ip。

1
vim config.yaml
1
# Example configuration file, it's safe to copy this as the default config file without any modification. # You don't have to copy this file to your instance, # just run `./act_runner generate-config > config.yaml` to generate a config file. log: # The level of logging, can be trace, debug, info, warn, error, fatal level: info runner: # Where to store the registration result. file: .runner # Execute how many tasks concurrently at the same time. capacity: 1 # Extra environment variables to run jobs. envs: A_TEST_ENV_NAME_1: a_test_env_value_1 A_TEST_ENV_NAME_2: a_test_env_value_2 RUNNER_TOOL_CACHE: /toolcache # Extra environment variables to run jobs from a file. # It will be ignored if it's empty or the file doesn't exist. env_file: .env # The timeout for a job to be finished. # Please note that the Gitea instance also has a timeout (3h by default) for the job. # So the job could be stopped by the Gitea instance if it's timeout is shorter than this. timeout: 3h # Whether skip verifying the TLS certificate of the Gitea instance. insecure: false # The timeout for fetching the job from the Gitea instance. fetch_timeout: 5s # The interval for fetching the job from the Gitea instance. fetch_interval: 2s # The labels of a runner are used to determine which jobs the runner can run, and how to run them. # Like: ["macos-arm64:host", "ubuntu-latest:docker://node:16-bullseye", "ubuntu-22.04:docker://node:16-bullseye"] # If it's empty when registering, it will ask for inputting labels. # If it's empty when execute `deamon`, will use labels in `.runner` file. labels: [] cache: # Enable cache server to use actions/cache. enabled: true # The directory to store the cache data. # If it's empty, the cache data will be stored in $HOME/.cache/actcache. dir: "" # The host of the cache server. # It's not for the address to listen, but the address to connect from job containers. # So 0.0.0.0 is a bad choice, leave it empty to detect automatically. host: "your-ip" # The port of the cache server. # 0 means to use a random available port. port: 8088 # The external cache server URL. Valid only when enable is true. # If it's specified, act_runner will use this URL as the ACTIONS_CACHE_URL rather than start a server by itself. # The URL should generally end with "/". external_server: "" container: # Specifies the network to which the container will connect. # Could be host, bridge or the name of a custom network. # If it's empty, act_runner will create a network automatically. network: "" # Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker). privileged: false # And other options to be used when the container is started (eg, --add-host=my.gitea.url:host-gateway). options: # The parent directory of a job's working directory. # If it's empty, /workspace will be used. workdir_parent: # Volumes (including bind mounts) can be mounted to containers. Glob syntax is supported, see https://github.com/gobwas/glob # You can specify multiple volumes. If the sequence is empty, no volumes can be mounted. # For example, if you only allow containers to mount the `data` volume and all the json files in `/src`, you should change the config to: # valid_volumes: # - data # - /src/*.json # If you want to allow any volume, please use the following configuration: # valid_volumes: # - '**' valid_volumes: [] # overrides the docker client host with the specified one. # If it's empty, act_runner will find an available docker host automatically. # If it's "-", act_runner will find an available docker host automatically, but the docker host won't be mounted to the job containers and service containers. # If it's not empty or "-", the specified docker host will be used. An error will be returned if it doesn't work. docker_host: "" # Pull docker image(s) even if already present force_pull: false host: # The parent directory of a job's working directory. # If it's empty, $HOME/.cache/act/ will be used. workdir_parent:
1
<span data-line-num="1">docker compose up -d</span>

重新刷新页面，可看到新的runner

runner

安装k3s

参考文档docs.k3s.io/zh/quick-st…

在需要部署应用的服务器(例如192.168.0.100)上安装K3s,安装命令

1
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

如果暴漏为https域名，可以省略以下步骤

私有仓库镜像配置

如果是配置了https域名的可以忽略本步

docker允许访问gitea仓库

1
vim /etc/docker/daemon.json
1
{ "log-driver": "json-file", "log-opts": {"max-size": "100m", "max-file": "100"}, "insecure-registries": [ "your-ip:3000" ] }

k3s允许访问gitea

1
vim /etc/rancher/k3s/registries.yaml
1
mirrors: gitea.mydomain.com: endpoint: - "http://your-ip:3000" configs: "your-ip:3000": auth: username: your-gitea-name # this is the registry username password: your-gitea-password # this is the registry password
1
2
<span data-line-num="1">systemctl restart docker</span>
<span data-line-num="2">systemctl restart k3s</span>

持续集成

以下步骤将会以ruoyi-vue-plus为例。

启用action

点击右上角+-迁移外部仓库将ruoyi-vue-plus仓库迁移到本地gitea中。待迁移完成后，点击项目设置-仓库-启用actions,点击更新仓库设置按钮，开启项目action.

image-20230915152636599

image-20230915152859461

获取gitea部署token

在个人信息及配置-应用中生成一个token，注意权限选择repository和user都是read，其他不选。

请注意令牌只会显示一次，复制保管好。

image-20230915150958396.png

添加CI文件

以最新分支5.X为例，切换到5.X分支，在项目中添加.gitea/workflows/main.yml文件

1
name: Ruoyi Vue Plus run-name: CI/CD runner on: push: # Sequence of patterns matched against refs/heads branches: - 5.X jobs: ci-and-cd: runs-on: ubuntu-latest container: image: catthehacker/ubuntu:act-latest env: # cache bug https://gitea.com/gitea/act_runner/issues/70 RUNNER_TOOL_CACHE: /toolcache REGISTRY: gitea.mydomain.com APP_NAME: ruoyi-admin steps: - name: checkout from git uses: actions/checkout@v3 - name: Set up Java uses: graalvm/setup-graalvm@v1 with: java-version: '17.0.7' distribution: 'graalvm' cache: 'maven' - name: Build with Maven run: mvn package -DskipTests -Pprod - name: Extract metadata (tags, labels) for Docker id: meta uses: docker/metadata-action@v4 with: images: ${{ env.REGISTRY }}/devops/${{ env.APP_NAME }} - name: Login to Docker Hub uses: docker/login-action@v2 with: registry: ${{ env.REGISTRY }} username: ${{ secrets.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_PASSWORD }} - name: Build and push uses: docker/build-push-action@master with: context: ./ruoyi-admin push: ${{ github.event_name != 'pull_request' }} tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }}
1
- name: Build and push uses: docker/build-push-action@master with: context: ./ruoyi-admin push: ${{ github.event_name != 'pull_request' }} tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} - name: k8s Check and Apply New Deployment uses: actions-hub/kubectl@master env: KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }} with: args: apply -f http://${{ secrets.DEVOPS_GITEA_TOKEN }}@${{ env.REGISTRY }}/devops/ruoyi-vue-plus/raw/branch/master/.deploy/deployment.yml - name: k8s Update Deployment uses: actions-hub/kubectl@master env: KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }} with: args: rollout restart deployment ${{ env.APP_NAME }}-${{ gitea.ref_name }}

设置action密钥

可以在个人中心，action中设置全局的，也可以在仓库中设置对应仓库的。

添加docker hub配置

在actions中，添加密钥，名称为DOCKERHUB_USERNAME和DOCKERHUB_PASSWORD，分别对应你的gitea账号和密码。添加DEVOPS_GITEA_TOKEN，即上一步得到的token。

添加kube-config配置

获取kube-config，默认放在/etc/rancher/k3s/k3s.yaml，复制内容，并修改ip为192.168.0.100，

1
cp /etc/rancher/k3s/k3s.yaml myk3s.yaml vim myk3s.yaml # 修改 server: https://127.0.0.1:6443 为 server: https://192.168.0.100:6443 # 获取KUBE_CONFIG cat myk3s.yaml | base64

添加acitons密钥KUBE_CONFIG,值为上面输出的Base64

image-20230921105647611

创建k8s secret

1
kubectl create secret docker-registry dockerhub-id --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>

创建deployment

在.deploy目录下创建文件deployment.yml文件。

1
--- ## k8s默认是访问不了外部服务的，需要代理,以mysql和redis为例，代理myservice名称到192.168.0.100，配置文件中对应的ip修改为myservice apiVersion: v1 kind: Endpoints metadata: name: myservice subsets: - addresses: - ip: 192.168.0.100 ports: - port: 6379 protocol: TCP name: redis - port: 3306 protocol: TCP name: mysql --- apiVersion: v1 kind: Service metadata: name: myservice spec: ports: - port: 6379 targetPort: 6379 protocol: TCP name: redis - port: 3306 targetPort: 3306 protocol: TCP name: mysql --- apiVersion: apps/v1 kind: Deployment metadata: name: ruoyi-admin spec: replicas: 2 revisionHistoryLimit: 3 selector: matchLabels: app: ruoyi-admin template: metadata: labels: app: ruoyi-admin spec: containers: - image: gitea.mydomain.com/devops/ruoyi-admin:5.X name: ruoyi-admin imagePullPolicy: Always readinessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 1 successThreshold: 5 ports: - containerPort: 8080 imagePullSecrets: - name: dockerhub-id --- apiVersion: v1 kind: Service metadata: name: ruoyi-admin spec: type: NodePort externalTrafficPolicy: Local ports: - name: http-8080 protocol: TCP port: 8080 nodePort: 30088 selector: app: ruoyi-admin

修改application-prod.yml

由于k8s默认是访问不了外部服务的，需要代理,以mysql和redis为例，代理myservice名称到192.168.0.100，配置文件中对应的ip修改为myservice。

修改mysql的ip为myservice,如下：

1
url: jdbc:mysql://myservice:3306/ruoyi-vue?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true

redis同上

推送并部署

提交代码，查看actions是否成功。运行成功后，访问http://192.168.0.100:30088 即可访问成功。

image-20230915160639074

Q&A

Q:actions中的Set up job慢，github访问超时

A:将github action仓库导入到本地gitea，修改ci文件为本地url，例如checkout，将 github.com/actions/che… 同步到本地，修改ci为

1
steps: - name: checkout from git uses: https://gitea.mydomain.com/devops/checkout@v3

其他同理。

Q：我需要部署的服务器是私有的，需要VPN才能访问，怎么办？

A：如果该私有服务器能访问到gitea，只需要在该私有服务器上部署一个runner，修改该runner的GITEA_RUNNER_LABELS标签，例如GITEA_RUNNER_LABELS: test-runner然后修改ci文件，指定cd步骤为test-runner即可。

ci:

1
name: Ruoyi Vue Plus run-name: CI/CD runner on: push: # Sequence of patterns matched against refs/heads branches: - 5.X jobs: ci: runs-on: ubuntu-latest container: image: catthehacker/ubuntu:act-latest env: # cache bug https://gitea.com/gitea/act_runner/issues/70 RUNNER_TOOL_CACHE: /toolcache REGISTRY: gitea.mydomain.com APP_NAME: ruoyi-admin steps: - name: checkout from git uses: actions/checkout@v3 - 同上，将Deploy to k8s步骤移到后面 cd: needs: ci runs-on: test-runner # 指定runner container: image: catthehacker/ubuntu:act-latest env: # cache bug https://gitea.com/gitea/act_runner/issues/70 RUNNER_TOOL_CACHE: /toolcache REGISTRY: gitea.mydomain.com APP_NAME: ruoyi-admin steps: - name: k8s Check and Apply New Deployment uses: actions-hub/kubectl@master env: KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }} with: args: apply -f http://${{ secrets.DEVOPS_GITEA_TOKEN }}@${{ env.REGISTRY }}/devops/ruoyi-vue-plus/raw/branch/master/.deploy/deployment.yml - name: k8s Update Deployment uses: actions-hub/kubectl@master env: KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }} with: args: rollout restart deployment ${{ env.APP_NAME }}-${{ gitea.ref_name }}