标签：INFO maven Pipeline git kubesphere devops sample 安装
简单研究下kubesphere 中devops 的使用。
1. 安装
实际上就是修改安装过程中的 cluster-configuration.yaml 文件的devops.enable 设为true 即可, kebesphere 会自己下载jenkins，我们通过kubesphere 操作流水线最终也会到jenkins 中操作对应的流水线。
参考: https://kubesphere.io/zh/docs/pluggable-components/devops/
安装过程中出现的问题: nfs服务器磁盘空间、k8snode01 磁盘空间、k8snode01 内存问题等等。建议在虚拟机初始化就设置的大一点。
1. 安装成功后查看相关pods、service 如下：
[root@k8smaster01 kubesphere]# kubectl get pods -n kubesphere-devops-system NAME READY STATUS RESTARTS AGE devops-27439710-f4q5f 0/1 Completed 0 8m12s devops-apiserver-7c6774fff5-nmj84 1/1 Running 0 21m devops-controller-98975d478-4xhxn 1/1 Running 0 21m devops-jenkins-64464f495f-4z8br 1/1 Running 1 21m s2ioperator-0 1/1 Running 0 21m [root@k8smaster01 kubesphere]# kubectl get svc -n kubesphere-devops-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE devops-apiserver ClusterIP 10.1.248.197 <none> 9090/TCP 21m devops-jenkins NodePort 10.1.45.197 <none> 80:30180/TCP 21m devops-jenkins-agent ClusterIP 10.1.113.105 <none> 50000/TCP 21m s2ioperator-metrics-service ClusterIP 10.1.161.145 <none> 8080/TCP 21m s2ioperator-trigger-service ClusterIP 10.1.119.55 <none> 8081/TCP 21m webhook-server-service ClusterIP 10.1.46.76 <none> 443/TCP 21m
2. 然后登录kubesphere 控制台看到如下:
2. 登录jenkins
上面可以看到service， 30180 端口可以访问jenkins，在找密码的时候发现其没有密码文件。 最后通过官网发现： 是用kubesphere 的用户体系进行登录。
参考: https://v2-1.docs.kubesphere.io/docs/zh-CN/devops/jenkins-setting/
接下来可以简单的看一下jenkins 控制台界面安装的一些插件等信息。
2. 使用
接下来就是使用kubesphere 的devops 发布一个简单的springboot 项目。 这里使用github作为代码仓库，阿里镜像仓库作为镜像仓库。
官网对其运行过程解释如下：
首先，Jenkins Master 创建一个 Pod 来运行流水线。Kubernetes 创建 Pod 作为 Jenkins Master 的 Agent，该 Pod 会在流水线完成之后销毁。主要流程包括克隆代码、构建和推送镜像以及部署工作负载。
官方提供的两个java 项目：
https://github.com/kubesphere/devops-java-sample
https://github.com/kubesphere/devops-maven-sample 本次测试使用这个项目进行测试
搭建过程参考：
https://kubesphere.io/docs/devops-user-guide/how-to-use/create-a-pipeline-using-jenkinsfile/
1. fork项目到自己的github 仓库
如果网络波动太大的话可以使用国内的gitee，作为代码仓库进行测试。我下面使用gitee。
2. 到kubesphere 进行操作
这里用户都用admin 进行操作，如果想测试kubesphere 的用户体系可以自己创建多个用户进行操作。
1. 到访问控制-》企业空间创建一个企业空间 dev， 用于独立的测试。这个可以理解为大的一个资源隔离的空间。
一个空间可以有多个普通项目和devops 项目。
普通项目可以用来管理独立的kubernetes 资源，包括pod、svc 等，实际kubesphere 后台的操作就是新建了一个对应的namespace用于隔离相应资源，namespace 的名称就是项目名称；
devops 项目可以包含多个jenkins流水线， 用于devops 操作，kubesphere 后台也创建了一个对应的namespace，namespace名称为devops 项目名称加5位随机字母。
2. dev 企业空间创建一个项目，名称为 demo。 查看namespace如下
[root@k8smaster01 ~]# kubectl get ns|grep demo demo59wjg Active 26s
3. 在demo 项目下面建立自己的凭证：
1》gitee 的凭证
2》镜像仓库的凭证
3》kubeconfig 用于kubernetes 使用
4. 本地复制项目Jenkinsfile-online，命名为Jenkinsfile。 原来文件用于备份，新文件用于jenkins 流水线脚本。修改Jenkinsfile 里面内容的凭证信息，最终如下：
1 pipeline {
2 agent {
3 node {
4 label 'maven'
5 }
6 }
7
8 parameters {
9 string(name:'TAG_NAME',defaultValue: '',description:'')
10 }
11
12 environment {
13 DOCKER_CREDENTIAL_ID = 'ali-registery'
14 GITHUB_CREDENTIAL_ID = 'gitee-secret'
15 KUBECONFIG_CREDENTIAL_ID = 'demo-kubeconfig'
16 REGISTRY = 'registry.cn-hangzhou.aliyuncs.com'
17 DOCKERHUB_NAMESPACE = 'qlq_repository'
18 GITHUB_ACCOUNT = 'qiao-zhi'
19 APP_NAME = 'devops-maven-sample'
20 }
21
22 stages {
23 stage ('checkout scm') {
24 steps {
25 checkout(scm)
26 }
27 }
28
29 stage ('unit test') {
30 steps {
31 container ('maven') {
32 sh 'mvn clean test'
33 }
34 }
35 }
36
37 stage ('build & push') {
38 steps {
39 container ('maven') {
40 sh 'mvn clean package -DskipTests'
41 sh 'docker build -f Dockerfile-online -t $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER .'
42 withCredentials([usernamePassword(passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,credentialsId : "$DOCKER_CREDENTIAL_ID" ,)]) {
43 sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
44 sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER'
45 }
46 }
47 }
48 }
49
50 stage('push latest'){
51 when{
52 branch 'master'
53 }
54 steps{
55 container ('maven') {
56 sh 'docker tag $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:latest '
57 sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:latest '
58 }
59 }
60 }
61
62 stage('deploy to dev') {
63 when{
64 branch 'master'
65 }
66 steps {
67 input(id: 'deploy-to-dev', message: 'deploy to dev?')
68 container ('maven') {
69 withCredentials([
70 kubeconfigFile(
71 credentialsId: env.KUBECONFIG_CREDENTIAL_ID,
72 variable: 'KUBECONFIG')
73 ]) {
74 sh 'envsubst < deploy/dev-all-in-one/devops-sample.yaml | kubectl apply -f -'
75 }
76 }
77 }
78 }
79 stage('push with tag'){
80 when{
81 expression{
82 return params.TAG_NAME =~ /v.*/
83 }
84 }
85 steps {
86 container ('maven') {
87 input(id: 'release-image-with-tag', message: 'release image with tag?')
88 withCredentials([usernamePassword(credentialsId: "$GITHUB_CREDENTIAL_ID", passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
89 sh 'git config --global user.email "kubesphere@yunify.com" '
90 sh 'git config --global user.name "kubesphere" '
91 sh 'git tag -a $TAG_NAME -m "$TAG_NAME" '
92 sh 'git push http://$GIT_USERNAME:$GIT_PASSWORD@gitee.com/$GITHUB_ACCOUNT/$APP_NAME.git --tags --ipv4'
93 }
94 sh 'docker tag $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:$TAG_NAME '
95 sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:$TAG_NAME '
96 }
97 }
98 }
99 stage('deploy to production') {
100 when{
101 expression{
102 return params.TAG_NAME =~ /v.*/
103 }
104 }
105 steps {
106 input(id: 'deploy-to-production', message: 'deploy to production?')
107 container ('maven') {
108 withCredentials([
109 kubeconfigFile(
110 credentialsId: env.KUBECONFIG_CREDENTIAL_ID,
111 variable: 'KUBECONFIG')
112 ]) {
113 sh 'envsubst < deploy/prod-all-in-one/devops-sample.yaml | kubectl apply -f -'
114 }
115 }
116 }
117 }
118 }
119 }
和原来官网项目文件相比修改的包括environment 中的凭证的id 和 自己的相关信息； 92 行推送tag 时候的相关的github 信息替换为gitee 服务器信息和用$APP_NAME 获取仓库名称。
简单理解上面相关凭证用于拉代码、推镜像、部署到kubernetes。 REGISTRY 是docker 镜像仓库地址；DOCKERHUB_NAMESPACE 是镜像仓库命名空间；GITHUB_ACCOUNT 用于下面拼接github 代码仓库路径； APP_NAME 是打包名称和镜像名称。
5. 到阿里镜像仓库创建新的镜像仓库，名称为 devops-maven-sample(这里创建为公共仓库，不然拉取镜像会失败， 想解决参考： https://developer.aliyun.com/ask/11155?spm=a2c6h.13706215.ask-content.1.3c732a3aUTxxTA)
6. 到kubesphere demo 项目下新建流水线， 名称为 devops-maven-sample, 然后选择代码仓库，后面默认即可(Jenkinsfile 名称也匹配)：
7. 点到流水线中，然后点击扫描项目，获取git 上面的Jenkinsfile 文件和代码分支，最终的扫描日志如下：
Started by user admin
[Sun Mar 06 03:47:01 UTC 2022] Starting branch indexing...
> git --version # timeout=10
> git --version # 'git version 2.11.0'
using GIT_ASKPASS to set credentials
> git ls-remote --symref -- https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
> git rev-parse --is-inside-work-tree # timeout=10
Setting origin to https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
Fetching & pruning origin...
Listing remote references...
> git config --get remote.origin.url # timeout=10
> git --version # timeout=10
> git --version # 'git version 2.11.0'
using GIT_ASKPASS to set credentials
> git ls-remote -h -- https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
Fetching upstream changes from origin
> git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --tags --progress --prune -- origin +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking branches...
Checking branch master
‘Jenkinsfile’ found
Met criteria
No changes detected: master (still at 4b2c785bec6d7c4d2899ec78682b431455529c16)
Processed 1 branches
[Sun Mar 06 03:47:10 UTC 2022] Finished branch indexing. Indexing took 9.2 sec
Finished: SUCCESS
8. 接下来在kubernetes 环境中创建两个namespace， 不创建namespace 在创建kubernetes 资源的时候会报错
创建dev 和 prod 环境对应的namespace
kubectl create ns kubesphere-sample-dev
kubectl create ns kubesphere-sample-prod
9. 然后选择master分支后，点击运行，输入参数v1(该参数用于jenkins 流水线脚本进行判断是否需要推送代码tag和镜像tag)运行流水线，等待其结果：(当我们不输入tag的时候相当于只是部署到开发环境，不打tag等操作)
中间build 过程中，我们全部选择处理
10. 完成后输出如下：
查看日志如下：
Started by user admin
> git rev-parse --is-inside-work-tree # timeout=10
Setting origin to https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
Fetching origin...
Fetching upstream changes from origin
> git --version # timeout=10
> git --version # 'git version 2.11.0'
> git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --tags --progress -- origin +refs/heads/*:refs/remotes/origin/* # timeout=10
Seen branch in repository origin/master
Seen 1 remote branch
Obtained Jenkinsfile from 67401e9a817a371f9da32d5606d591e7b3e0598f
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
‘maven-2whbc’ is offline
Agent maven-2whbc is provisioned from template maven
---
apiVersion: "v1"
kind: "Pod"
metadata:
annotations: {}
labels:
jenkins: "slave"
jenkins/label-digest: "f02c587acd12db3d7b4a28edb5c2eae5f526ce28"
jenkins/label: "maven"
name: "maven-2whbc"
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: "node-role.kubernetes.io/worker"
operator: "In"
values:
- "ci"
weight: 1
containers:
- args:
- "********"
- "maven-2whbc"
command:
- "jenkins-slave"
env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_TUNNEL"
value: "devops-jenkins-agent.kubesphere-devops-system:50000"
- name: "JENKINS_AGENT_NAME"
value: "maven-2whbc"
- name: "JENKINS_NAME"
value: "maven-2whbc"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "http://devops-jenkins.kubesphere-devops-system:80/"
image: "jenkins/jnlp-slave:3.27-1"
imagePullPolicy: "IfNotPresent"
name: "jnlp"
resources:
limits:
memory: "1536Mi"
cpu: "500m"
requests:
memory: "400Mi"
cpu: "50m"
tty: false
volumeMounts:
- mountPath: "/root/.sonar/cache"
name: "volume-2"
readOnly: false
- mountPath: "/root/.m2"
name: "volume-1"
readOnly: false
- mountPath: "/var/run/docker.sock"
name: "volume-0"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- command:
- "cat"
image: "kubesphere/builder-maven:v3.2.0"
imagePullPolicy: "IfNotPresent"
name: "maven"
resources:
limits:
ephemeral-storage: "10Gi"
memory: "8192Mi"
cpu: "4000m"
requests:
ephemeral-storage: "1Gi"
memory: "100Mi"
cpu: "100m"
tty: true
volumeMounts:
- mountPath: "/opt/apache-maven-3.5.3/conf/settings.xml"
name: "config-volume"
subPath: "settings.xml"
- mountPath: "/root/.sonar/cache"
name: "volume-2"
readOnly: false
- mountPath: "/root/.m2"
name: "volume-1"
readOnly: false
- mountPath: "/var/run/docker.sock"
name: "volume-0"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
workingDir: "/home/jenkins/agent"
nodeSelector: {}
restartPolicy: "Never"
securityContext:
fsGroup: 1000
tolerations:
- effect: "NoSchedule"
key: "node.kubernetes.io/ci"
operator: "Exists"
- effect: "PreferNoSchedule"
key: "node.kubernetes.io/ci"
operator: "Exists"
volumes:
- hostPath:
path: "/var/run/docker.sock"
name: "volume-0"
- hostPath:
path: "/var/data/jenkins_sonar_cache"
name: "volume-2"
- hostPath:
path: "/var/data/jenkins_maven_cache"
name: "volume-1"
- emptyDir:
medium: ""
name: "workspace-volume"
- configMap:
items:
- key: "MavenSetting"
path: "settings.xml"
name: "ks-devops-agent"
name: "config-volume"
Running on maven-2whbc in /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential gitee-secret
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
Cloning repository https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git init /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master # timeout=10
Fetching upstream changes from https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git --version # timeout=10
> git --version # 'git version 2.11.0'
using GIT_ASKPASS to set credentials
> git fetch --no-tags --progress -- https://gitee.com/Qiao-Zhi/devops-maven-sample.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision 67401e9a817a371f9da32d5606d591e7b3e0598f (master)
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f 67401e9a817a371f9da32d5606d591e7b3e0598f # timeout=10
Commit message: "v1"
> git rev-list --no-walk 67401e9a817a371f9da32d5606d591e7b3e0598f # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (checkout scm)
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential gitee-secret
Fetching changes from the remote Git repository
Fetching without tags
> git rev-parse --is-inside-work-tree # timeout=10
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
Fetching upstream changes from https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git --version # timeout=10
> git --version # 'git version 2.11.0'
using GIT_ASKPASS to set credentials
> git fetch --no-tags --progress -- https://gitee.com/Qiao-Zhi/devops-maven-sample.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking out Revision 67401e9a817a371f9da32d5606d591e7b3e0598f (master)
Commit message: "v1"
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (unit test)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
> git config core.sparsecheckout # timeout=10
> git checkout -f 67401e9a817a371f9da32d5606d591e7b3e0598f # timeout=10
+ mvn clean test
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< io.kubesphere.devops:devops-sample >-----------------
[INFO] Building devops-sample :: HelloWorld Demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ devops-sample ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (agent-for-ut) @ devops-sample ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ devops-sample ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.kubesphere.devops.HelloWorldControllerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.527 s - in io.kubesphere.devops.HelloWorldControllerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.872 s
[INFO] Finished at: 2022-03-06T04:25:53Z
[INFO] ------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (build & push)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ mvn clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< io.kubesphere.devops:devops-sample >-----------------
[INFO] Building devops-sample :: HelloWorld Demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ devops-sample ---
[INFO] Deleting /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (agent-for-ut) @ devops-sample ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ devops-sample ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ devops-sample ---
[INFO] Building jar: /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/devops-sample-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- spring-boot-maven-plugin:2.1.11.RELEASE:repackage (repackage) @ devops-sample ---
[INFO] Replacing main artifact with repackaged archive
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.580 s
[INFO] Finished at: 2022-03-06T04:26:16Z
[INFO] ------------------------------------------------------------------------
[Pipeline] sh
+ docker build -f Dockerfile-online -t registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3 .
Sending build context to Docker daemon 17.52MB
Step 1/4 : FROM java:openjdk-8-jre-alpine
---> fdc893b19a14
Step 2/4 : WORKDIR /home
---> Using cache
---> 1e0dc3b20420
Step 3/4 : COPY target/*.jar /home
---> fd0896260f0e
Step 4/4 : ENTRYPOINT java -jar *.jar
---> Running in 27d46c26a862
Removing intermediate container 27d46c26a862
---> 8867ca53b5ca
Successfully built 8867ca53b5ca
Successfully tagged registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3
[Pipeline] withCredentials
Masking supported pattern matches of $DOCKER_USERNAME or $DOCKER_PASSWORD
[Pipeline] {
[Pipeline] sh
+ echo ****
+ docker login registry.cn-hangzhou.aliyuncs.com -u **** --password-stdin
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[Pipeline] sh
+ docker push registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample]
0e1bd0a16495: Preparing
20dd87a4c2ab: Preparing
78075328e0da: Preparing
9f8566ee5135: Preparing
20dd87a4c2ab: Layer already exists
78075328e0da: Layer already exists
9f8566ee5135: Layer already exists
0e1bd0a16495: Pushed
SNAPSHOT-master-3: digest: sha256:91d99fde2a9b9a4f15ead003c8446fb3532aeb7c3173ebaad09d69237eb22756 size: 1159
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (push latest)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ docker tag registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3 registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:latest
[Pipeline] sh
+ docker push registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:latest
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample]
0e1bd0a16495: Preparing
20dd87a4c2ab: Preparing
78075328e0da: Preparing
9f8566ee5135: Preparing
78075328e0da: Layer already exists
20dd87a4c2ab: Layer already exists
9f8566ee5135: Layer already exists
0e1bd0a16495: Layer already exists
latest: digest: sha256:91d99fde2a9b9a4f15ead003c8446fb3532aeb7c3173ebaad09d69237eb22756 size: 1159
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy to dev)
[Pipeline] input
deploy to dev?
Proceed or Abort
Approved by admin
[Pipeline] container
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $KUBECONFIG
[Pipeline] {
[Pipeline] sh
+ envsubst
+ kubectl apply -f -
deployment.apps/ks-sample-dev configured
service/ks-sample-dev unchanged
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (push with tag)
[Pipeline] container
[Pipeline] {
[Pipeline] input
release image with tag?
Proceed or Abort
Approved by admin
[Pipeline] withCredentials
Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[Pipeline] {
[Pipeline] sh
+ git config --global user.email kubesphere@yunify.com
[Pipeline] sh
+ git config --global user.name kubesphere
[Pipeline] sh
+ git tag -a v1 -m v1
[Pipeline] sh
+ git push http://****:****@gitee.com/****/devops-maven-sample.git --tags --ipv4
remote: Powered by [01;33mGITEE.COM [0m[[01;35mGNK-6.3[0m][0m
To http://gitee.com/****/devops-maven-sample.git
* [new tag] v1 -> v1
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] sh
+ docker tag registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3 registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:v1
[Pipeline] sh
+ docker push registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:v1
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample]
0e1bd0a16495: Preparing
20dd87a4c2ab: Preparing
78075328e0da: Preparing
9f8566ee5135: Preparing
9f8566ee5135: Layer already exists
0e1bd0a16495: Layer already exists
20dd87a4c2ab: Layer already exists
78075328e0da: Layer already exists
v1: digest: sha256:91d99fde2a9b9a4f15ead003c8446fb3532aeb7c3173ebaad09d69237eb22756 size: 1159
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy to production)
[Pipeline] input
deploy to production?
Proceed or Abort
Approved by admin
[Pipeline] container
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $KUBECONFIG
[Pipeline] {
[Pipeline] sh
+ envsubst
+ kubectl apply -f -
deployment.apps/ks-sample created
service/ks-sample created
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
View Code
1》jenkins 查看日志如下
日志如下：
Started by user admin
> git rev-parse --is-inside-work-tree # timeout=10
Setting origin to https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
Fetching origin...
Fetching upstream changes from origin
> git --version # timeout=10
> git --version # 'git version 2.11.0'
> git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --tags --progress -- origin +refs/heads/*:refs/remotes/origin/* # timeout=10
Seen branch in repository origin/master
Seen 1 remote branch
Obtained Jenkinsfile from 67401e9a817a371f9da32d5606d591e7b3e0598f
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
‘maven-2whbc’ is offline
Agent maven-2whbc is provisioned from template maven
---
apiVersion: "v1"
kind: "Pod"
metadata:
annotations: {}
labels:
jenkins: "slave"
jenkins/label-digest: "f02c587acd12db3d7b4a28edb5c2eae5f526ce28"
jenkins/label: "maven"
name: "maven-2whbc"
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: "node-role.kubernetes.io/worker"
operator: "In"
values:
- "ci"
weight: 1
containers:
- args:
- "********"
- "maven-2whbc"
command:
- "jenkins-slave"
env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_TUNNEL"
value: "devops-jenkins-agent.kubesphere-devops-system:50000"
- name: "JENKINS_AGENT_NAME"
value: "maven-2whbc"
- name: "JENKINS_NAME"
value: "maven-2whbc"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "http://devops-jenkins.kubesphere-devops-system:80/"
image: "jenkins/jnlp-slave:3.27-1"
imagePullPolicy: "IfNotPresent"
name: "jnlp"
resources:
limits:
memory: "1536Mi"
cpu: "500m"
requests:
memory: "400Mi"
cpu: "50m"
tty: false
volumeMounts:
- mountPath: "/root/.sonar/cache"
name: "volume-2"
readOnly: false
- mountPath: "/root/.m2"
name: "volume-1"
readOnly: false
- mountPath: "/var/run/docker.sock"
name: "volume-0"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- command:
- "cat"
image: "kubesphere/builder-maven:v3.2.0"
imagePullPolicy: "IfNotPresent"
name: "maven"
resources:
limits:
ephemeral-storage: "10Gi"
memory: "8192Mi"
cpu: "4000m"
requests:
ephemeral-storage: "1Gi"
memory: "100Mi"
cpu: "100m"
tty: true
volumeMounts:
- mountPath: "/opt/apache-maven-3.5.3/conf/settings.xml"
name: "config-volume"
subPath: "settings.xml"
- mountPath: "/root/.sonar/cache"
name: "volume-2"
readOnly: false
- mountPath: "/root/.m2"
name: "volume-1"
readOnly: false
- mountPath: "/var/run/docker.sock"
name: "volume-0"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
workingDir: "/home/jenkins/agent"
nodeSelector: {}
restartPolicy: "Never"
securityContext:
fsGroup: 1000
tolerations:
- effect: "NoSchedule"
key: "node.kubernetes.io/ci"
operator: "Exists"
- effect: "PreferNoSchedule"
key: "node.kubernetes.io/ci"
operator: "Exists"
volumes:
- hostPath:
path: "/var/run/docker.sock"
name: "volume-0"
- hostPath:
path: "/var/data/jenkins_sonar_cache"
name: "volume-2"
- hostPath:
path: "/var/data/jenkins_maven_cache"
name: "volume-1"
- emptyDir:
medium: ""
name: "workspace-volume"
- configMap:
items:
- key: "MavenSetting"
path: "settings.xml"
name: "ks-devops-agent"
name: "config-volume"
Running on maven-2whbc in /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential gitee-secret
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
Cloning repository https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git init /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master # timeout=10
Fetching upstream changes from https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git --version # timeout=10
> git --version # 'git version 2.11.0'
using GIT_ASKPASS to set credentials
> git fetch --no-tags --progress -- https://gitee.com/Qiao-Zhi/devops-maven-sample.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision 67401e9a817a371f9da32d5606d591e7b3e0598f (master)
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f 67401e9a817a371f9da32d5606d591e7b3e0598f # timeout=10
Commit message: "v1"
> git rev-list --no-walk 67401e9a817a371f9da32d5606d591e7b3e0598f # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (checkout scm)
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential gitee-secret
Fetching changes from the remote Git repository
Fetching without tags
> git rev-parse --is-inside-work-tree # timeout=10
> git config remote.origin.url https://gitee.com/Qiao-Zhi/devops-maven-sample.git # timeout=10
Fetching upstream changes from https://gitee.com/Qiao-Zhi/devops-maven-sample.git
> git --version # timeout=10
> git --version # 'git version 2.11.0'
using GIT_ASKPASS to set credentials
> git fetch --no-tags --progress -- https://gitee.com/Qiao-Zhi/devops-maven-sample.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking out Revision 67401e9a817a371f9da32d5606d591e7b3e0598f (master)
Commit message: "v1"
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (unit test)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
> git config core.sparsecheckout # timeout=10
> git checkout -f 67401e9a817a371f9da32d5606d591e7b3e0598f # timeout=10
+ mvn clean test
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< io.kubesphere.devops:devops-sample >-----------------
[INFO] Building devops-sample :: HelloWorld Demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ devops-sample ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (agent-for-ut) @ devops-sample ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ devops-sample ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.kubesphere.devops.HelloWorldControllerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.527 s - in io.kubesphere.devops.HelloWorldControllerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.872 s
[INFO] Finished at: 2022-03-06T04:25:53Z
[INFO] ------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (build & push)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ mvn clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< io.kubesphere.devops:devops-sample >-----------------
[INFO] Building devops-sample :: HelloWorld Demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ devops-sample ---
[INFO] Deleting /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (agent-for-ut) @ devops-sample ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ devops-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ devops-sample ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ devops-sample ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ devops-sample ---
[INFO] Building jar: /home/jenkins/agent/workspace/59wjg_devops-maven-sample_master/target/devops-sample-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- spring-boot-maven-plugin:2.1.11.RELEASE:repackage (repackage) @ devops-sample ---
[INFO] Replacing main artifact with repackaged archive
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.580 s
[INFO] Finished at: 2022-03-06T04:26:16Z
[INFO] ------------------------------------------------------------------------
[Pipeline] sh
+ docker build -f Dockerfile-online -t registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3 .
Sending build context to Docker daemon 17.52MB
Step 1/4 : FROM java:openjdk-8-jre-alpine
---> fdc893b19a14
Step 2/4 : WORKDIR /home
---> Using cache
---> 1e0dc3b20420
Step 3/4 : COPY target/*.jar /home
---> fd0896260f0e
Step 4/4 : ENTRYPOINT java -jar *.jar
---> Running in 27d46c26a862
Removing intermediate container 27d46c26a862
---> 8867ca53b5ca
Successfully built 8867ca53b5ca
Successfully tagged registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3
[Pipeline] withCredentials
Masking supported pattern matches of $DOCKER_USERNAME or $DOCKER_PASSWORD
[Pipeline] {
[Pipeline] sh
+ echo ****
+ docker login registry.cn-hangzhou.aliyuncs.com -u **** --password-stdin
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[Pipeline] sh
+ docker push registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample]
0e1bd0a16495: Preparing
20dd87a4c2ab: Preparing
78075328e0da: Preparing
9f8566ee5135: Preparing
20dd87a4c2ab: Layer already exists
78075328e0da: Layer already exists
9f8566ee5135: Layer already exists
0e1bd0a16495: Pushed
SNAPSHOT-master-3: digest: sha256:91d99fde2a9b9a4f15ead003c8446fb3532aeb7c3173ebaad09d69237eb22756 size: 1159
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (push latest)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ docker tag registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3 registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:latest
[Pipeline] sh
+ docker push registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:latest
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample]
0e1bd0a16495: Preparing
20dd87a4c2ab: Preparing
78075328e0da: Preparing
9f8566ee5135: Preparing
78075328e0da: Layer already exists
20dd87a4c2ab: Layer already exists
9f8566ee5135: Layer already exists
0e1bd0a16495: Layer already exists
latest: digest: sha256:91d99fde2a9b9a4f15ead003c8446fb3532aeb7c3173ebaad09d69237eb22756 size: 1159
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy to dev)
[Pipeline] input
deploy to dev?
Proceed or Abort
Approved by admin
[Pipeline] container
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $KUBECONFIG
[Pipeline] {
[Pipeline] sh
+ envsubst
+ kubectl apply -f -
deployment.apps/ks-sample-dev configured
service/ks-sample-dev unchanged
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (push with tag)
[Pipeline] container
[Pipeline] {
[Pipeline] input
release image with tag?
Proceed or Abort
Approved by admin
[Pipeline] withCredentials
Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[Pipeline] {
[Pipeline] sh
+ git config --global user.email kubesphere@yunify.com
[Pipeline] sh
+ git config --global user.name kubesphere
[Pipeline] sh
+ git tag -a v1 -m v1
[Pipeline] sh
+ git push http://****:****@gitee.com/****/devops-maven-sample.git --tags --ipv4
remote: Powered by [01;33mGITEE.COM [0m[[01;35mGNK-6.3[0m][0m
To http://gitee.com/****/devops-maven-sample.git
* [new tag] v1 -> v1
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] sh
+ docker tag registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:SNAPSHOT-master-3 registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:v1
[Pipeline] sh
+ docker push registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample:v1
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/qlq_repository/devops-maven-sample]
0e1bd0a16495: Preparing
20dd87a4c2ab: Preparing
78075328e0da: Preparing
9f8566ee5135: Preparing
9f8566ee5135: Layer already exists
0e1bd0a16495: Layer already exists
20dd87a4c2ab: Layer already exists
78075328e0da: Layer already exists
v1: digest: sha256:91d99fde2a9b9a4f15ead003c8446fb3532aeb7c3173ebaad09d69237eb22756 size: 1159
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy to production)
[Pipeline] input
deploy to production?
Proceed or Abort
Approved by admin
[Pipeline] container
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $KUBECONFIG
[Pipeline] {
[Pipeline] sh
+ envsubst
+ kubectl apply -f -
deployment.apps/ks-sample created
service/ks-sample created
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
View Code
2》 kubernetes 环境查看相关的资源
[root@k8smaster01 ~]# kubectl get deployments,pods,svc -n kubesphere-devops-dev No resources found in kubesphere-devops-dev namespace. [root@k8smaster01 ~]# kubectl get deployments,pods,svc -n kubesphere-sample-dev NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/ks-sample-dev 1/1 1 1 12h NAME READY STATUS RESTARTS AGE pod/ks-sample-dev-5dc9786dcc-6f9fx 1/1 Running 0 5m48s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ks-sample-dev NodePort 10.1.66.88 <none> 8080:30861/TCP 14h [root@k8smaster01 ~]# kubectl get deployments,pods,svc -n kubesphere-sample-prod NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/ks-sample 2/2 2 2 4m57s NAME READY STATUS RESTARTS AGE pod/ks-sample-68b68dfbc6-dpkwk 1/1 Running 0 4m57s pod/ks-sample-68b68dfbc6-qs2sw 1/1 Running 0 4m57s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ks-sample NodePort 10.1.21.72 <none> 8080:30961/TCP 4m57s
3》gitee 查看打的tag 信息
4》阿里镜像仓库查看版本信息如下：
5》测试访问
xx@xx MINGW64 /d/study/devops-maven-sample (master)
$ curl http://192.168.13.107:30861
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2 100 2 0 0 166 0 --:--:-- --:--:-- --:--:-- 181v1
xx@xx MINGW64 /d/study/devops-maven-sample (master)
$ curl http://192.168.13.107:30961
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2 100 2 0 0 68 0 --:--:-- --:--:-- --:--:-- 71v1
11. 总结
(0) 前置
其实通过kubesphere 流水线操作，最终还是都会到Jenkins中进行操作，其操作和自定义流水线一样。大致就是打代码、测试代码、打包、制作镜像并打tag、传送到镜像仓库、创建kubernetes 相关资源。在jenkins 执行过程中，会创建一个代理pod，比如：
[root@k8smaster01 ~]# kubectl get pods -A| grep maven kubesphere-devops-worker maven-cfqfb 2/2 Running 0 47s kubesphere-devops-worker maven-gzjdm 1/2 Error 0 18h
其中git 拉取代码、maven打包都是在下面pods 操作的。进入一个pod，查看其环境如下：
(1)Jenkins file文件查看分析
pipeline {
agent {
node {
label 'maven'
}
}
parameters {
string(name:'TAG_NAME',defaultValue: '',description:'')
}
environment {
DOCKER_CREDENTIAL_ID = 'ali-registery'
GITHUB_CREDENTIAL_ID = 'gitee-secret'
KUBECONFIG_CREDENTIAL_ID = 'demo-kubeconfig'
REGISTRY = 'registry.cn-hangzhou.aliyuncs.com'
DOCKERHUB_NAMESPACE = 'qlq_repository'
GITHUB_ACCOUNT = 'qiao-zhi'
APP_NAME = 'devops-maven-sample'
}
stages {
stage ('checkout scm') {
steps {
checkout(scm)
}
}
stage ('unit test') {
steps {
container ('maven') {
sh 'mvn clean test'
}
}
}
stage ('build & push') {
steps {
container ('maven') {
sh 'mvn clean package -DskipTests'
sh 'docker build -f Dockerfile-online -t $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER .'
withCredentials([usernamePassword(passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,credentialsId : "$DOCKER_CREDENTIAL_ID" ,)]) {
sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER'
}
}
}
}
stage('push latest'){
when{
branch 'master'
}
steps{
container ('maven') {
sh 'docker tag $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:latest '
sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:latest '
}
}
}
stage('deploy to dev') {
when{
branch 'master'
}
steps {
input(id: 'deploy-to-dev', message: 'deploy to dev?')
container ('maven') {
withCredentials([
kubeconfigFile(
credentialsId: env.KUBECONFIG_CREDENTIAL_ID,
variable: 'KUBECONFIG')
]) {
sh 'envsubst < deploy/dev-all-in-one/devops-sample.yaml | kubectl apply -f -'
}
}
}
}
stage('push with tag'){
when{
expression{
return params.TAG_NAME =~ /v.*/
}
}
steps {
container ('maven') {
input(id: 'release-image-with-tag', message: 'release image with tag?')
withCredentials([usernamePassword(credentialsId: "$GITHUB_CREDENTIAL_ID", passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
sh 'git config --global user.email "kubesphere@yunify.com" '
sh 'git config --global user.name "kubesphere" '
sh 'git tag -a $TAG_NAME -m "$TAG_NAME" '
sh 'git push http://$GIT_USERNAME:$GIT_PASSWORD@gitee.com/$GITHUB_ACCOUNT/$APP_NAME.git --tags --ipv4'
}
sh 'docker tag $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:$TAG_NAME '
sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:$TAG_NAME '
}
}
}
stage('deploy to production') {
when{
expression{
return params.TAG_NAME =~ /v.*/
}
}
steps {
input(id: 'deploy-to-production', message: 'deploy to production?')
container ('maven') {
withCredentials([
kubeconfigFile(
credentialsId: env.KUBECONFIG_CREDENTIAL_ID,
variable: 'KUBECONFIG')
]) {
sh 'envsubst < deploy/prod-all-in-one/devops-sample.yaml | kubectl apply -f -'
}
}
}
}
}
}
前面定义了一些全局环境变量，按阶段进行分析
1》checkout scm 拉取代码
2》unit test 执行单元测试
3》build & push: 执行mvn打包， 然后基于Dockerfile-online 构造本地镜像，镜像的版本用代码分支和buildnumber 做区分，然后将镜像推送代码镜像仓库
4》push latest： 当代码分支是master 的时候，将3》的镜像打包为latest版本的镜像，然后推到镜像仓库
5》deploy to dev：当代码分支是master 的时候，相当于是执行 deploy/dev-all-in-one/devops-sample.yaml文件，相当于到kubernetes 创建资源
6》push with tag：当我们传递的参数是vxxx的时候，将代码打上相应的标签并推送到代码仓库；将镜像也打上相应的标签并推送到相应的镜像仓库
7》deploy to production：当我们传递的参数是vxxx的时候， 需要我们验证下是否需要将相关资源创建到kubernetes环境的prod 仓库，使用的文件是deploy/prod-all-in-one/devops-sample.yaml。
envsubst < deploy/prod-all-in-one/devops-sample.yaml | kubectl apply -f -
envsubst 是用shell格式字符串中的值替换环境变量。要替换的变量应位于${var}或$var格式。例如
[root@k8smaster01 ~]# export MYKEY=123456 [root@k8smaster01 ~]# echo $MYKEY 123456 [root@k8smaster01 ~]# cat test.txt $MYKEY [root@k8smaster01 ~]# envsubst < test.txt 123456
(2) 几个重要kubernetes 文件如下：
deploy/dev-all-in-one/devops-sample.yaml：
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: kubesphere
component: ks-sample-dev
tier: backend
name: ks-sample-dev
namespace: kubesphere-sample-dev
spec:
progressDeadlineSeconds: 600
replicas: 1
selector:
matchLabels:
app: kubesphere
component: ks-sample-dev
tier: backend
template:
metadata:
labels:
app: kubesphere
component: ks-sample-dev
tier: backend
spec:
containers:
- env:
- name: CACHE_IGNORE
value: js|html
- name: CACHE_PUBLIC_EXPIRATION
value: 3d
image: $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER
readinessProbe:
httpGet:
path: /
port: 8080
timeoutSeconds: 10
failureThreshold: 30
periodSeconds: 5
imagePullPolicy: Always
name: ks-sample
ports:
- containerPort: 8080
protocol: TCP
resources:
limits:
cpu: 300m
memory: 600Mi
requests:
cpu: 100m
memory: 100Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
labels:
app: kubesphere
component: ks-sample-dev
name: ks-sample-dev
namespace: kubesphere-sample-dev
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
nodePort: 30861
selector:
app: kubesphere
component: ks-sample-dev
tier: backend
sessionAffinity: None
type: NodePort
View Code
deploy/prod-all-in-one/devops-sample.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: kubesphere
component: ks-sample
tier: backend
name: ks-sample
namespace: kubesphere-sample-prod
spec:
progressDeadlineSeconds: 600
replicas: 2
selector:
matchLabels:
app: kubesphere
component: ks-sample
tier: backend
strategy:
rollingUpdate:
maxSurge: 100%
maxUnavailable: 100%
type: RollingUpdate
template:
metadata:
labels:
app: kubesphere
component: ks-sample
tier: backend
spec:
containers:
- env:
- name: CACHE_IGNORE
value: js|html
- name: CACHE_PUBLIC_EXPIRATION
value: 3d
image: $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:$TAG_NAME
readinessProbe:
httpGet:
path: /
port: 8080
timeoutSeconds: 10
failureThreshold: 30
periodSeconds: 5
imagePullPolicy: Always
name: ks
ports:
- containerPort: 8080
protocol: TCP
resources:
limits:
cpu: 300m
memory: 600Mi
requests:
cpu: 100m
memory: 100Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
labels:
app: kubesphere
component: ks-sample
name: ks-sample
namespace: kubesphere-sample-prod
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
nodePort: 30961
selector:
app: kubesphere
component: ks-sample
tier: backend
sessionAffinity: None
type: NodePort
View Code
代码仓库和修改后配置文件：https://gitee.com/Qiao-Zhi/devops-maven-sample
补充： 在安装过程中遇到一些磁盘不足，报错信息如下:
Warning Evicted 49m kubelet The node had condition: [DiskPressure].
解决参考：
https://blog.51cto.com/riverxyz/2758421
vm中对centos 扩容: https://ld246.com/article/1566021346577
补充： 如果是阿里私有镜像仓库，需要创建secret 然后拉取，如下
1. 创建私有镜像仓库: 账号密码记得替换
kubectl create secret docker-registry registry-aliyun --namespace=kubesphere-sample-dev \
--docker-server=registry.cn-hangzhou.aliyuncs.com --docker-username=usernamre \
--docker-password=pwd --docker-email=xxx@163.com
2. 修改yaml文件，增加镜像拉取secret
(1) deploy/dev-all-in-one/devops-sample.yaml：
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: kubesphere
component: ks-sample-dev
tier: backend
name: ks-sample-dev
namespace: kubesphere-sample-dev
spec:
progressDeadlineSeconds: 600
replicas: 1
selector:
matchLabels:
app: kubesphere
component: ks-sample-dev
tier: backend
template:
metadata:
labels:
app: kubesphere
component: ks-sample-dev
tier: backend
spec:
containers:
- env:
- name: CACHE_IGNORE
value: js|html
- name: CACHE_PUBLIC_EXPIRATION
value: 3d
image: $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER
readinessProbe:
httpGet:
path: /
port: 8080
timeoutSeconds: 10
failureThreshold: 30
periodSeconds: 5
imagePullPolicy: Always
name: ks-sample
ports:
- containerPort: 8080
protocol: TCP
resources:
limits:
cpu: 300m
memory: 600Mi
requests:
cpu: 100m
memory: 100Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullSecrets:
- name: registry-aliyun
dnsPolicy: ClusterFirst
restartPolicy: Always
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
labels:
app: kubesphere
component: ks-sample-dev
name: ks-sample-dev
namespace: kubesphere-sample-dev
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
nodePort: 30861
selector:
app: kubesphere
component: ks-sample-dev
tier: backend
sessionAffinity: None
type: NodePort
View Code
(2) deploy/prod-all-in-one/devops-sample.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: kubesphere
component: ks-sample
tier: backend
name: ks-sample
namespace: kubesphere-sample-prod
spec:
progressDeadlineSeconds: 600
replicas: 2
selector:
matchLabels:
app: kubesphere
component: ks-sample
tier: backend
strategy:
rollingUpdate:
maxSurge: 100%
maxUnavailable: 100%
type: RollingUpdate
template:
metadata:
labels:
app: kubesphere
component: ks-sample
tier: backend
spec:
containers:
- env:
- name: CACHE_IGNORE
value: js|html
- name: CACHE_PUBLIC_EXPIRATION
value: 3d
image: $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:$TAG_NAME
readinessProbe:
httpGet:
path: /
port: 8080
timeoutSeconds: 10
failureThreshold: 30
periodSeconds: 5
imagePullPolicy: Always
name: ks
ports:
- containerPort: 8080
protocol: TCP
resources:
limits:
cpu: 300m
memory: 600Mi
requests:
cpu: 100m
memory: 100Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullSecrets:
- name: registry-aliyun
dnsPolicy: ClusterFirst
restartPolicy: Always
terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
labels:
app: kubesphere
component: ks-sample
name: ks-sample
namespace: kubesphere-sample-prod
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
nodePort: 30961
selector:
app: kubesphere
component: ks-sample
tier: backend
sessionAffinity: None
type: NodePort
View Code
标签：INFO,maven,Pipeline,git,kubesphere,devops,sample,安装 来源： https://www.cnblogs.com/qlqwjy/p/15965008.html
本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享； 2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关； 3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关； 4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除； 5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。