Jenkins helm kubernetes java

jenkins

helm repo add jenkins https://charts.jenkins.io
helm install jenkins jenkins/jenkins -f values.yaml

https://www.jenkins.io/zh/doc/book/
插件站点:
维护方	镜像中心地址
官方 				https://updates.jenkins.io/update-center.json
Jenkins 中文社区	 	https://updates.jenkins-zh.cn/update-center.json
清华大学				https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json
华为开源镜像站		https://mirrors.huaweicloud.com/jenkins/updates/update-center.json
腾讯					https://mirrors.cloud.tencent.com/jenkins/updates/update-center.json
ustc				https://mirrors.ustc.edu.cn/jenkins/updates/update-center.json
bit					https://mirror.bit.edu.cn/jenkins/updates/update-center.json
lework	https://cdn.jsdelivr.net/gh/lework/jenkins-update-center/updates/tencent/update-center.json https://cdn.jsdelivr.net/gh/lework/jenkins-update-center/updates/tsinghua/update-center.json https://cdn.jsdelivr.net/gh/lework/jenkins-update-center/updates/ustc/update-center.json https://cdn.jsdelivr.net/gh/lework/jenkins-update-center/updates/bit/update-center.json

挂载目录 jenkins_home目录下 updates文件夹修改国内连接提速。
sed -i 's/https:\/\/updates.jenkins.io\/download/http:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g' /var/lib/jenkins/updates/default.json
sed -i 's/https:\/\/www.google.com/https:\/\/www.baidu.com/g' /var/lib/jenkins/updates/default.json
到 Jenkins 的 Manage Jenkins -> Manage Plugins -> Advanced 页面,修改 Update Site 的 URL 地址
将 https://updates.jenkins.io/update-center.json 改为 https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json
关闭检查:  JAVA_OPTS=-Dhudson.model.DownloadService.noSignatureCheck=true

必备插件:
中文支持: Locale plugin   Localization: Chinese (Simplified)    zh_CN
Active Choices   
Blue Ocean
Blue Ocean Core JS
Blue Ocean Pipeline Editor
Dashboard for Blue Ocean
Kubernetes
Kubernetes CLI
Kubernetes Credentials
Delivery Pipeline
Git Parameter
Hidden Parameter
List Git Branches ParameterVersion
ldap
Pipeline
Maven Integration
Docker
Docker Pipeline
Matrix Authorization Strategy
Pipeline Utility Steps
gitlab api
GitLab
GitLab Authentication
Generic Webhook Trigger
Build Authorization Token Root
JIRA Integration for Blue OceanVersion
Autofavorite for Blue Ocean

## 接入ldap
server:  openldap:389
root DN:   空   Allow blank rootDN
User search base:  ou=people,dc=vimll,dc=com
User search filter: uid={0}
Group search base : ou=groups,dc=vimll,dc=com
Group search filter:  空
Search for LDAP groups containing user
Manager DN:  cn=admin,dc=vimll,dc=com
Manager Password:  •••••••••••
Display Name LDAP attribute:  cn
Email Address LDAP attribute:  mail

授权策略使用安全矩阵:通过LDAP增加用户到不同用户组来控制权限。


## 配置jenkins使用kubernetes
在Manage Plugin页面,搜索Kubernetes,勾选并安装。
Configure Clouds
kubernetes    名称将会在Jenkins Pipeline中用到,配置多个Kubernetes云时,需要为每一个云都指定一个不同的名称。
https://192.168.9.33:16443
禁用 HTTPS 证书检查
Kubernetes 服务证书 key:  
从Kubernetes API server的/root/.kube/config文件中,获取/root/.kube/config中certificate-authority-data的内容base64解码即可。
#echo certificate-authority-data的内容 | base64 -d > ca.pem
ca.crt的内容就是Kubernetes服务证书key

## 生成凭据
将/root/.kube/config中client-certificate-data和client-key-data的内容base64解码
#echo client-certificate-data的内容 | base64 -d > ~/client.pem
#echo client-key-data的内容 | base64 -d > ~/client.key
openssl pkcs12 -export -out cert.pfx -inkey client.key -in client.pem -certfile ca.pem
自定义一个password并牢记
点击凭据Add,选择类型是Cetificate,点击Upload certificate,选取前面生成cert.pfx文件,输入生成cert.pfx文件时的密码,就完成了凭据的添加
Manage Jenkins -> Configure Global Security -> TCP port for inbound agents 指定端口 50000
Jenkins 地址:  http://jenkins:80
Jenkins 通道:  http://jenkins:50000

创建凭据:系统->全局凭据->添加凭据

常用命令pipeline
podTemplate{
    node('jenkins-jenkins-agent') {
       //git clone
        stage('git') {
        	// 拉取的“分支”、“显示拉取日志”、“拉取的凭据”、“拉取的地址”
            git branch: "master" ,changelog: true , credentialsId: "gitlab", url:  'https://g.vimll.com:9888/root/helloworld.git'    
        }
       //自动安装maven build java
        stage('maven') {
            withMaven (
                maven: 'maven'   
            ) {
            sh "mvn clean package"
            sh "java -cp target/helloworld-1.1.jar com.coveros.demo.helloworld.HelloWorld"
            }
        }

    }
}


// 此方法是设置docker仓库地址,然后选择存了用户名、密码的凭据ID进行验证。注意,只有在此方法之中才生效。
## docker插件相关格式  https://docs.cloudbees.com/docs/admin-resources/latest/plugins/docker-workflow
docker.withRegistry('https://docker.mycorp.com/', 'docker-login') {
  git '…'
  docker.build('myapp').push('latest')
}

## 内网使用http连接镜像库 /etc/docker/daemon.json 增加 "insecure-registries":["192.168.9.6:50000"],   

## https://github.com/jenkinsci/kubernetes-plugin
## https://jenkins.io/doc/pipeline/steps/kubernetes-cli/
// 提供 kubectl 执行的环境,其中得设置存储了 token 的凭据ID和 kubernetes api 地址
        withKubeConfig([credentialsId: "kubectl",serverUrl: "https://kubernetes.default.svc.cluster.local"]) {
            sh "kubectl version"
            sh "kubectl get node"
}


##系统管理—>Managed files—>Add a new Config—>Global Maven settings.xml
在里面添加一个全局的 setting.xml 设置,为了加快 jar 包的下载速度,这里将仓库地址指向 aliyun Maven 仓库地址。
<mirror>
    <id>alimaven</id>
    <name>aliyun maven</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
</mirror>

## 系统管理—>Managed files—>Add a new Config—>Global Maven settings.xml
## https://jenkins.io/doc/pipeline/steps/config-file-provider/
// 生成 settings.xml 文件,这个方法第一个参数是引用文件ID,第二个是生成的文件名
configFileProvider([configFile(fileId: "572fd620-85af-4ed0-b66a-74af59a8c94a", targetLocation: "settings.xml")]) {
    // 只有在方法里面该文件才存在
    echo "cat settings.xml"
}

## 读取 pom.xml 文件  https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/
pom = readMavenPom file: "./pom.xml"  
echo "${pom.artifactId}:${pom.version}"

###测试###
SpringBoot源码: 用于测试的 helloworld 的SpringBoot项目。
Dockerfile: 用于 Docker 编译镜像的文件,比如打包的基础镜像等等。
values.yaml: 用于 Helm 启动的chart的配置文件,里面设置了一些chart的配置信息,告知该如何启动应用程序。
https://github.com/my-dlq/springboot-helloworld
## jdk8基础镜像 Dockerfile
FROM registry.cn-shanghai.aliyuncs.com/mydlq/openjdk:8u201-jdk-alpine3.9
VOLUME /tmp
ADD target/*.jar app.jar
RUN sh -c 'touch /app.jar'
ENV JAVA_OPTS="-Xmx512M -Xms256M -Xss256k -Duser.timezone=Asia/Shanghai"
ENV APP_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar $APP_OPTS" ]

## 测试 kubernetes 中 jenkins agent 代理
// 代理名称,填写系统设置中设置的 Cloud 中 Template 模板的 label
def label = "jenkins-jenkins-agent"    
// 调用Kubernetes提供的方法
podTemplate(label: label,cloud: 'kubernetes' ){
    // 在代理节点上运行脚本
    node (label) {
        echo "测试 kubernetes 中 jenkins agent 代理!~"
    }
}



## 部署模板
deployment.yaml
------------------------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: APP_NAME
    version: v1
  name: APP_NAME-v1
spec:
  minReadySeconds: 10
  replicas: 1
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: APP_NAME
      version: v1
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: APP_NAME
        version: v1
    spec:
      containers:
      - name: APP_NAME
        image: IMAGE_NAME
        env:
        - name: BUILD
          value: "BUILD_NUMBER"
        #- name: LOGGER
        #  value: nest-log4j
        #- name: NEST_HOME
        #  value: /nest
        #- name: SPRING_CONFIG_LOCATION
        #  value: classpath:/,classpath:/config/,file:./,file:./config/,file:/config/
        #- name: SPRING_PROFILES_ACTIVE
        #  value: production
        - name: JAVA_OPTS
          value: -server -XX:+UseParallelGC -XX:+UseParallelOldGC
        #volumeMounts:
        #- name: config-volume
        #  mountPath: /nest/config
        #- name: spring-config
        #  mountPath: /config
        ports:
        - name: http
          containerPort: 8080
        livenessProbe:
          httpGet:
            path: /hello
            port: http
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 5
          successThreshold: 1
          failureThreshold: 1
          timeoutSeconds: 2
        readinessProbe:
          httpGet:
            path: /hello
            port: http
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 5
          successThreshold: 1
          failureThreshold: 1
          timeoutSeconds: 2
        resources:
          limits:
            memory: 800M
          requests:
            cpu: 100m
            memory: 100Mi
        lifecycle:
          preStop:
            exec:
              command: ["/bin/bash", "-c", "sleep 20"]
      imagePullSecrets:
      - name: docker-vimll
      #volumes:
      #- name: config-volume
      #  configMap:
      #    name: nesthome-config
      #- name: spring-config
      #  configMap:
      #    name: APP_NAME-config
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: APP_NAME
  name: APP_NAME
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/port: '11111'
spec:
  ports:
  - name: http
    port: 80
    targetPort: http
  selector:
    app: APP_NAME
  type: ClusterIP

-----------------------------------------      

测试运行环境:
def label = "jenkins-jenkins-agent"
podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git阶段'){
            echo "1、开始拉取代码"
            sh "git version"
        }
        stage('Maven阶段'){
                echo "2、开始Maven编译、推送到本地库"
                sh "mvn -version"
        }
        stage('Docker阶段'){
                echo "3、开始读取Maven pom变量,并执行Docker编译、推送、删除"
                sh "docker version"
        }
         stage('kubectl'){
                echo "4、开始检测Kubectl环境与执行部署"
                	sh "kubectl version"
            }
    }
}


测试工具认证环境:
def label = "jenkins-jenkins-agent"
podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git阶段'){
            echo "1、开始拉取代码"
            sh "git version"
            git branch: "master" ,changelog: true , credentialsId: "gitlab", url:  'https://g.vimll.com:9888/root/helloworld.git'    
        }
        stage('Maven阶段'){
                echo "2、开始Maven编译、推送到本地库"
                sh "mvn -version"
                sh "mvn clean package" 
                sh "java -cp target/helloworld-1.1.jar com.coveros.demo.helloworld.HelloWorld"
        }
        stage('Docker阶段'){
                echo "3、开始读取Maven pom变量,并执行Docker编译、推送、删除"
                sh "docker version"
                hub = "g.vimll.com:9888"
                project_name = "helloworld"
                docker.withRegistry("https://${hub}", "gitlab") {
                    def customImage = docker.build("${hub}/root/${project_name}:test")
                    echo "推送镜像"
                    customImage.push('test')
                }
        }
         stage('kubectl'){
                echo "4、开始检测Kubectl环境与执行部署"
                withKubeConfig([credentialsId: "kubectl",serverUrl: "https://kubernetes.default.svc.cluster.local"]) {
                	sh "kubectl version"
                	sh "kubectl get node"
                }
            }
    }
}


测试helloword正式流程:
def label = "jenkins-jenkins-agent"
podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git阶段'){
            echo "1、开始拉取代码"
            sh "git version"
            git branch: "master" ,changelog: true , credentialsId: "gitlab", url:  'https://g.vimll.com:9888/root/springboot-helloworld.git'    
        }
        stage('Maven阶段'){
                echo "2、开始Maven编译、推送到本地库"
                sh "mvn -version"
                configFileProvider([configFile(fileId: "572fd620-85af-4ed0-b66a-74af59a8c94a", targetLocation: "settings.xml")]) {
                sh "mvn clean install -Dmaven.test.skip=true --settings settings.xml" 
                sh "ls -l target/*"
                }
        }
        stage('Docker阶段'){
                echo "3、开始读取Maven pom变量,并执行Docker编译、推送、删除"
                sh "docker version"
                echo "读取 pom.xml 参数"
                pom = readMavenPom file: './pom.xml'
                sh "echo ${pom.artifactId}:${pom.version}"
                sh "cat ./pom.xml"
                hub = "192.168.9.6:50000"
                project_name = "helloworld"
                docker.withRegistry("http://${hub}", "gitlab") {
                    def customImage = docker.build("${hub}/root/${project_name}/${pom.artifactId}:${pom.version}")
                    echo "推送镜像"
                    customImage.push("${pom.version}")
                }
        }
         stage('kubectl'){
                script{
					env.VAR3="$BUILD_NUMBER" 
					env.IMAGE="${hub}/root/${project_name}/${pom.artifactId}:${pom.version}"
					env.APP_NAME="${pom.artifactId}"
				}
                echo "4、开始检测Kubectl环境与执行部署"
                withKubeConfig([credentialsId: "kubectl",serverUrl: "https://kubernetes.default.svc.cluster.local"]) {
                	sh "kubectl version"
                	sh "kubectl get node"
                	sh "echo $IMAGE"
                	sh "echo $APP_NAME"
                	sh 'cat deployment.yaml | sed "s#IMAGE_NAME#$IMAGE#g" | sed "s#APP_NAME#$APP_NAME#g"|sed "s#BUILD_NUMBER#$BUILD_NUMBER#g"|kubectl apply -f -'
                }
            }
    }
}

curl 10.131.15.248:8080/hello