wordpress怎么发布,太原网站制作优化seo公司,wordpress门户主体,模板网站建设价位通过本文#xff0c;您将可以学习到 sonarqube、git\gitlab、shell、sonar-scanner、sonarlint
一、前言 sonarqube 是一款开源的静态代码扫描工具。
实际生产应用中#xff0c;sonarqube 如何落地#xff0c;需要考虑以下四个维度#xff1a;
1、规则的来源
现在规则的…通过本文您将可以学习到 sonarqube、git\gitlab、shell、sonar-scanner、sonarlint
一、前言 sonarqube 是一款开源的静态代码扫描工具。
实际生产应用中sonarqube 如何落地需要考虑以下四个维度
1、规则的来源
现在规则的来源主要有三个途径
1sonar way 官方内置规则支持多种语言
2应用市场插件
3自定义规则插件
自定义规则插件缺点是难度大、开发周期长、版本迭代兼容要自己维护优点是可定制化、灵活性较高。
2、扫描的分支
sonarqube 社区版只支持对 main 或者叫 master 这一个分支进行扫描。不过可通过导入插件增加对多分支的支持。
3、全量扫描or增量扫描
全量扫描是针对整个工程代码进行扫描。这是 sonarqube 默认的扫描行为。
增量扫描是针对当次提交代码进行扫描。需要自己开发 shell 脚本支持。
全量扫描的缺点是每次都要针对整个工程代码进行扫描扫描时间长。我能不能只针对我当前提交的代码进行扫描呢这便是增量扫描。增量扫描的缺点是开发难度大。
4、扫描的时机
现在主流的接入方式有三种扫描时机
1Jenkins流水线 git 拉取代码后maven 构建项目前扫描。
扫描失败中止流水线。
2gitlab CI 流水线
与 Jenkins 流水线类似也是扫描失败中止流水线。
3) git commit
git 提交代码时触发扫描。包括客户端钩子和服务端钩子。
客户端钩子——pre-commit。开发人员 git commit 提交代码到本地仓库时触发扫描扫描完获取报告失败则拒绝提交。
服务端 git hook 分为三种分别是
pre-receive推送前updatepost-receive推送后
这三个步骤就是我们本地 push 完代码服务端要做的事情如图所示 从质量保证 QAQuality Assurance的角度来说代码扫描的时机越早越好。如果是到项目构建的时候再来扫描扫描不通过就得打回去开发重新修改修改完重新测试重新启动流水线构建整个周期会被拉长。
所以 pre-commit 和 pre-receive 的效果会比代码已经进入仓库了再来扫描的 Jenkins流水线、gitlab CI 流水线要好。
5、服务端or客户端
服务端扫描 pre-receive 好还是客户端扫描 pre-commit 好呢
答案显而易见服务端扫描好。
因为客户端扫描严重依赖开发人员本地环境下的 pre-commit 文件如果这个文件被开发人员删除了那客户端扫描无从谈起。而且如果扫描依赖一些软件如 sonar-scanner则每个开发的本地环境都需要安装这个在项目比较大开发人员比较多的时候很难做到协调一致。 本文采用 pre-receive 钩子的方式支持增量扫描与多分支扫描。 二、环境准备
1、一台装有 gitlab 的服务器。
gitlab 的安装可以参考我的博文。docker 安装跟 Linux 裸机安装有所不同推荐使用 docker 安装。
docker 下安装 gitlab_gitlab docker 安装没有initial_root_password文件-CSDN博客
2、一台装有 sonarqube 9.9 LTS 的服务器。
sonarqube 的安装可以参考我的博文。docker compose 下安装 sonarqube带多分支插件_docker-compose sonarqube-CSDN博客
三、步骤
1、安装 sonar-scanner
gitlab 是 docker 安装的情况
将 sonar-scanner-cli-5.0.1.3006-linux.zip 上传到 gitlab 服务器某一个文件夹下。
就比如 /usr/local/sonar
# 安装 unzip 命令
yum -y install unzip# 解压 sonar-scanner-cli-5.0.1.3006-linux.zip
unzip sonar-scanner-cli-5.0.1.3006-linux.zip# 给文件夹重命名
mv sonar-scanner-cli-5.0.1.3006-linux.zip sonar-scanner# 查看当前有多少运行中的容器
docker ps# 将 sonar-scanner 文件夹内容拷贝进 gitlab 容器 /usr/local/sonar-scanner 下
docker cp sonar-scanner gitlab:/usr/local/sonar-scanner # 进入容器查看 sonar-scanner 文件夹是否拷贝到位
docker exec -it gitlab /bin/bash
cd /usr/local
ls
# 修改 sonar-scanner 配置文件
cd sonar-scanner/conf
vi sonar-scanner.properties sonar-scanner.properties 配置内容如下
#Configure here general information about the environment, such as SonarQube server connection details for example
#No information about specific project should appear here#----- Default SonarQube server
# 该项可配可不配
sonar.host.urlhttp://你的sonarqubeIP:9000#----- Default source code encoding
sonar.sourceEncodingUTF-8 至此sonar-scanner 安装完成。
gitlab 是 Linux 裸机安装的情况
2、在 gitlab 管理中心找到项目的相对路径
以 root 用户登录 gitlab 拼接上 gitlab 的安装目录就是项目的绝对路径
就比如我的 gitlab 是 docker 安装的默认安装于 /srv/gitlab
则我的项目 sonar-test存放于宿主机目录 这个路径其实是会挂载进容器的这个后面就能知道 3、创建服务端 hook 文件
在刚才找到的项目路径下创建 custom_hooks 文件夹和 pre-receive 文件
# 创建 custom_hooks 文件夹
mkdir custom_hooks
# 进入 custom_hooks 文件夹
cd custom_hooks
# 创建 pre-receive 文件
touch pre-receive
# 修改 pre-receive 文件
vim pre-receive pre-receive 文件内容如下
#!/bin/bashecho 开始代码扫描...GIT_VERSIONgit version
echo git版本是$GIT_VERSION# 获取当前路径
BASE_PATHpwd
echo 当前路径是$BASE_PATHACCOUNTwhoami
echo 当前账户是$ACCOUNT# 从标准输入流读入参数
read normalInput
ARR($normalInput)
OLD_REVISION${ARR[0]}
CURRENT_REVISION${ARR[1]}
BRANCH${ARR[2]}echo 旧修订ID$OLD_REVISION
echo 新修订ID$CURRENT_REVISIONecho 当前项目$GL_PROJECT_PATH
PROJECT_NAME$(echo $GL_PROJECT_PATH | awk BEGIN{FS/}{print $NF})
echo 项目名称$PROJECT_NAMEecho 当前分支$BRANCH
BRANCH_NAME$(echo $BRANCH | awk BEGIN{FS/}{print $NF})
echo 分支名称$BRANCH_NAMEexport SONAR_SCANNER/usr/local/sonar-scanner
PATH$SONAR_SCANNER/bin:$PATH
export PATHif [ $BRANCH_NAME develop ]; then# 过滤出当前提交中的 java 文件FILESgit diff --name-only $OLD_REVISION $CURRENT_REVISION | grep -e \.java$if [ -n $FILES ]; thenSONARDIR$BASE_PATH/sonarmkdir -p ${SONARDIR}TEMPDIR$BASE_PATH/tmpfor FILE in ${FILES}; do# 创建目录并舍弃输出mkdir -p ${TEMPDIR}/dirname ${FILE} /dev/null# 创建文件git show $CURRENT_REVISION:$FILE ${TEMPDIR}/${FILE}done;echo 进入临时文件夹cd $TEMPDIRpwdSONAR_USER_HOME$SONARDIR sonar-scanner -Dsonar.languagejava -Dsonar.projectKey$PROJECT_NAME -Dsonar.host.urlhttp://你的sonarqubeIP:端口 -Dsonar.login你的项目Token -Dsonar.branch.name$BRANCH_NAME -Dsonar.projectVersion1.0 -Dsonar.java.binaries./ -Dsonar.scm.disabledtruesleep 5srt$(curl -u 你的项目Token: http://你的sonarqubeIP:端口/api/qualitygates/project_status?projectKey$PROJECT_NAME\branch$BRANCH_NAME | awk -F : {print $3} | awk -F , {print $1})echo 代码检测结果$rtecho 删除临时文件夹rm -rf $TEMPDIRif [ $rt \ERROR\ ];thenecho 代码检测失败拒绝提交 exit 1elif [ $rt \OK\ ];thenecho 代码检测成功进入仓库exit 0fi fifi说明未完待续 4、修改文件权限
chmod 777 pre-receive 5、验证
(1) 新建质量配置
在 sonarqube 中点击 Quality Profiles 质量配置
在 Filter profiles by 输入 Java找到默认配置 Sonar way。打开右侧齿轮点击 Copy。 新建一个质量配置名字叫 test 点击 Rules将其他质量配置都去掉只保留一条规则
Zero should not be a possible denominator 并设置为 Blocker 阻断 将 test 质量配置设置为 DEFAULT (2) 新增质量阈
新增一个名叫 test 的质量阈 删除所有默认配置 增加一个针对所有代码Overall Code阻断问题Blocker issues大于is greater than0
的质量条件。表示代码检测中只要出现一个以上阻断问题代码检测就不通过。 3配置项目
打开项目配置为项目添加一个 Java 语言的质量配置选择我们刚才新建的 test
该质量配置现在也是我们实例默认DEFAULT的质量配置。 配置项目质量阈选择我们刚才新建的 test保存。 在项目里面随便写几行不符合规则的代码然后 commit 到本地仓库push 到远程仓库的时候
会提示被拦截。 在 git Console 标签页会看到我们 SHELL 脚本写的日志 Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 remote: 开始代码扫描... remote: git版本是git version 2.43.0 remote: 当前路径是/var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git remote: 当前账户是git remote: 旧修订ID871ca52c316d810194a44657f6e3f46aee7fffb9 remote: 新修订IDc83f9099f6bbd3ba04691cd7a9e315604923ba60 remote: 当前项目macrosoft/sonar-test remote: 项目名称sonar-test remote: 当前分支refs/heads/master remote: 分支名称master remote: 进入临时文件夹 remote: /var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/tmp remote: INFO: Scanner configuration file: /usr/local/sonar-scanner/conf/sonar-scanner.properties remote: INFO: Project root configuration file: NONE remote: INFO: SonarScanner 5.0.1.3006 remote: INFO: Java 17.0.7 Eclipse Adoptium (64-bit) remote: INFO: Linux 3.10.0-1062.18.1.el7.x86_64 amd64 remote: INFO: User cache: /var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/sonar/cache remote: INFO: Analyzing on SonarQube server 9.9.4.87374 remote: INFO: Default locale: en_US, source code encoding: UTF-8 remote: INFO: Load global settings remote: INFO: Load global settings (done) | time138ms remote: INFO: Server id: 243B8A4D-AY4oeopTrXOQ26gUW39N remote: INFO: User cache: /var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/sonar/cache remote: INFO: Load/download plugins remote: INFO: Load plugins index remote: INFO: Load plugins index (done) | time102ms remote: INFO: Load/download plugins (done) | time319ms remote: INFO: Process project properties remote: INFO: Process project properties (done) | time1ms remote: INFO: Execute project builders remote: INFO: Execute project builders (done) | time4ms remote: INFO: Project key: sonar-test remote: INFO: Base dir: /var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/tmp remote: INFO: Working dir: /var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/tmp/.scannerwork remote: INFO: Load project settings for component key: sonar-test remote: INFO: Load project settings for component key: sonar-test (done) | time25ms remote: INFO: Load project branches remote: INFO: Load project branches (done) | time39ms remote: INFO: Load branch configuration remote: INFO: Load branch configuration (done) | time13ms remote: INFO: Load quality profiles remote: INFO: Load quality profiles (done) | time64ms remote: INFO: Load active rules remote: INFO: Load active rules (done) | time1141ms remote: INFO: Load analysis cache remote: INFO: Load analysis cache (404) | time15ms remote: INFO: Branch name: master remote: INFO: Load project repositories remote: INFO: Load project repositories (done) | time32ms remote: INFO: Indexing files... remote: INFO: Project configuration: remote: INFO: 3 files indexed remote: INFO: Quality profile for java: test remote: INFO: ------------- Run sensors on module sonar-test remote: INFO: Load metrics repository remote: INFO: Load metrics repository (done) | time29ms remote: INFO: Sensor JavaSensor [java] remote: INFO: Configured Java source version (sonar.java.source): none remote: INFO: JavaClasspath initialization remote: INFO: JavaClasspath initialization (done) | time5ms remote: INFO: JavaTestClasspath initialization remote: INFO: JavaTestClasspath initialization (done) | time1ms remote: INFO: Server-side caching is enabled. The Java analyzer will not try to leverage data from a previous analysis. remote: INFO: Using ECJ batch to parse 3 Main java source files with batch size 98 KB. remote: INFO: Starting batch processing. remote: INFO: The Java analyzer cannot skip unchanged files in this context. A full analysis is performed for all files. remote: INFO: 100% analyzed remote: INFO: Batch processing: Done. remote: INFO: Did not optimize analysis for any files, performed a full analysis for all 3 files. remote: INFO: No Test source files to scan. remote: INFO: No Generated source files to scan. remote: INFO: Sensor JavaSensor [java] (done) | time2039ms remote: INFO: Sensor JaCoCo XML Report Importer [jacoco] remote: INFO: sonar.coverage.jacoco.xmlReportPaths is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml remote: INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer remote: INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time5ms remote: INFO: Sensor CSS Rules [javascript] remote: INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped. remote: INFO: Sensor CSS Rules [javascript] (done) | time2ms remote: INFO: Sensor C# Project Type Information [csharp] remote: INFO: Sensor C# Project Type Information [csharp] (done) | time1ms remote: INFO: Sensor C# Analysis Log [csharp] remote: INFO: Sensor C# Analysis Log [csharp] (done) | time34ms remote: INFO: Sensor C# Properties [csharp] remote: INFO: Sensor C# Properties [csharp] (done) | time1ms remote: INFO: Sensor SurefireSensor [java] remote: INFO: parsing [/var/opt/gitlab/git-data/repositories/hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/tmp/target/surefire-reports] remote: INFO: Sensor SurefireSensor [java] (done) | time4ms remote: INFO: Sensor HTML [web] remote: INFO: Sensor HTML [web] (done) | time10ms remote: INFO: Sensor TextAndSecretsSensor [text] remote: INFO: 3 source files to be analyzed remote: INFO: 3/3 source files have been analyzed remote: INFO: Sensor TextAndSecretsSensor [text] (done) | time64ms remote: INFO: Sensor VB.NET Project Type Information [vbnet] remote: INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time3ms remote: INFO: Sensor VB.NET Analysis Log [vbnet] remote: INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time42ms remote: INFO: Sensor VB.NET Properties [vbnet] remote: INFO: Sensor VB.NET Properties [vbnet] (done) | time0ms remote: INFO: Sensor com.github.mc1arke.sonarqube.plugin.scanner.ScannerPullRequestPropertySensor remote: INFO: Sensor com.github.mc1arke.sonarqube.plugin.scanner.ScannerPullRequestPropertySensor (done) | time2ms remote: INFO: Sensor IaC Docker Sensor [iac] remote: INFO: 0 source files to be analyzed remote: INFO: 0/0 source files have been analyzed remote: INFO: Sensor IaC Docker Sensor [iac] (done) | time149ms remote: INFO: ------------- Run sensors on project remote: INFO: Sensor Analysis Warnings import [csharp] remote: INFO: Sensor Analysis Warnings import [csharp] (done) | time3ms remote: INFO: Sensor Zero Coverage Sensor remote: INFO: Sensor Zero Coverage Sensor (done) | time17ms remote: INFO: Sensor Java CPD Block Indexer remote: INFO: Sensor Java CPD Block Indexer (done) | time38ms remote: INFO: SCM Publisher is disabled remote: INFO: CPD Executor 3 files had no CPD blocks remote: INFO: CPD Executor Calculating CPD for 0 files remote: INFO: CPD Executor CPD calculation finished (done) | time0ms remote: INFO: Analysis report generated in 137ms, dir size97.8 kB remote: INFO: Analysis report compressed in 55ms, zip size16.8 kB remote: INFO: Analysis report uploaded in 44ms remote: INFO: ANALYSIS SUCCESSFUL, you can find the results at: http://47.120.3.240:9000/dashboard?idsonar-testbranchmaster remote: INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report remote: INFO: More about the report processing at http://你的sonarqube:9000/api/ce/task?idAY52Q8sMVWuDRjAxy0tr remote: INFO: Analysis total time: 9.598 s remote: INFO: ------------------------------------------------------------------------ remote: INFO: EXECUTION SUCCESS remote: INFO: ------------------------------------------------------------------------ remote: INFO: Total time: 12.726s remote: INFO: Final Memory: 19M/70M remote: INFO: ------------------------------------------------------------------------ remote: % Total % Received % Xferd Average Speed Time Time Time Current remote: Dload Upload Total Spent Left Speed remote: remote: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 remote: 100 370 0 370 0 0 14036 0 --:--:-- --:--:-- --:--:-- 14230 remote: 代码检测结果ERROR remote: 删除临时文件夹 remote: 代码检测失败拒绝提交 error: failed to push some refs to http://你的 sonarqube IP/macrosoft/sonar-test.git To http://47.119.174.163/macrosoft/sonar-test.git ! refs/heads/master:refs/heads/master [remote rejected] (pre-receive hook declined) Done 在 sonarqube 对应分支就能看到检测失败而且报错内容就是我们代码写的内容。
