How can I improve jenkins performance with codebuild builds?

by Felipe   Last Updated July 12, 2019 08:26 AM

Our Jenkins is setup in aws and we did not manage to use slaves. Since the platform is big and some artifacts contain many others, our jenkins comes to his limits when multiple developers commit to different repositories and it is forced to run multiple jobs at the same time.

The aim is to: - Stay with jenkins since our processes are documented based on it and we use many plugins e.g. test result summary and github integration - Run jobs in codebuild and get feedback in jenkins to improve the performance

Are there best practices for this?



Answers 1


We did the following steps to build big artifacts outside of jenkins: - Install jenkins codebuild plugin - Create jenkins pipeline - Store settings.xml for maven build in s3 - Store access in system manager parameters to use in codebuild and maven

  • Create codebuild project with the necessary permissions and following functionality: -- Get settings.xml from s3 -- run maven with the necessary access data -- store tests results in s3

  • Create jenkinsfile whith following functionality: -- get commitID and run codebuild with it -- get generated files of test results from s3 and pass it to jenkins -- delete generated files from s3 -- pass files to jenkins to show test results

With this approach we managed to reduce the runtime to 5 mins.

We next challenge we had was to build and angular application on top of a java microservice, create a docker image and push it to different environments. This jobs was running around 25 mins in jenkins.

We did the following steps to build the docker images outside of jenkins: - Install jenkins codebuild plugin - Create jenkins pipeline - Store settings.xml for maven build in s3 - Store access in system manager parameters to use in codebuild and maven

  • Create codebuild project with the necessary permissions and following functionality: -- Get settings.xml from s3 -- login into ecr in all environments -- build the angular app -- build the java app -- copy necessary files for docker build -- build docker image -- push to all envoronments

  • Create jenkinsfile whith following functionality: -- get branch names of both repositories to build the docker image from -- get branch latest commitID -- call the codebuild projects with both commitIDs (notice that the main repository will need the buildspec)

With this approach we managed to reduce the runtime to 5 mins.

Sample code in: https://github.com/felipeloha/samples/tree/master/jenkins-codebuild

Felipe
Felipe
July 12, 2019 08:25 AM

Related Questions


Updated July 20, 2019 04:26 AM

Updated December 08, 2017 16:26 PM

Updated January 30, 2018 22:26 PM

Updated February 18, 2019 16:26 PM