Continuous Security with Jenkins and Amazon Inspector

Amazon Inspector
Image by Amazon Web Services, Inc. ©

Amazon Inspector is an automated security assessment service on Amazon Web Services (AWS). It allows to identify security vulnerabilities at operating system and network levels by scanning the host against a knowledge base of security best practices and rules.

I recently integrated Amazon Inspector to run in a Jenkins job so that security testing can be automated and performed prior to deployment to production.

AWS Configuration

The first thing to do is to set up the assessment target and assessment template in Amazon Inspector. An assessment target allows to select the EC2 instances via their tags in order to include them in the security scan. Here is an example of my assessment target for the EC2 instances tagged as gadictionaries-leap-ogl-stage-v360 :

Assessment TargetThe assessment template allows to specify the type of scan and its duration and is linked to the assessment target set up above. Here is an example of my assessment template (ARN is masked for security reasons). I selected the Common Vulnerabilities and Exposures (CVE) rule package scanning for 15 minutes (one hour is the recommended duration time to reliable results).

Assessment Template

Jenkins Configuration

We now move to the Jenkins configuration in order to run the security scan via a Jenkins job instead of using the AWS console.

The first thing to do is to make sure that openssh is installed on the instance where Jenkins is running and on the host you want to check. For example, on Ubuntu you can install openssh with:

Then install the SSH Agent plugin in Jenkins. This will provide Jenkins with SSH credentials to automatically login into a machine on the cloud. Add the credentials in Jenkins -> Credentials -> System -> Global credentials (unrestricted) -> Add credentials -> SSH Username with private key. This is an example of my credentials for user jenkins (private key details are obfuscated):

set up mock SSH credentialsThen create a Jenkins job and select the SSH agent credentials for user jenkins in Build Environment:

SSH agent credentialsThis will allow Jenkins to ssh into the machine with the private key stored securely (make sure you only grant permission to configure Jenkins to administrators otherwise your private keys are not safe).

I like to parameterize my builds so that I can run Amazon Inspector on a specific EC2 instance within a given Elastic Beanstalk stack:

Parameters

Then we set up build and post-build actions. The build executes a shell script invoke_aws_inspector.sh pulled from the version control system. The post-build action provides the location of the JUnit file.

The shell script invoke_aws_inspector.sh looks like this:

The shell script works as follows:

  • line 4 allows Jenkins to ssh into a host (I’m using AWS EC2 as you can guess by the username ec2-user, replace it with your default username but do not user root). Note that the environment variable $HOSTNAME is passed from the parameter we set up earlier. The EOF allows to run a sequence of commands directly on the host so that you don’t have to disconnect every time. The single quotes are important, don’t skip them!
  • lines 5-8 install and start the Amazon Inspector agent on the host
  • lines 12-13 configure and set up a Python script  execute_aws_inspector.py  for running Amazon Inspector (we’ll see it in a minute)
  • lines 16-18 remove the Amazon Inspector agent so that no trace is left on the host
  • the final EOF disconnect Jenkins from host

The Python script execute_aws_inspector.py uses the Boto3 library for interacting with AWS services. The script looks like this:

The Python script works as follows:

  • lines 10-17 read the environment variable set in the parameterized build and select the correct template (I set up two different template for two different stacks, the ARNs are obfuscated for security reasons)
  • lines 20-26 run the assessment template and waits a bit longer than 15 minutes so that the scan can finish
  • lines 29-39 filters findings with severities High and Medium
  • lines 42-58 serialize the findings into a JUnit report so that they can be automatically read by Jenkins

Finally, here is an example of Test Result Trend and JUnit test results showing security vulnerabilities on an EC2 instance running unpatched packages:

JUnit Report

Happy security testing with Jenkins and Amazon Inspector!

Leave a Reply

Your email address will not be published. Required fields are marked *