Sanity

“Any daily work task that takes 5 minutes will cost over 20 hours a year, or over half of a work week. Even if it takes 20 hours to automate that daily 5 minute task, the automation will break even in a year.” -- Anthony J. Stieber, Breaking into Information Security

“If we start being honest about our pain, our anger, and our shortcomings instead of pretending they don’t exist, then maybe we’ll leave the world a better place than we found it.” -– Russell Wilson, NFL Star

Taking care for a running online application requires repetitive, dull little tasks. Check the web server response. See if certificates will expire (or even worse: Already did.) Check the server time. It adds up for one application, and we certainly have more than one. Sanity runs sanity checks on targets you define. It's meant to be run by CI pipelines and the like, producing output like a unit test suite.

Implemented checks

• Bitbucket sanity
• Bamboo sanity
• Static web response sanity
• SSL sanity
• DNS sanity

Test code is in lib/sanity/tests for reference and details.

Results are written to files, one per application in the targets definition. Result files are created in path ./run/**

Available output formats

Output formats can be activated using the --outputs command line parameter. Multiple values may be separated by comma.

Name	Parameter	Description
Console	console	Test result written to the console, colorized if available
Junit	junit	Results as JUnit-compatible XML files, one file per suite
Report	report	HTML report file, containing all suites
SslCalendar	ssl_calendar	Expiration dates of SSL tests, if any. .ics files, one per suite

Defaults are defined in Sanity::Options::DEFAULTS.

Example target file

Create a file targets.json in the base directory of this repo. It may contain multiple apps, and for each app any combination of supported tests along with any number of targets (at least one, though).

{
  "app_1": {
    "Bitbucket": ["CODE/slug1", "CODE/slug2", "CODE/slug3"],
    "Bamboo": ["PROJECTKEY-PLANKEY"],
    "StaticWeb": ["https://software-berater.net/assets/css/main.css"],
    "SSL": ["https://software-berater.net/"],
    "DNS": ["www.software-berater.net"]
  }
}

Target configuration (optional)

Target definition may contain configuration, if needed and supported by the test.

{
  "app_1": {
    // complex configuration
    "StaticWeb": {
      "config": { "key": "value" },
      "targets": ["https://software-berater.net/assets/css/main.css"]
    },
    // simple configuration
    "SSL": ["https://software-berater.net/"]
  }
}

Include/exclude tests per suite

To limit the list of tests applied to each target, a config element may include such data:

"config": {
  "include": ["array", "of", "tests", "to", "run"]
}

If include is given, only the given test methods are executed. This is a per-suite setting.

"config": {
  "exclude": ["array", "of", "excluded", "tests"]
}

If exclude is given, the given test methods are not executed. This is a per-suite setting.

Secrets handling

Bamboo and Bitbucket sanity checks need a user token for access authentication. Create a file .env from the template env.sample.txt and fill in the token values.

Execution

To run the docker image as provided, use this sequence:

1. Create an empty directory
2. Create .env file
3. Create a file targets.json there (see below for examples)
4. Run docker run -v $(pwd):/app sanity:latest
5. Test results will be in directory ./run.

Or run locally. Usage: sanity [options] TARGET [TARGET...]

1. Create .env file
2. Create a file targets.json (see below for examples)
3. Run exe/sanity targets.json
4. Test results will be in directory ./run.

Multiple targets

Multiple target definitions may be given on the command line. Ruby glob patterns are supported. Target definitions will me merged in order of precendence, later definitions will override earlier ones.

Developer information

The software was written using the Ruby programming language. The project uses standard Ruby technology like RSpec and Bundler, and features RVM-integration to ease development.

Initialization flow

Unit tests

Run unit test by executing rake. Any other option provided by RSpec also works.

Create Docker image

Create a docker image locally by issuing docker build -t sanity .

How to release a new version

1. Run rake spec locally to make sure all tests succeed.
2. Increment version in semver.txt
3. Run bundle install to incorporate new version number into Gemfile.lock
4. Commit both semver and Gemfile changes to the develop branch
5. Create a pull request (PR) from develop to master.
6. After CI build succeeds, merge the PR without deleting the develop branch.
7. Reference new version in sanity repos using this image, eg SSENG/infra-sanity.