Implementing Unit Testing

2019-02-03 Technical and Programming

When developing any dynamic functionality it is best practice to implement testing. This will ensure anytime the code is deployed it is in a working state and will provide a good user experience to the visitors/users of your service.

# Pre-requisites

This article will be based on the blog described in Hexo, AWS and Serverless Framework,

Securing A Test Environment Using AWS WAF and Securing S3 Bucket from Direct Access; however, the basic concepts

should be applicable to any NodeJS functionality or any static site.

# Install a test framework for Javascript

Although it’s possible to write a custom test framework, it’s much easier to use an existing solution. This example will use Jest. Start by installing Jest with NPM.

1
npm i jest --save-dev

# Create a test

The next step is to create a test. As the example project only has one JavaScript function these instructions will only implement a test for it. There is no reason many more tests can’t be created. To make it easy to find tests the new test will be created in tests/functions/urlRewrite.test.js.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// Load the file to test
const urlRewrite = require('../../functions/urlRewrite');
// Load some data that can be reused for other lambda@Edge functions
const lambdaAtEdgeFixture = require('../fixtures/lambdaAtEdge');

test(
    'url-rewrite handler appends index.html to root request',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.root_object.Records[0].cf.request.uri + 'index.html' };
        expect(
            urlRewrite.handler(
                lambdaAtEdgeFixture.event.root_object,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

test(
    'url-rewrite handler appends index.html to directory requests',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.subdirectory.Records[0].cf.request.uri + 'index.html' };
        expect(
            urlRewrite.handler(
                lambdaAtEdgeFixture.event.subdirectory,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

test(
    'url-rewrite handler does not append index.html to root-directory requests',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.file.Records[0].cf.request.uri };
        expect(
            urlRewrite.handler(
                lambdaAtEdgeFixture.event.file,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

test(
    'url-rewrite handler does not append index.html to non-directory requests',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.file_in_subdirectory.Records[0].cf.request.uri };
        expect(
            urlRewrite.handler(
                lambdaAtEdgeFixture.event.file_in_subdirectory,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

The 4th line of the test includes a file that doesn’t exist yet. This file contains some sample values that can be reused if we need to test other Lambda@Edge functions. Create the file at tests/fixtures/lambdaAtEdge.js (this is a large file so the content is at the bottom of this post).

Test that the test suite works.

1
jest

Now that the javascript is tested, it’s worth testing the links on the static site to make sure they are valid. For this broken-link-checker-local can do all the hard work, so install it.

1
npm i broken-link-checker-local --save-dev

As long as the site has been deployed previously (it has been if you’re following all the blog posts) it’s possible to test for broken links.

1
blcl --filter-level 3 --get --recursive public

# Add NPM scripts to test more easily

Remembering the right commands, especially when managing multiple sites, can be confusing and easy to forget. To avoid this, it’s possible to define some scripts in the pacakge.json file. The commands to add are as follows:

  • clean: a simple wrapper for hexo clean
  • build: a wrapper for hexo generate
  • buildclean: a wrapper for both clean and build
  • linkcheck: run the link checking script
  • jest: run the jest tests
  • test: run an end-to-end test
1
2
3
4
5
6
7
8
"scripts": {
  "clean": "hexo clean",
  "build": "hexo generate",
  "cleanbuild": "npm run clean && npm run build",
  "linkcheck": "blcl --filter-level 3 --get --recursive public",
  "jest": "jest",
  "test": "npm run cleanbuild && npm run jest && npm run linkcheck"
}

# Ignore static site files when running Jest

Because the static site can contain files that Jest will assume are tests a configuration needs to be added to package.json to ignore these files.

1
2
3
4
5
6
7
"jest": {
  "testPathIgnorePatterns": [
    "/node_modules/",
    "/source/code/",
    "/public/"
  ]
}

# Run the full test suite

Running a full test is now as simple as using NPM to run the test command.

1
npm run test

Based on the posts undertaken in the examples the following links will show as broken in a number of files:

  • atom.xml
  • favicon.png

# The Final Configuration Files

test/functions/urlRewrite.test.jsview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// Load the file to test
const urlRewriteTest = require('../../functions/urlRewrite');
// Load some data that can be reused for other lambda@Edge functions
const lambdaAtEdgeFixture = require('../fixtures/lambdaAtEdge');

test(
    'url-rewrite handler appends index.html to root request',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.root_object.Records[0].cf.request.uri + 'index.html' };
        expect(
            urlRewriteTest.handler(
                lambdaAtEdgeFixture.event.root_object,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

test(
    'url-rewrite handler appends index.html to directory requests',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.subdirectory.Records[0].cf.request.uri + 'index.html' };
        expect(
            urlRewriteTest.handler(
                lambdaAtEdgeFixture.event.subdirectory,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

test(
    'url-rewrite handler does not append index.html to root-directory requests',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.file.Records[0].cf.request.uri };
        expect(
            urlRewriteTest.handler(
                lambdaAtEdgeFixture.event.file,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);

test(
    'url-rewrite handler does not append index.html to non-directory requests',
    () => {
        const expectedResponse = { uri: lambdaAtEdgeFixture.event.file_in_subdirectory.Records[0].cf.request.uri };
        expect(
            urlRewriteTest.handler(
                lambdaAtEdgeFixture.event.file_in_subdirectory,
                lambdaAtEdgeFixture.context.webClient,
                lambdaAtEdgeFixture.callback
            )
        ).toEqual(expect.objectContaining(expectedResponse));
    }
);
package.jsonview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
{
  "name": "hexo-site",
  "version": "0.0.0",
  "private": true,
  "hexo": {
    "version": "3.8.0"
  },
  "devDependencies": {
    "@silvermine/serverless-plugin-cloudfront-lambda-edge": "^2.1.1",
    "broken-link-checker-local": "^0.2.0",
    "hexo": "^3.7.0",
    "hexo-generator-archive": "^0.1.5",
    "hexo-generator-category": "^0.1.3",
    "hexo-generator-index": "^0.2.1",
    "hexo-generator-tag": "^0.2.0",
    "hexo-renderer-ejs": "^0.3.1",
    "hexo-renderer-marked": "^0.3.2",
    "hexo-renderer-stylus": "^0.3.3",
    "hexo-server": "^0.3.1",
    "jest": "^24.0.0",
    "serverless-lambda-version": "^0.1.2",
    "serverless-plugin-scripts": "^1.0.2",
    "serverless-s3-deploy": "^0.8.0"
  },
  "scripts": {
    "clean": "hexo clean",
    "build": "hexo generate",
    "cleanbuild": "npm run clean && npm run build",
    "linkcheck": "blcl --filter-level 3 --get --recursive --exclude http://2019-01-28-securing-s3.demo.alphageek.com.au.s3-website-us-east-1.amazonaws.com --exclude http://localhost:4000 --exclude http://dev./%3Cyour_domain%3E/ public",
    "jest": "jest",
    "test": "npm run cleanbuild && npm run jest && npm run linkcheck"
  },
  "jest": {
    "testPathIgnorePatterns": [
      "/node_modules/",
      "/source/code/",
      "/public/"
    ]
  }
}
serverless.ymlview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
# The name of your project
service: **project**

# Plugins for additional Serverless functionality
plugins:
  - serverless-s3-deploy
  - serverless-plugin-scripts
  - '@silvermine/serverless-plugin-cloudfront-lambda-edge'

# Configuration for AWS
provider:
  name: aws
  runtime: nodejs8.10
  profile: serverless
  # Some future functionality requires us to use us-east-1 at this time
  region: us-east-1

  # This enables us to use the default stage definition, but override it from the command line
  stage: ${opt:stage, self:provider.stage}
  # This enables us to prepend the stage name for non-production environments
  domain:
    fulldomain:
      prod: ${self:custom.domain.domain}
      other: ${self:custom.stage}.${self:custom.domain.domain}
    # This value has been customised so I can maintain multiple demonstration sites
    domain: ${self:custom.postname}.${self:custom.domain.zonename}
    domainname: ${self:custom.domain.fulldomain.${self:custom.stage}, self:custom.domain.fulldomain.other}
    # DNS Zone name (this is only required so I can maintain multiple demonstration sites)
    zonename: alphageek.com.au
    cacheControlMaxAgeHTMLByStage:
      # HTML Cache time for production environment
      prod: 3600
      # HTML Cache time for other environments
      other: 0
    cacheControlMaxAgeHTML: ${self:custom.domain.cacheControlMaxAgeHTMLByStage.${self:custom.stage}, self:custom.domain.cacheControlMaxAgeHTMLByStage.other}
    sslCertificateARN: arn:aws:acm:us-east-1:165657443288:certificate/61d202ea-12f2-4282-b602-9c3b83183c7a
  assets:
    targets:
      # Configuration for HTML files (overriding the default cache control age)
      - bucket:
          Ref: WebsiteS3Bucket
        acl: private
        files:
          - source: ./public/
            headers:
              CacheControl: max-age=${self:custom.domain.cacheControlMaxAgeHTML}
            empty: true
            globs:
              - '**/*.html'
      # Configuration for all assets
      - bucket:
          Ref: WebsiteS3Bucket
        acl: private
        files:
          - source: ./public/
            empty: true
            globs:
              - '**/*.js'
              - '**/*.css'
              - '**/*.jpg'
              - '**/*.png'
              - '**/*.gif'
  scripts:
    hooks:
      # Run these commands when creating the deployment artifacts
      package:createDeploymentArtifacts: >
        hexo clean &&
        hexo generate
      # Run these commands after infrastructure changes have been completed
      deploy:finalize: >
        sls s3deploy -s ${self:custom.stage}
  # AWS Region to S3 website hostname mapping
  s3DNSName:
    us-east-2: s3-website.us-east-2.amazonaws.com
    us-east-1: s3-website-us-east-1.amazonaws.com
    us-west-1: s3-website-us-west-1.amazonaws.com
    us-west-2: s3-website-us-west-2.amazonaws.com
    ap-south-1: s3-website.ap-south-1.amazonaws.com
    ap-northeast-3: s3-website.ap-northeast-3.amazonaws.com
    ap-northeast-2: s3-website.ap-northeast-2.amazonaws.com
    ap-southeast-1: s3-website-ap-southeast-1.amazonaws.com
    ap-southeast-2: s3-website-ap-southeast-2.amazonaws.com
    ap-northeast-1: s3-website-ap-northeast-1.amazonaws.com
    ca-central-1: s3-website.ca-central-1.amazonaws.com
    eu-central-1: s3-website.eu-central-1.amazonaws.com
    eu-west-1: s3-website-eu-west-1.amazonaws.com
    eu-west-2: s3-website.eu-west-2.amazonaws.com
    eu-west-3: s3-website.eu-west-3.amazonaws.com
    eu-north-1: s3-website.eu-north-1.amazonaws.com
    sa-east-1: s3-website-sa-east-1.amazonaws.com
  # Determine what resources file to include based on the current stage
  customConfigFile: ${self:custom.customConfigFiles.${self:custom.stage}, self:custom.customConfigFiles.other}
  customConfigFiles:
    prod: prod
    other: other

# Define the Lambda functions for the site
functions:
  # This function will be deployed to Lambda@Edge and rewrite URLs to include index.html
  urlrewrite:
    name: ${self:service}-${self:custom.stage}-cf-url-rewriter
    handler: functions/urlRewrite.handler
    memorySize: 128
    timeout: 1
    lambdaAtEdge:
      distribution: WebsiteCloudFrontDistribution
      eventType: origin-request

# Define the resources we will need to host the site
resources:
  # Include the resources file
  - ${file(config/resources.yml)}
  # Include the outputs file
  - ${file(config/outputs.yml)}
  # Include a custom configuration file based on the environment
  - ${file(config/resources/environment/${self:custom.customConfigFile}.yml)}
config/resources.ymlview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
Resources:
  # Set-up an S3 bucket to store the site
  WebsiteS3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: BucketOwnerFullControl
      BucketName: ${self:custom.domain.domainname}
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
  # Set-up a policy on the bucket so it can be used as a website
  WebsiteBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      PolicyDocument:
        Id:
          Fn::Join:
            - ""
            - - ${self:service.name}
              - BucketPolicy
        Statement:
          - Sid: CloudFrontForGetBucketObjects
            Effect: Allow
            Principal:
              CanonicalUser:
                Fn::GetAtt:
                  - CloudFrontIdentity
                  - S3CanonicalUserId
            Action: 's3:GetObject'
            Resource:
              Fn::Join:
                - ''
                -
                  - 'arn:aws:s3:::'
                  - Ref: WebsiteS3Bucket
                  - /*
      Bucket:
        Ref: WebsiteS3Bucket
  # Configure CloudFront to get all content from S3
  WebsiteCloudFrontDistribution:
    Type: 'AWS::CloudFront::Distribution'
    Properties:
      DistributionConfig:
        WebACLId:
          Ref: CustomAuthorizationHeaderRestriction
        Aliases:
          - ${self:custom.domain.domainname}
          - www.${self:custom.domain.domainname}
        CustomErrorResponses:
          - ErrorCode: '404'
            ResponsePagePath: "/error.html"
            ResponseCode: '200'
            ErrorCachingMinTTL: '30'
        DefaultCacheBehavior:
          Compress: true
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: all
          SmoothStreaming: false
          TargetOriginId: defaultOrigin
          ViewerProtocolPolicy: redirect-to-https
        DefaultRootObject: index.html
        Enabled: true
        Origins:
          - DomainName:
              Fn::GetAtt:
                - WebsiteS3Bucket
                - DomainName
            Id: defaultOrigin
            S3OriginConfig:
              OriginAccessIdentity:
                Fn::Join:
                  - "/"
                  - - origin-access-identity
                    - cloudfront
                    - Ref: CloudFrontIdentity
        PriceClass: PriceClass_All
        ViewerCertificate:
          AcmCertificateArn: ${self:custom.domain.sslCertificateARN}
          SslSupportMethod: sni-only
  # DNS Record for the domain
  WebsiteDNSRecord:
    Type: "AWS::Route53::RecordSet"
    Properties:
      AliasTarget:
        DNSName:
          Fn::GetAtt:
            - WebsiteCloudFrontDistribution
            - DomainName
        HostedZoneId: Z2FDTNDATAQYW2
      HostedZoneName: ${self:custom.domain.domain}.
      Name: ${self:custom.domain.domainname}
      Type: 'A'
  # DNS Record for www.domain
  WebsiteWWWDNSRecord:
    Type: "AWS::Route53::RecordSet"
    Properties:
      AliasTarget:
        DNSName:
          Fn::GetAtt:
            - WebsiteCloudFrontDistribution
            - DomainName
        HostedZoneId: Z2FDTNDATAQYW2
      HostedZoneName: ${self:custom.domain.domain}.
      Name: www.${self:custom.domain.domainname}
      Type: 'A'
  # Predicate to match the authorization header
  CustomAuthorizationHeader:
    Type: AWS::WAF::ByteMatchSet
    Properties:
      ByteMatchTuples:
        -
          FieldToMatch:
            Type: HEADER
            Data: Authorization
          TargetString:
            Fn::Join:
              - " "
              - - Custom
                - "**Password**"
          TextTransformation: NONE
          PositionalConstraint: EXACTLY
      Name:
        Fn::Join:
          - "_"
          - - ${self:custom.domain.domainname}
            - Authorization
            - Header
  CustomAuthorizationHeaderRule:
    Type: AWS::WAF::Rule
    Properties:
      Name:
        Fn::Join:
          - "_"
          - - ${self:custom.domain.domainname}
            - Authorization
            - Header
            - Rule
      MetricName:
        Fn::Join:
          - ""
          - - ${self:custom.stage}
            - ${self:service.name}
            - Authorization
            - Header
            - Rule
      Predicates:
        -
          DataId:
            Ref: CustomAuthorizationHeader
          Negated: false
          Type: ByteMatch
config/outputs.ymlview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Outputs:
  WebsiteURL:
    Value:
      Fn::GetAtt:
        - WebsiteS3Bucket
        - WebsiteURL
    Description: URL for my website hosted on S3
  S3BucketSecureURL:
    Value:
      Fn::Join:
        - ''
        -
          - 'https://'
          - Fn::GetAtt:
              - WebsiteS3Bucket
              - DomainName
    Description: Secure URL of S3 bucket to hold website content
config/prod.ymlview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Resources:
  # Allow the custom authorisation header in the production environment
  CustomAuthorizationHeaderRestriction:
    Type: AWS::WAF::WebACL
    Properties:
      DefaultAction:
        Type: ALLOW
      Name:
        Fn::Join:
          - "_"
          - - ${self:custom.domain.domainname}
            - Authorization
            - Header
            - Restriction
      MetricName:
        Fn::Join:
          - ""
          - - ${self:custom.stage}
            - ${self:service.name}
            - Authorization
            - Header
            - Restriction
      Rules:
        -
          Action:
            Type: ALLOW
          Priority: 1
          RuleId:
            Ref: CustomAuthorizationHeaderRule
config/other.ymlview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Resources:
  # Require the custom authorisation header with the correct password in non-production environment
  CustomAuthorizationHeaderRestriction:
    Type: AWS::WAF::WebACL
    Properties:
      DefaultAction:
        Type: BLOCK
      Name:
        Fn::Join:
          - "_"
          - - ${self:custom.domain.domainname}
            - Authorization
            - Header
            - Restriction
      MetricName:
        Fn::Join:
          - ""
          - - ${self:custom.stage}
            - ${self:service.name}
            - Authorization
            - Header
            - Restriction
      Rules:
        -
          Action: ALLOW
          Priority: 1
          RuleId:
            Ref: CustomAuthorizationHeaderRule

# Example Site