To create contract tests for all methods of an endpoint, specify the endpoint you want to test. In this example, we are using https://demoshop.skyramp.dev as the base URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.

You can find the used API specification here.

skyramp generate contract rest https://demoshop.skyramp.dev/api/v1/products \
--language python \
--framework pytest \
--api-schema

This command generates a fully executable Python file (products_contract_test.py). The contents of the generated test can be found here.

Explanation of Command

• NOTE: No API method is specified in this generation command. When paired with an OpenAPI spec, Skyramp will generate tests for all endpoints at the URL, its direct parent (if any), and its direct children.

• https://demoshop.skyramp.dev/api/v1/products: Defines the URL to the endpoint we aim to test.

• --language: Specifies the test output language. For contract testing, we currently support Python, Typescript, and Java.

• --framework: Specifies the test execution framework of choice.

• --api-schema: Points to the OpenAPI schema used to generate the test.

Adjustments

These flags will help you tune the contract test. Additional flags are explained here.

• --asserts: Specify how many assertions Skyramp should generate on the response body for the contract test. Skyramp evaluates the endpoint to determine what values are expected, and then the test checks to see whether they are returned. You can find out more information about asserts here (Example 4).

  • Specifying all will return an assertion for every API response key available in the endpoint.

  • Specifying none will assert response code and check schema but will not return any body value assertions.

  • If this flag is not specified, Skyramp generates a test that checks the API schema and generates assertions for the HTTP response code and the first three response body values.

• --response-status-code: Specify the expected status code. By default, Skyramp either asserts against the defined status code in the API specification or defaults to 20X.

• --output: Specify the name of the generated test file.

• --output-dir: Specify the directory to store the generated test file.

Execute the Contract Test

You can execute the generated tests without any additional adjustments to the code. However, based on the application you want to test, you can pass your authentication token to Skyramp Tests via an environment variable.

Set environment variable for authentication

To test against an application that requires authentication, pass your token using our environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods (Example 3). If your API does not require any authentication, you can skip this step and just run the test.

Skyramp’s sample application requires a session identifier. Quickly obtain your session_id via the Demo Shop UI and set it as your authentication header. Additional information and ways to obtain the session_id can be found here.

export SKYRAMP_TEST_TOKEN=$your_auth_token

Run the Test

Run the tests using Pytest. If you don’t have Pytest, refer to the Installation Guide for setup instructions:

python3 -m

Review Test Results

We are using Pytest’s shortened traceback (--tb=short) output in this guide, printing a line for each test that is being run and listing all failures at the end. You can adjust the output behavior following this documentation.

This test consists of the following 5 methods:

• test_products_post - creates a new product

• test_products_get - fetches a list of products

• test_products_product_id_get - fetches a specific product (requires path parameter)

• test_products_product_id_put - updates a specific product (requires path parameter)

• test_products_product_id_delete - deletes a specific product (requires path parameter)

For the methods that require a path parameter, Skyramp tries to identify a relevant path parameter from the example section in your OpenAPI specification. If no examples exist in the spec, Skyramp uses the value 0 by default.

Test failure

In this instance, we can see that only 2 out of the 5 test cases pass. This is because the failing endpoints are attempting to read/update/delete a product ID 0 that does not exist in the application’s database.

Successful test

To ensure that all functions of the contract test succeed, you simply need to update the product_id path parameter. Skyramp generates parameterized test code, allowing you to quickly update the value at the top of each function that requires a path parameter.

In our Demo Shop, product_id=1 exists by default. If product_id=1 was deleted due to some prior action, you can reset the database as described here.

For our Demo Shop, update the path parameter for the following functions with product_id=1:

• test_products_product_id_get - line 112

• test_products_product_id_put - line 149

• test_products_product_id_delete - line 197

Now, if you re-run the test, all 5 test cases will pass:

Next Steps

Congratulations! You have successfully generated a contract test for all methods of a REST endpoint. Learn more about how you can adjust the test to fit your needs on the Test File Anatomy page.

Related topics

• Contract Test File Anatomy

• Skyramp Demo Shop

• Flags Glossary