skyramp generate fuzz rest https://demoshop.skyramp.dev/api/v1/products \
--language python \
--framework pytest \
--api-schema

This command generates a fully executable Python file (products_fuzz_test.py). The contents of the generated test can be found here.

Explanation of Command

• https://demoshop.skyramp.dev/api/v1/products: Defines the URL to the endpoint we aim to test.

• NOTE: No API method is specified in this generation command. When paired with an OpenAPI spec, Skyramp will generate tests for all endpoints at the URL, its direct parent (if any), and its direct children.

• --language: Specifies the test output language. For fuzz testing, we currently support Python and TypeScript.

• --framework: Specify the test execution framework of choice.

• --api-schema: Points to the OpenAPI schema used to generate the test. We also support sample data as an input for fuzz test generation.

Adjustments

These flags will help you tune the fuzz test. Additional flags are explained here.

• --auth-header: This flag allows you to specify the key of your authentication header, e.g. --auth-header X-API-KEY. By default, we assume Bearer.

• --response-status-code: Specify the expected status code. For fuzz tests, we default to 40x.

• --path-params: This flag allows you to override path parameters from your endpoint URL or the pre-defined values in the API schema, e.g. --path-params id=3fa85f64-5717-4562-b3fc-2c963f66afa6

• --output: Specify the name of the generated test file.

• --output-dir: Specify the directory to store the generated test file in.

Execute the Fuzz Test

You can execute the generated tests without any additional adjustments to the code. However, based on the application you want to test, you can pass your authentication token to Skyramp Tests via an environment variable. In a later section, we will elaborate on how to make changes to the code, if needed.

Set environment variable for authentication

To test against an application that requires authentication, pass your token using our environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods (Example 3). If your API does not require any authentication, you can skip this step and just run the test.

Skyramp’s sample application requires a session identifier. Quickly obtain your session_id via the Demo Shop UI and set it as your authentication header. Additional information and ways to obtain the session_id can be found here.

export SKYRAMP_TEST_TOKEN=$your_auth_token

Run the Test

Run the test using Pytest. If you don’t have Pytest, refer to the Installation Guide for setup instructions:

 python3 -m pytest products_fuzz_test.py --tb

Review Test Results

We are using Pytest’s shortened traceback output (--tb=short) in this guide, printing a line for each test that is being run and listing all failures at the end. You can adjust the output behavior by following this documentation.

This test consists of the following 5 methods:

• test_products_post - creates a new product

• test_products_get - fetches a list of products

• test_products_product_id_get - fetches a specific product (requires path parameter)

• test_products_product_id_put - updates a specific product (requires path parameter)

• test_products_product_id_delete - deletes a specific product (requires path parameter)

For the methods that require a path parameter, Skyramp tries to identify a relevant path parameter from the example section in your OpenAPI specification. If no examples exist in the spec, Skyramp uses the value 0 by default.

Test failure

In this instance, we can see that only 1 out of the 5 test cases pass.

Successful test

To fix the test, we need to update the expected status codes for each request body tested.

Here is the fix for each test:

• test_products_post

  • Problem: Python evaluates True fuzzed value to true

  • Solution: Update the fuzzed expected response code body for in_stock field (line 132) from 40x to 20x

    expected_products_post_status_code = {
            "category": "40x",
            "description": "40x",
            "image_url": "40x",
            "in_stock": "20x",
            "name": "40x",
            "price": "40x"
    
    

  • test_products_product_id_get, test_products_product_id_put, test_products_product_id_delete

    • Problem: Product 0 does not exist in Demo Shop

    • Solution: Change the expected_code of the original non-fuzzed request (lines 194, 254, 304) to 40x

      products_product_id_PUT_response = client.send_request(
              url=URL,
              path="/api/v1/products/{product_id}",
              method="PUT",
              body=products_product_id_PUT_request_body,
              headers=headers,
              path_params={"product_id": product_id},
              expected_code="40x"
      
      

Adding these fixes will cause the fuzz test to pass.

Next Steps

Congratulations! You have successfully generated a fuzz test for all methods of a REST endpoint. Learn more about how you can adjust the test to fit your needs on the Test File Anatomy page.

Related topics

• Test File Anatomy

• Skyramp Demo Shop

• Flags Glossary