Generation

Getting Started

Getting Started

Smoke Test

Load Testing

This guide will walk you through generating load tests with the Skyramp CLI.

1. Overview

Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.

Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.

2. Generate Load Test for REST APIs

This section explains how you can use Skyramp to generate load tests for REST APIs. You can generate load tests for:

  • a specific method of an endpoint

To reliably generate test cases, provide at least one of the following inputs:

  • OpenAPI schema (JSON or YAML file)

  • Sample request data (JSON blob or JSON file)

While we focus on using an API schema as input for this guide, you can find detailed explanations on achieving test generation from sample data further down - this allows you to have more control over the generated body values.

Load Testing

This guide will walk you through generating load tests with the Skyramp CLI.

1. Overview

Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.

Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.

2. Generate Load Test for REST APIs

This section explains how you can use Skyramp to generate load tests for REST APIs. You can generate load tests for:

  • a specific method of an endpoint

To reliably generate test cases, provide at least one of the following inputs:

  • OpenAPI schema (JSON or YAML file)

  • Sample request data (JSON blob or JSON file)

While we focus on using an API schema as input for this guide, you can find detailed explanations on achieving test generation from sample data further down - this allows you to have more control over the generated body values.

Load Testing

This guide will walk you through generating load tests with the Skyramp CLI.

1. Overview

Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.

Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.

2. Generate Load Test for REST APIs

This section explains how you can use Skyramp to generate load tests for REST APIs. You can generate load tests for:

  • a specific method of an endpoint

To reliably generate test cases, provide at least one of the following inputs:

  • OpenAPI schema (JSON or YAML file)

  • Sample request data (JSON blob or JSON file)

While we focus on using an API schema as input for this guide, you can find detailed explanations on achieving test generation from sample data further down - this allows you to have more control over the generated body values.

Python

2.1 Single Method

To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.

You can find the used API specification here.

skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json

This command generates a load test file: products_POST_load_test.py.

Adjustments

Below are a few flags to customize the test generation.

Load Flags:

  • --default-load-count: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set if load-duration is not specified.

  • --default-load-duration: Duration of the load test execution in seconds; this flag cannot be used in combination with default-load-count[default=5]

  • --default-load-num-threads: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.

  • --default-load-rampup-duration: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]

  • --default-load-rampup-interval: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]

  • --default-load-target-rps: Specify the maximum RPS of the load test [default=None]

Output Flags:

  • --output: Specify the name of the generated test file.

  • --output-dir: Specify the directory to store the generated test file.

3. Execute Load Test

Now that the load test has been generated and saved, you can execute the test without any additional adjustments to the code.

3.1 Set env variable for authentication (if needed)

Skyramp’s sample application (link) doesn't require authentication, however, as most applications do, you can quickly pass your token via an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

You can now execute the test using Pytest.

# Prerequisites 
pip install pytest

# Execution of load test for products/POST 
python3 -m pytest products_POST_load_test.py

4. Skyramp Test File

This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.

  • At the top of each file, we show when the test was generated and what command was used

  • Below, we import all relevant libraries and specify the URL for all test requests

  • Next, we define the load test configurations

  • We define a function for the method that is tested. It consists of:

    • Invocation of Skyramp Client

    • Definition of the authentication header

    • Creation of a Skyramp scenario

      • The scenario serves as encapsulation of a series of requests relevant for the test case

    • Definition of the request body (based on API schema or sample data)

    • Creation of asynch request

    • Combination of load config and asynch request to send to test runner

Asynch Test Execution Behavior

  • Default behavior is that Skyramp worker tries to maximize RPS for the 5 second load test window and we collect latency and failure rate of those requests

  • Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.

4.1 Single Method

# Generated by Skyramp v0.5.6 on 2025-02-28 15:55:24.367378 -0500 EST m=+0.246110834
# Command: skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework pytest \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

load_config = skyramp.LoadTestConfig(
    load_duration=5,
    load_num_threads=1,
    load_target_rps=None,
    load_count=None,
    load_rampup_duration=None,
    load_rampup_interval=None
)
def test_api_v_1_products_post(load_test_config:skyramp.LoadTestConfig=load_config):
    # Invocation of Skyramp Client
    client = skyramp.Client()

    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")
    
    # Definition of Skyramp Scenario
    scenario = skyramp.AsyncScenario(name="scenario")

    # Request Body
    api_v1_products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Creation of asynchronous request
    api_v1_products_POST_response = scenario.add_async_request(
        name="products_POST_WkJc",
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=api_v1_products_POST_request_body,
        headers=headers
    )
    client.send_scenario(
        scenario,
        load_test_config=load_test_config
    )


if __name__ == "__main__":
    test_api_v_1_products_post()

4.1.1 Changing load test configuration

You can easily change the load test configuration using the flags in the generate command or updating the value in the generated code.

Generated Load Config (lines 15-22)

load_config = skyramp.LoadTestConfig(
    load_duration=5,
    load_num_threads=1,
    load_target_rps=None,
    load_count=None,
    load_rampup_duration=None,
    load_rampup_interval=None
)

Adjusted Load Config

load_config = skyramp.LoadTestConfig(
    load_duration=20,
    load_num_threads=2,
    load_target_rps=1000,
    load_count=None,
    load_rampup_duration=2,
    load_rampup_interval=5
)

Python

2.1 Single Method

To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.

You can find the used API specification here.

skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json

This command generates a load test file: products_POST_load_test.py.

Adjustments

Below are a few flags to customize the test generation.

Load Flags:

  • --default-load-count: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set if load-duration is not specified.

  • --default-load-duration: Duration of the load test execution in seconds; this flag cannot be used in combination with default-load-count[default=5]

  • --default-load-num-threads: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.

  • --default-load-rampup-duration: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]

  • --default-load-rampup-interval: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]

  • --default-load-target-rps: Specify the maximum RPS of the load test [default=None]

Output Flags:

  • --output: Specify the name of the generated test file.

  • --output-dir: Specify the directory to store the generated test file.

3. Execute Load Test

Now that the load test has been generated and saved, you can execute the test without any additional adjustments to the code.

3.1 Set env variable for authentication (if needed)

Skyramp’s sample application (link) doesn't require authentication, however, as most applications do, you can quickly pass your token via an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

You can now execute the test using Pytest.

# Prerequisites 
pip install pytest

# Execution of load test for products/POST 
python3 -m pytest products_POST_load_test.py

4. Skyramp Test File

This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.

  • At the top of each file, we show when the test was generated and what command was used

  • Below, we import all relevant libraries and specify the URL for all test requests

  • Next, we define the load test configurations

  • We define a function for the method that is tested. It consists of:

    • Invocation of Skyramp Client

    • Definition of the authentication header

    • Creation of a Skyramp scenario

      • The scenario serves as encapsulation of a series of requests relevant for the test case

    • Definition of the request body (based on API schema or sample data)

    • Creation of asynch request

    • Combination of load config and asynch request to send to test runner

Asynch Test Execution Behavior

  • Default behavior is that Skyramp worker tries to maximize RPS for the 5 second load test window and we collect latency and failure rate of those requests

  • Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.

4.1 Single Method

# Generated by Skyramp v0.5.6 on 2025-02-28 15:55:24.367378 -0500 EST m=+0.246110834
# Command: skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework pytest \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

load_config = skyramp.LoadTestConfig(
    load_duration=5,
    load_num_threads=1,
    load_target_rps=None,
    load_count=None,
    load_rampup_duration=None,
    load_rampup_interval=None
)
def test_api_v_1_products_post(load_test_config:skyramp.LoadTestConfig=load_config):
    # Invocation of Skyramp Client
    client = skyramp.Client()

    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")
    
    # Definition of Skyramp Scenario
    scenario = skyramp.AsyncScenario(name="scenario")

    # Request Body
    api_v1_products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Creation of asynchronous request
    api_v1_products_POST_response = scenario.add_async_request(
        name="products_POST_WkJc",
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=api_v1_products_POST_request_body,
        headers=headers
    )
    client.send_scenario(
        scenario,
        load_test_config=load_test_config
    )


if __name__ == "__main__":
    test_api_v_1_products_post()

4.1.1 Changing load test configuration

You can easily change the load test configuration using the flags in the generate command or updating the value in the generated code.

Generated Load Config (lines 15-22)

load_config = skyramp.LoadTestConfig(
    load_duration=5,
    load_num_threads=1,
    load_target_rps=None,
    load_count=None,
    load_rampup_duration=None,
    load_rampup_interval=None
)

Adjusted Load Config

load_config = skyramp.LoadTestConfig(
    load_duration=20,
    load_num_threads=2,
    load_target_rps=1000,
    load_count=None,
    load_rampup_duration=2,
    load_rampup_interval=5
)

Python

2.1 Single Method

To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.

You can find the used API specification here.

skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json

This command generates a load test file: products_POST_load_test.py.

Adjustments

Below are a few flags to customize the test generation.

Load Flags:

  • --default-load-count: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set if load-duration is not specified.

  • --default-load-duration: Duration of the load test execution in seconds; this flag cannot be used in combination with default-load-count[default=5]

  • --default-load-num-threads: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.

  • --default-load-rampup-duration: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]

  • --default-load-rampup-interval: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]

  • --default-load-target-rps: Specify the maximum RPS of the load test [default=None]

Output Flags:

  • --output: Specify the name of the generated test file.

  • --output-dir: Specify the directory to store the generated test file.

3. Execute Load Test

Now that the load test has been generated and saved, you can execute the test without any additional adjustments to the code.

3.1 Set env variable for authentication (if needed)

Skyramp’s sample application (link) doesn't require authentication, however, as most applications do, you can quickly pass your token via an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

You can now execute the test using Pytest.

# Prerequisites 
pip install pytest

# Execution of load test for products/POST 
python3 -m pytest products_POST_load_test.py

4. Skyramp Test File

This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.

  • At the top of each file, we show when the test was generated and what command was used

  • Below, we import all relevant libraries and specify the URL for all test requests

  • Next, we define the load test configurations

  • We define a function for the method that is tested. It consists of:

    • Invocation of Skyramp Client

    • Definition of the authentication header

    • Creation of a Skyramp scenario

      • The scenario serves as encapsulation of a series of requests relevant for the test case

    • Definition of the request body (based on API schema or sample data)

    • Creation of asynch request

    • Combination of load config and asynch request to send to test runner

Asynch Test Execution Behavior

  • Default behavior is that Skyramp worker tries to maximize RPS for the 5 second load test window and we collect latency and failure rate of those requests

  • Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.

4.1 Single Method

# Generated by Skyramp v0.5.6 on 2025-02-28 15:55:24.367378 -0500 EST m=+0.246110834
# Command: skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework pytest \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

load_config = skyramp.LoadTestConfig(
    load_duration=5,
    load_num_threads=1,
    load_target_rps=None,
    load_count=None,
    load_rampup_duration=None,
    load_rampup_interval=None
)
def test_api_v_1_products_post(load_test_config:skyramp.LoadTestConfig=load_config):
    # Invocation of Skyramp Client
    client = skyramp.Client()

    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")
    
    # Definition of Skyramp Scenario
    scenario = skyramp.AsyncScenario(name="scenario")

    # Request Body
    api_v1_products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Creation of asynchronous request
    api_v1_products_POST_response = scenario.add_async_request(
        name="products_POST_WkJc",
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=api_v1_products_POST_request_body,
        headers=headers
    )
    client.send_scenario(
        scenario,
        load_test_config=load_test_config
    )


if __name__ == "__main__":
    test_api_v_1_products_post()

4.1.1 Changing load test configuration

You can easily change the load test configuration using the flags in the generate command or updating the value in the generated code.

Generated Load Config (lines 15-22)

load_config = skyramp.LoadTestConfig(
    load_duration=5,
    load_num_threads=1,
    load_target_rps=None,
    load_count=None,
    load_rampup_duration=None,
    load_rampup_interval=None
)

Adjusted Load Config

load_config = skyramp.LoadTestConfig(
    load_duration=20,
    load_num_threads=2,
    load_target_rps=1000,
    load_count=None,
    load_rampup_duration=2,
    load_rampup_interval=5
)

© 2025 Skyramp, Inc. All rights reserved.

© 2025 Skyramp, Inc. All rights reserved.

© 2025 Skyramp, Inc. All rights reserved.