Generation
Getting Started
Getting Started
Smoke Test
Load Testing
This guide will walk you through generating load tests with the Skyramp CLI.
1. Overview
Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.
2. Generate Load Test for REST APIs
This section explains how you can use Skyramp to generate load tests for REST APIs. You can generate load tests for:
a specific method of an endpoint
To reliably generate test cases, provide at least one of the following inputs:
OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)
While we focus on using an API schema as input for this guide, you can find detailed explanations on achieving test generation from sample data further down - this allows you to have more control over the generated body values.
Load Testing
This guide will walk you through generating load tests with the Skyramp CLI.
1. Overview
Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.
2. Generate Load Test for REST APIs
This section explains how you can use Skyramp to generate load tests for REST APIs. You can generate load tests for:
a specific method of an endpoint
To reliably generate test cases, provide at least one of the following inputs:
OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)
While we focus on using an API schema as input for this guide, you can find detailed explanations on achieving test generation from sample data further down - this allows you to have more control over the generated body values.
Load Testing
This guide will walk you through generating load tests with the Skyramp CLI.
1. Overview
Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.
2. Generate Load Test for REST APIs
This section explains how you can use Skyramp to generate load tests for REST APIs. You can generate load tests for:
a specific method of an endpoint
To reliably generate test cases, provide at least one of the following inputs:
OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)
While we focus on using an API schema as input for this guide, you can find detailed explanations on achieving test generation from sample data further down - this allows you to have more control over the generated body values.
Python
2.1 Single Method
To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products
as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.
You can find the used API specification here.
skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json
This command generates a load test file: products_POST_load_test.py
.
Adjustments
Below are a few flags to customize the test generation.
Load Flags:
--default-load-count
: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set ifload-duration
is not specified.--default-load-duration
: Duration of the load test execution in seconds; this flag cannot be used in combination withdefault-load-count
[default=5]--default-load-num-threads
: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.--default-load-rampup-duration
: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]--default-load-rampup-interval
: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]--default-load-target-rps
: Specify the maximum RPS of the load test [default=None]
Output Flags:
--output
: Specify the name of the generated test file.--output-dir
: Specify the directory to store the generated test file.
3. Execute Load Test
Now that the load test has been generated and saved, you can execute the test without any additional adjustments to the code.
3.1 Set env variable for authentication (if needed)
Skyramp’s sample application (link) doesn't require authentication, however, as most applications do, you can quickly pass your token via an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.
export SKYRAMP_TEST_TOKEN=$your_auth_token
3.2 Run the test
You can now execute the test using Pytest.
# Prerequisites
pip install pytest
# Execution of load test for products/POST
python3 -m pytest products_POST_load_test.py
4. Skyramp Test File
This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.
At the top of each file, we show when the test was generated and what command was used
Below, we import all relevant libraries and specify the URL for all test requests
Next, we define the load test configurations
We define a function for the method that is tested. It consists of:
Invocation of Skyramp Client
Definition of the authentication header
Creation of a Skyramp scenario
The scenario serves as encapsulation of a series of requests relevant for the test case
Definition of the request body (based on API schema or sample data)
Creation of asynch request
Combination of load config and asynch request to send to test runner
Asynch Test Execution Behavior
Default behavior is that Skyramp worker tries to maximize RPS for the 5 second load test window and we collect latency and failure rate of those requests
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.
4.1 Single Method
# Generated by Skyramp v0.5.6 on 2025-02-28 15:55:24.367378 -0500 EST m=+0.246110834
# Command: skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
# --api-schema openapi.json \
# --framework pytest \
# --language python \
# --method POST \
# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"
load_config = skyramp.LoadTestConfig(
load_duration=5,
load_num_threads=1,
load_target_rps=None,
load_count=None,
load_rampup_duration=None,
load_rampup_interval=None
)
def test_api_v_1_products_post(load_test_config:skyramp.LoadTestConfig=load_config):
# Invocation of Skyramp Client
client = skyramp.Client()
# Definition of authentication header
headers = {}
if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")
# Definition of Skyramp Scenario
scenario = skyramp.AsyncScenario(name="scenario")
# Request Body
api_v1_products_POST_request_body = r'''{
"category": "string",
"description": "string",
"image_url": "string",
"in_stock": false,
"name": "string",
"price": 0
}'''
# Creation of asynchronous request
api_v1_products_POST_response = scenario.add_async_request(
name="products_POST_WkJc",
url=URL,
path="/api/v1/products",
method="POST",
body=api_v1_products_POST_request_body,
headers=headers
)
client.send_scenario(
scenario,
load_test_config=load_test_config
)
if __name__ == "__main__":
test_api_v_1_products_post()
4.1.1 Changing load test configuration
You can easily change the load test configuration using the flags in the generate command or updating the value in the generated code.
Generated Load Config (lines 15-22)
load_config = skyramp.LoadTestConfig(
load_duration=5,
load_num_threads=1,
load_target_rps=None,
load_count=None,
load_rampup_duration=None,
load_rampup_interval=None
)
Adjusted Load Config
load_config = skyramp.LoadTestConfig(
load_duration=20,
load_num_threads=2,
load_target_rps=1000,
load_count=None,
load_rampup_duration=2,
load_rampup_interval=5
)
Python
2.1 Single Method
To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products
as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.
You can find the used API specification here.
skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json
This command generates a load test file: products_POST_load_test.py
.
Adjustments
Below are a few flags to customize the test generation.
Load Flags:
--default-load-count
: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set ifload-duration
is not specified.--default-load-duration
: Duration of the load test execution in seconds; this flag cannot be used in combination withdefault-load-count
[default=5]--default-load-num-threads
: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.--default-load-rampup-duration
: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]--default-load-rampup-interval
: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]--default-load-target-rps
: Specify the maximum RPS of the load test [default=None]
Output Flags:
--output
: Specify the name of the generated test file.--output-dir
: Specify the directory to store the generated test file.
3. Execute Load Test
Now that the load test has been generated and saved, you can execute the test without any additional adjustments to the code.
3.1 Set env variable for authentication (if needed)
Skyramp’s sample application (link) doesn't require authentication, however, as most applications do, you can quickly pass your token via an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.
export SKYRAMP_TEST_TOKEN=$your_auth_token
3.2 Run the test
You can now execute the test using Pytest.
# Prerequisites
pip install pytest
# Execution of load test for products/POST
python3 -m pytest products_POST_load_test.py
4. Skyramp Test File
This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.
At the top of each file, we show when the test was generated and what command was used
Below, we import all relevant libraries and specify the URL for all test requests
Next, we define the load test configurations
We define a function for the method that is tested. It consists of:
Invocation of Skyramp Client
Definition of the authentication header
Creation of a Skyramp scenario
The scenario serves as encapsulation of a series of requests relevant for the test case
Definition of the request body (based on API schema or sample data)
Creation of asynch request
Combination of load config and asynch request to send to test runner
Asynch Test Execution Behavior
Default behavior is that Skyramp worker tries to maximize RPS for the 5 second load test window and we collect latency and failure rate of those requests
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.
4.1 Single Method
# Generated by Skyramp v0.5.6 on 2025-02-28 15:55:24.367378 -0500 EST m=+0.246110834
# Command: skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
# --api-schema openapi.json \
# --framework pytest \
# --language python \
# --method POST \
# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"
load_config = skyramp.LoadTestConfig(
load_duration=5,
load_num_threads=1,
load_target_rps=None,
load_count=None,
load_rampup_duration=None,
load_rampup_interval=None
)
def test_api_v_1_products_post(load_test_config:skyramp.LoadTestConfig=load_config):
# Invocation of Skyramp Client
client = skyramp.Client()
# Definition of authentication header
headers = {}
if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")
# Definition of Skyramp Scenario
scenario = skyramp.AsyncScenario(name="scenario")
# Request Body
api_v1_products_POST_request_body = r'''{
"category": "string",
"description": "string",
"image_url": "string",
"in_stock": false,
"name": "string",
"price": 0
}'''
# Creation of asynchronous request
api_v1_products_POST_response = scenario.add_async_request(
name="products_POST_WkJc",
url=URL,
path="/api/v1/products",
method="POST",
body=api_v1_products_POST_request_body,
headers=headers
)
client.send_scenario(
scenario,
load_test_config=load_test_config
)
if __name__ == "__main__":
test_api_v_1_products_post()
4.1.1 Changing load test configuration
You can easily change the load test configuration using the flags in the generate command or updating the value in the generated code.
Generated Load Config (lines 15-22)
load_config = skyramp.LoadTestConfig(
load_duration=5,
load_num_threads=1,
load_target_rps=None,
load_count=None,
load_rampup_duration=None,
load_rampup_interval=None
)
Adjusted Load Config
load_config = skyramp.LoadTestConfig(
load_duration=20,
load_num_threads=2,
load_target_rps=1000,
load_count=None,
load_rampup_duration=2,
load_rampup_interval=5
)
Python
2.1 Single Method
To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products
as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.
You can find the used API specification here.
skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json
This command generates a load test file: products_POST_load_test.py
.
Adjustments
Below are a few flags to customize the test generation.
Load Flags:
--default-load-count
: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set ifload-duration
is not specified.--default-load-duration
: Duration of the load test execution in seconds; this flag cannot be used in combination withdefault-load-count
[default=5]--default-load-num-threads
: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.--default-load-rampup-duration
: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]--default-load-rampup-interval
: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]--default-load-target-rps
: Specify the maximum RPS of the load test [default=None]
Output Flags:
--output
: Specify the name of the generated test file.--output-dir
: Specify the directory to store the generated test file.
3. Execute Load Test
Now that the load test has been generated and saved, you can execute the test without any additional adjustments to the code.
3.1 Set env variable for authentication (if needed)
Skyramp’s sample application (link) doesn't require authentication, however, as most applications do, you can quickly pass your token via an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.
export SKYRAMP_TEST_TOKEN=$your_auth_token
3.2 Run the test
You can now execute the test using Pytest.
# Prerequisites
pip install pytest
# Execution of load test for products/POST
python3 -m pytest products_POST_load_test.py
4. Skyramp Test File
This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.
At the top of each file, we show when the test was generated and what command was used
Below, we import all relevant libraries and specify the URL for all test requests
Next, we define the load test configurations
We define a function for the method that is tested. It consists of:
Invocation of Skyramp Client
Definition of the authentication header
Creation of a Skyramp scenario
The scenario serves as encapsulation of a series of requests relevant for the test case
Definition of the request body (based on API schema or sample data)
Creation of asynch request
Combination of load config and asynch request to send to test runner
Asynch Test Execution Behavior
Default behavior is that Skyramp worker tries to maximize RPS for the 5 second load test window and we collect latency and failure rate of those requests
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test is executed without waiting for feedback. This allows for a more accurate understanding of the stresses a system experiences.
4.1 Single Method
# Generated by Skyramp v0.5.6 on 2025-02-28 15:55:24.367378 -0500 EST m=+0.246110834
# Command: skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
# --api-schema openapi.json \
# --framework pytest \
# --language python \
# --method POST \
# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"
load_config = skyramp.LoadTestConfig(
load_duration=5,
load_num_threads=1,
load_target_rps=None,
load_count=None,
load_rampup_duration=None,
load_rampup_interval=None
)
def test_api_v_1_products_post(load_test_config:skyramp.LoadTestConfig=load_config):
# Invocation of Skyramp Client
client = skyramp.Client()
# Definition of authentication header
headers = {}
if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")
# Definition of Skyramp Scenario
scenario = skyramp.AsyncScenario(name="scenario")
# Request Body
api_v1_products_POST_request_body = r'''{
"category": "string",
"description": "string",
"image_url": "string",
"in_stock": false,
"name": "string",
"price": 0
}'''
# Creation of asynchronous request
api_v1_products_POST_response = scenario.add_async_request(
name="products_POST_WkJc",
url=URL,
path="/api/v1/products",
method="POST",
body=api_v1_products_POST_request_body,
headers=headers
)
client.send_scenario(
scenario,
load_test_config=load_test_config
)
if __name__ == "__main__":
test_api_v_1_products_post()
4.1.1 Changing load test configuration
You can easily change the load test configuration using the flags in the generate command or updating the value in the generated code.
Generated Load Config (lines 15-22)
load_config = skyramp.LoadTestConfig(
load_duration=5,
load_num_threads=1,
load_target_rps=None,
load_count=None,
load_rampup_duration=None,
load_rampup_interval=None
)
Adjusted Load Config
load_config = skyramp.LoadTestConfig(
load_duration=20,
load_num_threads=2,
load_target_rps=1000,
load_count=None,
load_rampup_duration=2,
load_rampup_interval=5
)