Generation
Load Test
Load Testing
This guide describes generating load tests using the Skyramp CLI. We'll demonstrate using Skyramp’s Demo Shop API, a simple e-commerce API for product and order management. Learn more about the Demo Shop API.
If you haven’t already installed Skyramp, refer to the instructions in the Installation Guide.
Overview
Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.
Unlike other types of Skyramp tests, load tests are run asynchronously. This means that all requests are packaged in a scenario, sent to a worker (component responsible for executing the test), and executed without any individual responses being communicated back to the program runtime.
Why would I want to execute a test async?
For load testing scenarios with multiple chained API calls and simulations of high volume traffic, async test execution enables significantly higher test performance and simulate more realistic load on the system through concurrent requests and defining varying degrees of load on the system.
Generate Load Test for REST APIs
This section explains how you can use Skyramp to generate load tests for a single method of a REST endpoint. Learn more about generating load tests from traces.
To reliably generate single-method test cases, provide at least one of the following inputs:
OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)
In this guide, we'll use the OpenAPI schema approach. If you want more control over the generated body values, you can also do test generation from sample data.
Generate Load Test for a Single Method
To create a load test for a single method, specify the method you want to test against in the command. In this example, we are using https://demoshop.skyramp.dev/api as the URL to our service. When testing your service, replace it with the URL of the endpoint you want to test.
You can find the used API specification here.
Load Testing
This guide describes generating load tests using the Skyramp CLI. We'll demonstrate using Skyramp’s Demo Shop API, a simple e-commerce API for product and order management. Learn more about the Demo Shop API.
If you haven’t already installed Skyramp, refer to the instructions in the Installation Guide.
Overview
Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.
Unlike other types of Skyramp tests, load tests are run asynchronously. This means that all requests are packaged in a scenario, sent to a worker (component responsible for executing the test), and executed without any individual responses being communicated back to the program runtime.
Why would I want to execute a test async?
For load testing scenarios with multiple chained API calls and simulations of high volume traffic, async test execution enables significantly higher test performance and simulate more realistic load on the system through concurrent requests and defining varying degrees of load on the system.
Generate Load Test for REST APIs
This section explains how you can use Skyramp to generate load tests for a single method of a REST endpoint. Learn more about generating load tests from traces.
To reliably generate single-method test cases, provide at least one of the following inputs:
OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)
In this guide, we'll use the OpenAPI schema approach. If you want more control over the generated body values, you can also do test generation from sample data.
Generate Load Test for a Single Method
To create a load test for a single method, specify the method you want to test against in the command. In this example, we are using https://demoshop.skyramp.dev/api as the URL to our service. When testing your service, replace it with the URL of the endpoint you want to test.
You can find the used API specification here.
Load Testing
This guide describes generating load tests using the Skyramp CLI. We'll demonstrate using Skyramp’s Demo Shop API, a simple e-commerce API for product and order management. Learn more about the Demo Shop API.
If you haven’t already installed Skyramp, refer to the instructions in the Installation Guide.
Overview
Load testing assesses a system’s performance by simulating real-world traffic and usage patterns. It helps identify bottlenecks, scalability limits, and stability issues by measuring how the software behaves under expected and peak loads. Ensuring a system can handle demand without degradation is key to maintaining reliability and a smooth user experience.
Unlike other types of Skyramp tests, load tests are run asynchronously. This means that all requests are packaged in a scenario, sent to a worker (component responsible for executing the test), and executed without any individual responses being communicated back to the program runtime.
Why would I want to execute a test async?
For load testing scenarios with multiple chained API calls and simulations of high volume traffic, async test execution enables significantly higher test performance and simulate more realistic load on the system through concurrent requests and defining varying degrees of load on the system.
Generate Load Test for REST APIs
This section explains how you can use Skyramp to generate load tests for a single method of a REST endpoint. Learn more about generating load tests from traces.
To reliably generate single-method test cases, provide at least one of the following inputs:
OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)
In this guide, we'll use the OpenAPI schema approach. If you want more control over the generated body values, you can also do test generation from sample data.
Generate Load Test for a Single Method
To create a load test for a single method, specify the method you want to test against in the command. In this example, we are using https://demoshop.skyramp.dev/api as the URL to our service. When testing your service, replace it with the URL of the endpoint you want to test.
You can find the used API specification here.
Python
Java
skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema
This command generates one file:
products_POST_load_test.py
The contents of this file are explained in the Test File Anatomy.
Explanation of Command
https://demoshop.skyramp.dev/api/v1/products: Defines the URL to the endpoint we aim to test.--language: Specifies the test output language. We currently support Python, Java, and TypeScript.--framework: Specify the test execution framework of choice.--api-schema: Points to the OpenAPI schema used to generate the test. We also support sample data and traces as inputs for load test generation.
Adjustments
Below are a few flags to customize the test generation. Additional flags are explained here.
Load Test Flags:
--default-count: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set ifload-durationis set to 0.--default-duration: Duration of the load test execution in seconds; this flag cannot be used in combination withdefault-load-count[default=5]--default-num-threads: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.--default-rampup-duration: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]--default-rampup-interval: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]--default-target-rps: Specify the maximum RPS of the load test [default=None]
Output Flags:
--output: Specify the name of the generated test file.--output-dir: Specify the directory to store the generated test file.
Execute the Load Test
You can execute the generated tests without any additional adjustments to the code.
NOTE: Asynch Test Execution Behavior
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test scenario is executed without waiting for feedback from the program runtime. This allows for a more realistic simulation of load on the system.
If a load duration rather than a count is provided, the Skyramp worker tries to maximize RPS for the specified load test duration while collecting the latency and failure rate of those requests
Set environment variable for authentication (if applicable)
Skyramp’s sample application doesn't require any authentication.
To test against an application that does require authentication, pass your token using an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.
export SKYRAMP_TEST_TOKEN=$your_auth_tokenRun the Test
For load testing, we run the test using Python directly, which allows us to display more meaningful test results.
Based on the load configuration, the test will execute as many POST requests as possible against the service within a 5 second duration, collect failure rates and latency metrics, and validate against the expected response status code.
Review Test Results
The output shows the overall results and performance of the load test. From the below output, we can see for this run the POST request was executed 9 times with 0 failures over the span of the default duration of 5 seconds.
Next Steps
Congratulations, you just generated your first load test using Skyramp! To learn more about how to adjust the the test file, please go to the Test File Anatomy page.
Related topics
Python
Java
skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema
This command generates one file:
products_POST_load_test.py
The contents of this file are explained in the Test File Anatomy.
Explanation of Command
https://demoshop.skyramp.dev/api/v1/products: Defines the URL to the endpoint we aim to test.--language: Specifies the test output language. We currently support Python, Java, and TypeScript.--framework: Specify the test execution framework of choice.--api-schema: Points to the OpenAPI schema used to generate the test. We also support sample data and traces as inputs for load test generation.
Adjustments
Below are a few flags to customize the test generation. Additional flags are explained here.
Load Test Flags:
--default-count: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set ifload-durationis set to 0.--default-duration: Duration of the load test execution in seconds; this flag cannot be used in combination withdefault-load-count[default=5]--default-num-threads: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.--default-rampup-duration: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]--default-rampup-interval: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]--default-target-rps: Specify the maximum RPS of the load test [default=None]
Output Flags:
--output: Specify the name of the generated test file.--output-dir: Specify the directory to store the generated test file.
Execute the Load Test
You can execute the generated tests without any additional adjustments to the code.
NOTE: Asynch Test Execution Behavior
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test scenario is executed without waiting for feedback from the program runtime. This allows for a more realistic simulation of load on the system.
If a load duration rather than a count is provided, the Skyramp worker tries to maximize RPS for the specified load test duration while collecting the latency and failure rate of those requests
Set environment variable for authentication (if applicable)
Skyramp’s sample application doesn't require any authentication.
To test against an application that does require authentication, pass your token using an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.
export SKYRAMP_TEST_TOKEN=$your_auth_tokenRun the Test
For load testing, we run the test using Python directly, which allows us to display more meaningful test results.
Based on the load configuration, the test will execute as many POST requests as possible against the service within a 5 second duration, collect failure rates and latency metrics, and validate against the expected response status code.
Review Test Results
The output shows the overall results and performance of the load test. From the below output, we can see for this run the POST request was executed 9 times with 0 failures over the span of the default duration of 5 seconds.
Next Steps
Congratulations, you just generated your first load test using Skyramp! To learn more about how to adjust the the test file, please go to the Test File Anatomy page.
Related topics
Python
Java
skyramp generate load rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema
This command generates one file:
products_POST_load_test.py
The contents of this file are explained in the Test File Anatomy.
Explanation of Command
https://demoshop.skyramp.dev/api/v1/products: Defines the URL to the endpoint we aim to test.--language: Specifies the test output language. We currently support Python, Java, and TypeScript.--framework: Specify the test execution framework of choice.--api-schema: Points to the OpenAPI schema used to generate the test. We also support sample data and traces as inputs for load test generation.
Adjustments
Below are a few flags to customize the test generation. Additional flags are explained here.
Load Test Flags:
--default-count: Number of times Skyramp executes the defined request [default=None]; this flag needs to be set ifload-durationis set to 0.--default-duration: Duration of the load test execution in seconds; this flag cannot be used in combination withdefault-load-count[default=5]--default-num-threads: Number of concurrent threads for load test [default=1]. Concurrent threads represent virtual users enabling you to test the vertical scalability of the service.--default-rampup-duration: Specify the duration that Skyramp incrementally increases the requests per second (RPS) until the target RPS are reached [default=None]--default-rampup-interval: Specify how often Skyramp increases the RPS until target RPS are reached [default=None]--default-target-rps: Specify the maximum RPS of the load test [default=None]
Output Flags:
--output: Specify the name of the generated test file.--output-dir: Specify the directory to store the generated test file.
Execute the Load Test
You can execute the generated tests without any additional adjustments to the code.
NOTE: Asynch Test Execution Behavior
Unlike other types of Skyramp tests, load testing is asynchronous, and every command in the test scenario is executed without waiting for feedback from the program runtime. This allows for a more realistic simulation of load on the system.
If a load duration rather than a count is provided, the Skyramp worker tries to maximize RPS for the specified load test duration while collecting the latency and failure rate of those requests
Set environment variable for authentication (if applicable)
Skyramp’s sample application doesn't require any authentication.
To test against an application that does require authentication, pass your token using an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.
export SKYRAMP_TEST_TOKEN=$your_auth_tokenRun the Test
For load testing, we run the test using Python directly, which allows us to display more meaningful test results.
Based on the load configuration, the test will execute as many POST requests as possible against the service within a 5 second duration, collect failure rates and latency metrics, and validate against the expected response status code.
Review Test Results
The output shows the overall results and performance of the load test. From the below output, we can see for this run the POST request was executed 9 times with 0 failures over the span of the default duration of 5 seconds.
Next Steps
Congratulations, you just generated your first load test using Skyramp! To learn more about how to adjust the the test file, please go to the Test File Anatomy page.