Skyramp Test

Parameterize Tests

Introduction

You can make your tests more thorough and extensive by varying the inputs and conditions. Skyramp lets you specify overrides of global variables, scenario step variables, scenario step blob overrides, and override status codes for assertions.

Manual Overrides

In the provided Skyramp test setup, parameterization can be implemented by modifying the test data or scenario configurations to cover different cases. For instance, in the add_artists_negative_post_scenario_0 and add_artists_negative_post_scenario_1 functions, negative scenarios are defined by setting different values for the “name” field.

def add_artists_negative_post_scenario_0():
    scenario = skyramp.Scenario(
        "artists_negative_post_scenario_0",
        ignore=True,
        vars_={"token":"token"}
    )
    # Negative case of setting name to 0123456789
    scenario.add_request_v1(
        request=artists_POST_request,
        step_name="artists_negative_post_scenario-0",
        vars_override={},
        description="Negative case of setting name to 0123456789", 
        blob_override={"name":"0123456789"}
    )
    # Assert of scenario step artists_negative_post_scenario-0 - Negative case of setting name to 0123456789
    scenario.add_assert_v1(
        assert_value="requests.artists_POST.code",
		assert_expected_value="400", 
        assert_step_name="artists_negative_post_scenario-1",
        description="Assert of scenario step artists_negative_post_scenario-0 - Negative case of setting name to 0123456789"
    )
    return scenario

def add_artists_negative_post_scenario_1():
    scenario = skyramp.Scenario(
        "artists_negative_post_scenario_1",
        ignore=True,
        vars_={"token":"token"}
    )
    # Negative case of setting name to nil
    scenario.add_request_v1(
        request=artists_POST_request,
        step_name="artists_negative_post_scenario-2",
        vars_override={},
        description="Negative case of setting name to nil", 
        blob_override={"name": None}
    )
    # Assert of scenario step artists_negative_post_scenario-2 - Negative case of setting name to nil
    scenario.add_assert_v1(
        assert_value="requests.artists_POST.code",
		assert_expected_value="400", 
        assert_step_name="artists_negative_post_scenario-3",
        description="Assert of scenario step artists_negative_post_scenario-2 - Negative case of setting name to nil"
    )
    return scenario

By parameterizing these scenarios, the test runner can execute them with different inputs to validate the behavior under various conditions.

Parameterization via CSV

Alternatively, parameterization can be achieved by adding the required overrides to a CSV file, which is then passed as override_code_path to the generated execute_test function. This method allows for easy and organized management of different test scenarios.

Example

To execute the test with parameterized inputs, use the following call:

execute_test(override_code_path="parameterization.csv")

Here, parameterization.csv contains the overrides with headers in the first row:

In this example:

globalVars.X is set to "test".
globalVars.Y is set to "3".
clusters_crud_scenario-0.VarOverride overrides the scenario step variables with {"project":"my-project","token":"abc"}.
clusters_crud_scenario-0.BlobOverride overrides the blob with {"apiVersion":"你"}.
clusters_crud_scenario-1.StatusCode overrides the status code to "500".

By organizing parameterization in a CSV file, test scenarios can be easily scaled and managed, ensuring comprehensive coverage of various test cases and conditions.