Test Configuration

Introduction

Skyramp lets you control and modify how your tests run by setting the configuration options available in the tester_start_v1 function.

tester_start_v1 Function Parameters

1. scenarios: list of scenarios to execute; this is a Skyramp scenario object. The scenarios in the list will be executed sequentially.

2. test_name: name of the test to be used in generated test reports.

3. address: address of the Skyramp worker container. This option is specific to DockerClient.tester_start_v1. Please see Parallelism for how to run multiple workers at the same time.

4. cluster: name of the Kubernetes cluster where the test should run. This option is specific to K8SClient.tester_start_v1.

5. namespace: name of the Kubernetes namespace where the test should run. This option is specific to K8SClient.tester_start_v1.

6. blocked: boolean that determines whether the interpreter should wait for this call to complete or continue execution.

7. global_vars: dictionary to supply or overwrite variable test parameters, e.g. API tokens.

8. override_code_path: path to a csv file that contains set of variable test parameters that should be applied together to produce a test instance. As many test instances as there are rows in the csv file will be executed.

9. blobs: dictionary to change sample data used for requests.

10. skip_verify: boolean to determine whether TLS verification should be done for requests.

11. loadtest_config: options to change the test to a load test (see Performance Testing)

Example

#Define scenarios
def add_artists_functional_scenario():
    scenario = skyramp.Scenario(
        "artists_functional_scenario",
        ignore=False,
        vars_={"token":"_token"}
    )
    # Endpoint /v1/artists and Method POST
    scenario.add_request_v1(
        request=artists_POST_request,
        step_name="artists_functional_scenario-0",
        vars_override={},
        description="Endpoint /v1/artists and Method POST"
    )
    # Assert of scenario step artists_functional_scenario-0 - Endpoint /v1/artists and Method POST
    scenario.add_assert_v1(
        assert_value="requests.artists_POST.code",
        assert_expected_value="201", 
        assert_step_name="artists_functional_scenario-1",
        description="Assert of scenario step artists_functional_scenario-0 - Endpoint /v1/artists and Method POST"
    )
    return scenario
def get_test_scenarios():
    scenario = skyramp.Scenario(
        "artists_full_scenario",
        ignore=True,
        vars_={"token":"globalVars.token"},
        headers={"X-API-Key":"vars.token"}
    )
    artists_functional_scenario = add_artists_functional_scenario()
    scenario.add_scenario_v1(
        artists_functional_scenario,
        step_name="artists_full_scenario-0" ,
        description="functional scenario of /v1/artists" ,
        vars_override={"token":"vars.token"}
    )
    return scenario
def execute_tests(
    address="localhost:35142",
    override_code_path=None,
    global_vars={"token":"defaultValue"},
    **kwargs
):
    docker_client = skyramp.DockerClient()
    scenarios = get_test_scenarios()
    status = docker_client.tester_start_v1(
        scenario=scenarios,
        test_name="artists test",
        address=address,
        blocked=True,
        global_vars=global_vars,
        override_code_path=override_code_path, 
        skip_verify=kwargs.get("skip_verify", False),
        blobs=kwargs.get("blobs", {}), 
        loadtest_config=skyramp.LoadTestConfig.from_kwargs(**kwargs)
    )
    return status

In this example:

• We first collect the scenarios to be executed through the get_test_scenarios() function and then pass them to tester_start_v1.

• Next, we set the name of the test to be used in reporting to "artists test"

• We specify the Skyramp worker address to be "localhost:35142"

• We require that the interpreter waits for the test to complete before continuing

• We use global_vars to pass the API token, "token":"defaultValue"

• In this simple case we don’t parameterize the test further. Please see Parameterize Tests for how to leverage override_code_path to execute multiple instances of the same test with different parameters.

• We turn off TLS verification.

• We don’t change the sample request data and so artists_request_data will be used.

• We don’t supply arguments to change the test to a load test. Please see Performance Testing for how to do load tests.