Run Batches

In the next steps, you will run the batches and review the results.

By default, batched test case jobfiles will be executed sequentially, in the order in which they were added to the batch. However, Job Files placed in separate execution queues are executed in parallel, potentially significantly decreasing the duration of testing stages in your CI/CD pipeline.

Step 1 - Run the LTROPS-2711 Topology System Batch

From your project's Batches page:

1. Click on the Run link for the LTROPS-2711 Topology System Batch

Step 2 - Run the LTROPS-2711 Topology Routing Batch

From your project's Batches page:

1. Click on the Run link for the LTROPS-2711 Topology Routing Batch

Step 3 - Review the LTROPS-2711 Topology System Batch results

From your project's Batches page:

1. Click on the Results link for the LTROPS-2711 Topology System Batch




2. Confirm that test cases are running





When a batch is run, the status of each test case will be initialized to "QUEUED" until the test case has completed and transitioned to a final status of either "PASSED" or "FAILED". The status of the test case will automatically refresh on this page as CXTM progresses through the queue of test cases.

The Job File column in this table provides a direct link to the Job File page of each test case. The Topology column shows the topology file used by each Job File. Notice how the Topology is set to the topology file that you selected earlier when changing the default topology for all Job Files. As each test case transitions to a "PASSED" or "FAILED" status, data will be populated in additional columns in this table. Take a moment to observe how this table is updated as each test case completes.



3. Confirm that test cases have Passed





Now that the batched test cases have completed, take a moment to review the populated table. Notice how each executed Job File has a Start and End timestamp that is used to calculate an execution Duration. The Duration value at the top of the page is the total time taken for all test cases to run to completion in the batch. By enabling individual execution queues, the testcases within the batch are run in parallel. So, the batch execution duration is not the aggregate of the individual test case execution durations, but rather it is approximately equal (within a few seconds) to the execution duration of the longest running test case.

The Results Status Count at the top provides a status summary, showing the total number of test cases in each status. Your output should show 4 test cases in PASSED state when this batch completes.

Like the Run History that is updated for each of individual test case, a new entry will be added to Batch Runs, providing an archival of batch runs and results.

Alert

Notify your proctor if any test cases have Failed before moving on.

Step 4 - Review the LTROPS-2711 Topology Routing Batch results

From your the LTROPS-2711 Topology System Batch results page:

1. Click on the Batches tab in the navigation breadcrumbs to return to the Batches page
2. Click on the Results link for the LTROPS-2711 Topology Routing Batch




3. Confirm that test cases are running, or have progressed to a completed status of Passed or Failed




4. Confirm that test cases have Passed





Alert

Notify your proctor if any test cases have Failed before moving on.

Step 5 - Review the Verify BGP Neighbor State on IOS-XE via CLI test case

From your the LTROPS-2711 Topology Routing Batch results page:

1. Click on the Details link associated with the Verify BGP Neighbor State on IOS-XE via CLI test case





Warning

NOTE: The order in which your test cases appear may differ from the example above.



2. Click on the on log.html link under Artifacts to open the Jobfile log



This link opens a detailed log.html report ("Jobfile Log") for all the procedures executed in the Job File.




3. Click on TEST 2. VERIFY ALL CONFIGURED BGP NEIGHBOR STATES ESTABLISHED FOR THE GIVEN DUT to expand
4. Click on FOR ${DEVICE} IN [ @{DEVICES} ] to expand
5. Click on ITERATION ${DEVICE} = CSR1Kv-01 to expand
6. Click on FOR ${neighbor} IN [ @{neighbors} ] to expand



In this expanded output, notice that this test case was only executed on your CSR1Kv device and the only BGP neighbor found is 4.4.4.4 (your N9Kv device). In the next steps, you will add the C8Kv device to the DEVICES list of this test case and review the result again after it has been executed in the pipeline to see the difference.


7. Close the Jobfile log tab in your browser and return to the Job File Results page

Step 6 - Return to Test Cases Page

From the Job File Results page:

1. Navigate back to your project home by clicking on the Project LTROPS-2711 tab in the navigation breadcrumbs.




2. Click on VIEW TEST CASES




3. Confirm that the test case state is "Passed" for all test cases

Now that all of the test cases have been run in their respective batch and Passed, the state of each test on your project's Test Cases page should have updated to reflect the most recent execution state.

Step 7 - Add C8Kv-01 to Verify OSPF Neighbor State on IOS-XE via CLI

In order to verify the OSPF and BGP operational state on the C8Kv device in the pipeline, you will need to edit the Parameter File in both the "Verify OSPF Neighbor State on IOS-XE via CLI" and "Verify BGP Neighbor State on IOS-XE via CLI" test cases to include your C8Kv device in the DEVICES list. There is no need to rerun these test cases now. The next time they will run is when executed in the pipeline after the proposed configuration changes have been pushed.

From your project's Test Cases page:

1. Locate the test case Verify OSPF Neighbor State on IOS-XE via CLI that has identifier 2.01
2. For this test case's row, click on the Yes link under the Job File? column




3. Click on the pencil icon in the Job File preview pane
4. Use the parameters below and insert the C8Kv-01 device on line 3 of the Parameter File section of your Job File.

Warning

NOTE: You may need to scroll down to see the Parameter File section.

DEVICES:
    - CSR1Kv-01
    - C8Kv-01

EXPECTED_STATE:  FULL



5. Scroll back up to the top of the Job File page and click Save




6. Navigate back to your project home by clicking on the Project LTROPS-2711 tab in the navigation breadcrumbs.




7. Click on VIEW TEST CASES

Step 8 - Add C8Kv-01 to Verify BGP Neighbor State on IOS-XE via CLI

From your project's Test Cases page:

1. Locate the test case Verify BGP Neighbor State on IOS-XE via CLI that has identifier 2.04
2. For this test case's row, click on the Yes link under the Job File? column




3. Click on the pencil icon in the Job File preview pane
4. Use the parameters below and insert the C8Kv-01 device on line 3 of the Parameter File section of your Job File.

Warning

NOTE: You may need to scroll down to see the Parameter File section.

DEVICES:
    - CSR1Kv-01
    - C8Kv-01

EXPECTED_STATE:  Established



5. Scroll back up to the top of the Job File page and click Save

Now, that you have successfully run your automated tests through the GUI, and added the C8Kv-01 device to the applicable test cases, proceed to the next section to enable Webex Notifications when your test case batch runs have completed in the pipeline.