Create Job File

Now that you have an understanding of Robot Framework and the CXTA collection of libraries, you will automate the test case that you added to your project earlier in the lab. In this section, you will develop a fully automated test case that utilizes the information you added to the CXTM Devices page to connect to the devices in your testbed, and verify that they are all running the expected software version.

Later in the lab, this same test case will be included in your pipeline to help verify that the testbed is in the desired state before introducing a configuration change.

Step 1 - Add a Job File to your test case

From the Test Cases page in your LTROPS-2711 project:

1. Locate the test case Verify Software Version on Devices that has identifier 1.01
2. Click on the on + icon in the Job File column of your test case



This link opens an integrated editor for the Robot Job File and Parameter File associated with the test case.

Step 2 - Configure the Job File Type and Image options

At the top of the page, select the following options from the dropdowns:

1. Job File Type: Script in CXTM
2. Editor File Mode: robot
3. Runtime Image Type: cxta
4. Runtime Image Version: cxta:24.5

The Job File Name defaults to the test case name. The Job File Type can either be Script in CXTM or Script in Git Repo if Git integration is configured and job files are stored in a source control repository. The Editor File Mode informs the editor of the language used in the Job File to provide the correct syntax highlighting, and the Runtime Image dropdown selects the CXTA version to be used by the Job File.

With these selections confirmed, you will develop the Job File itself next.

Step 3 - Add the Job File sections

1. Add the *** Settings *** and *** Test Cases *** sections by copying the Robot code below into your Job File, beginning on line 2.



    
*** Settings ***

*** Test Cases ***

    

As previously mentioned, Robot Job Files are organized into sections separated by section headers. You'll use *** Settings *** to import Libraries and Resource files, and *** Test Cases *** to define your test case procedures. You won't define variables or keywords directly in this example, so you do not need to include the *** Variables *** or *** Keywords *** sections.

2. Confirm that your Job File page looks like the following at this point.

Step 4 - Import Libraries and Resource files

1. Import the following Libraries and Resource file by copying lines 3-6 into the Script Text editor that will be used to develop your Job File.



    
*** Settings ***
Library  CXTA
Resource  cxta.robot

Library  BuiltIn

*** Test Cases ***


    

In the code snippet above, Library CXTA and Resource cxta.robot import the CXTA library and Keywords into the test case, and Library BuiltIn imports Builtin, one of the standard libraries distributed with open source Robot Framework. Notice the spacing used to comply with the space separated format.

Step 5 - Add Suite Setup and Teardown

1. Add the code on lines 8-12 below to your Job File to define the Suite Setup and Suite Teardown



    
*** Settings ***
Library  CXTA
Resource  cxta.robot

Library  BuiltIn

Suite Setup         Run Keywords
...                 load testbed

Suite Teardown      Run Keywords
...                 Disconnect From All Devices

*** Test Cases ***


    

In addition to importing Libraries and Resources, *** Settings *** is also where Suite Setup and Suite Teardown procedures are defined.

A test suite is all of the Test Cases, or procedures, in the *** Test Cases *** section of the Job File. The suite setup and teardown are keywords that can take arguments. A suite setup is executed before running any of the suite's test cases, so it's ideal for executing any prerequisite checks or actions that need to be done before executing the suite of test cases. In this example, suite setup is used to run the load testbed keyword to access the devices defined on your Devices page earlier.

The suite teardown, as you might imagine, is executed after all other test cases in the suite have been executed, and is used for cleaning up at the end of a Job File. As such, the suite teardown is executed even if there are failures in either the suite setup or in any of the test cases in the suite. Similarly, all the keywords in a suite teardown are executed even if one of them fails. In this example, the suite teardown is used to run the Disconnect From All Devices keyword to terminate the connections to your devices under test, and gracefully exit the test case.

Step 6 - Review the suite setup and teardown keywords

Run Keywords is in fact itself a keyword. It's included in the BuiltIn library that you imported above and can accept a list of keywords as arguments. In contrast, both load testbed and Disconnect From All Devices are CXTA.robot keywords developed by Cisco.

These keywords, and all the others covered in this lab, are documented in the Keyword Index of the Robot Documentation included in CXTM. Navigate to the CXTA Keyword Index that is included in the Robot Documentation of your CXTM instance. Locate the link to the Keyword Index from the menu on the left, use "Ctrl+F" to search for load testbed, , and click on the link for "load testbed". (Use your CXTM username and password if prompted for authentication)

The *** Settings *** section of your Job File is complete. Next, you'll create the procedure to connect to devices in your testbed.

Step 7 - Add a step to Connect to Devices

1. Add the code on lines 15-21 below to your the Job File under the *** Test Cases *** section

    
*** Settings ***
Library  CXTA
Resource  cxta.robot

Library  BuiltIn

Suite Setup         Run Keywords
...                 load testbed

Suite Teardown      Run Keywords
...                 Disconnect From All Devices

*** Test Cases ***
1. CONNECT TO DEVICES
    ${status}=  Run Keyword And Ignore Error  connect to device "${DEVICES}"
    IF  '${status[0]}' == 'FAIL'
        Fail  ++UNSUCCESSFUL++ ${DEVICES} not connected
    ELSE
        set test message  ++SUCCESSFUL++ ${DEVICES} connected \n  append=True
    END


    

Here is a brief explanation of the keywords and code just added to your test case.

Keyword	Description
Run Keyword And Ignore Error	From BuiltIn. Runs the keyword that is passed in as an argument. Returns either PASS or FAIL, and either the return value of the keyword or the received error message, which gets stored in ${status} in this example
connect to device "${device}"	From CXTA.robot. Passed in as an arg to the keyword above. Connects to the device passed in as an arg (the ${DEVICES} var in this example)
Fail	From BuiltIn. Fails the test with the given message ("++UNSUCCESSFUL++ ${DEVICES} not connected" in this example)
set test message	From BuiltIn. Sets the given test message ("++SUCCESSFUL++ ${DEVICES} connected" in this example)

Step 8 - Define the DEVICES variable

The procedure added above introduced an undefined variable (DEVICES). In this step, you'll define the variable in the Parameter File editor. Variables defined in Parameter File editor are always put in the /tmp/parameters.yaml file and accessible at run time.

In the Parameter File editor:

1. Add the following to define the DEVICES variable

Warning

NOTE: You may need to scroll down to see the Parameter File section.

    
DEVICES:  C8Kv-01


    

Your Parameter File editor should look like the following example.

The variable in this example (DEVICES) identifies just a single device (C8Kv-01) for now, so the code you added to the Job File above uses the scalar variable syntax ${var} in Robot. Later in the lab, you will change this variable type to a list to iterate over multiple devices in your topology.

Now that you have defined the variable used in the "CONNECT TO DEVICES" step, you will add another step to your job file to verify the software version running on the (C8Kv-01) device.

Step 9 - Import the Collections Library

The next step that you'll add to your test case introduces new keywords to the Job File. One of which belongs to the Collections library, so you'll need to import that library into your Job File under the *** Settings *** section.

1. Add Library Collections to line 7 of your Job File

    
*** Settings ***
Library  CXTA
Resource  cxta.robot

Library  BuiltIn
Library  Collections

Suite Setup         Run Keywords
...                 load testbed

Suite Teardown      Run Keywords
...                 Disconnect From All Devices

*** Test Cases ***
1. CONNECT TO DEVICES
    ${status}=  Run Keyword And Ignore Error  connect to device "${DEVICES}"
    IF  '${status[0]}' == 'FAIL'
        Fail  ++UNSUCCESSFUL++ ${DEVICES} not connected
    ELSE
        set test message  ++SUCCESSFUL++ ${DEVICES} connected \n  append=True
    END
    

    

Step 10 - Add a step to Verify Software Version

1. Add the code on lines 24-32 below to your the Job File under the *** Test Cases *** section

    
*** Settings ***
Library  CXTA
Resource  cxta.robot

Library  BuiltIn
Library  Collections

Suite Setup         Run Keywords
...                 load testbed

Suite Teardown      Run Keywords
...                 Disconnect From All Devices

*** Test Cases ***
1. CONNECT TO DEVICES
    ${status}=  Run Keyword And Ignore Error  connect to device "${DEVICES}"
    IF  '${status[0]}' == 'FAIL'
        Fail  ++UNSUCCESSFUL++ ${DEVICES} not connected
    ELSE
        set test message  ++SUCCESSFUL++ ${DEVICES} connected \n  append=True
    END

2. VERIFY SOFTWARE VERSION
    ${VERSION_OUTPUT}=  parse "show version" on device "${DEVICES}"
    ${CFG_VERSION}=  Get From Dictionary  ${VERSION_OUTPUT['version']}  version
    ${status}=  Run Keyword And Return Status  should be true  '${CFG_VERSION}' == '${EXP_XE_8K_VERSION}'
    IF  '${status}' == 'False'
        Fail  ++UNSUCCESSFUL++ Device ${DEVICES} is not running the expected version ${EXP_XE_8K_VERSION}
    ELSE
        set test message  ++SUCCESSFUL++ ${DEVICES} is running the expected version ${CFG_VERSION}\n  append=True
    END

    

Here is a brief explanation of the new keywords and code just added to your test case.

Keyword	Description
parse "${parser}" on device "${device}"	From genie.libs.robot.GenieRobot. Executes the command on the device and parses the output into a dictionary. In this example, the dictionary is stored in ${VERSION_OUTPUT} for use in the next line of your Job File.
Get From Dictionary	From Collections. Returns a value from the given dictionary ${VERSION_OUTPUT} based on the given key. The key is "version" in this example, and the key value is stored in the ${CFG_VERSION} variable for evaluation in the next line of your Job File.
Run Keyword And Return Status	From BuiltIn. Runs the keyword that is passed in as an argument and returns the status as True or False. In this example, it is used to run the "should be true" keyword to evaluate whether the device's software version matches the expected version.
should be true	From BuiltIn. Fails if the given condition is not true. See above for the explanation of use in this example.

In addition to new keywords, this step also introduced parsers. Parsers essentially take output from CLI-based show commands in human-readable format and turns it into machine-readable structured data in the form of dictionaries.

In this example, the keyword and parser come from the Genie library, which is bundled into CXTA by default. The Genie parsers list and their schemas are publicly available. You can refer to the published schema to identify the dictionary keys needed for your verification requirements.

Step 11 - Review the parsers schema

1. Click show version to access the Genie parser schema documentation
2. Scroll down to the iosxe schema and locate the version key in the schema

    
{
'version': {
    Optional  (str) xe_version: <class 'str'>,
    'version_short': <class 'str'>,
    'platform': <class 'str'>,
    'version': <class 'str'>,
    Optional  (str) label: <class 'str'>,
    <snip>
    },
}    

    

Take a moment to scroll down and review the iosxr and nxos schemas as well. You'll use them in an upcoming section.

3. Close this browser window and return to CXTM

Step 12 - Define the expected software version variable

The procedure introduced another undefined variable (EXP_XE_8K_VERSION). Define the variable in the Parameter File editor.

In the Parameter File editor:

1. Add the following to define the EXP_XE_8K_VERSION variable

Warning

NOTE: You may need to scroll down to see the Parameter File section.

    
DEVICES:  C8Kv-01

EXP_XE_8K_VERSION:  17.12.2

    

Step 13 - Save and Run

Congratulations! You have added all of the procedures and variables needed to the run this test case.

1. Confirm that your Script Text and Parameter File match the following, and copy/paste the complete output if needed

    
*** Settings ***
Library  CXTA
Resource  cxta.robot

Library  BuiltIn
Library  Collections

Suite Setup         Run Keywords
...                 load testbed

Suite Teardown      Run Keywords
...                 Disconnect From All Devices

*** Test Cases ***
1. CONNECT TO DEVICES
    ${status}=  Run Keyword And Ignore Error  connect to device "${DEVICES}"
    IF  '${status[0]}' == 'FAIL'
        Fail  ++UNSUCCESSFUL++ ${DEVICES} not connected
    ELSE
        set test message  ++SUCCESSFUL++ ${DEVICES} connected \n  append=True
    END

2. VERIFY SOFTWARE VERSION
    ${VERSION_OUTPUT}=  parse "show version" on device "${DEVICES}"
    ${CFG_VERSION}=  Get From Dictionary  ${VERSION_OUTPUT['version']}  version
    ${status}=  Run Keyword And Return Status  should be true  '${CFG_VERSION}' == '${EXP_XE_8K_VERSION}'
    IF  '${status}' == 'False'
        Fail  ++UNSUCCESSFUL++ Device ${DEVICES} is not running the expected version ${EXP_XE_8K_VERSION}
    ELSE
        set test message  ++SUCCESSFUL++ ${DEVICES} is running the expected version ${CFG_VERSION}\n  append=True
    END
        

    




    
DEVICES:  C8Kv-01

EXP_XE_8K_VERSION:  17.12.2

    



2. Click on the SAVE AND RUN button in the upper right-hand corner of the page

After clicking the SAVE AND RUN button, CXTM will return you to a Job File page where you can access and edit details related to the Job File, including the Execution status, Run History, and more.

Step 14 - Review the Execution status

From the test case Job File page:

Step 15 - Review the Run History

From the test case Job File page:

Step 16 - Access the Job File result

From the test case Job File page:

Step 17 - Review the Job File result

From the Job File Results page:

1. Click on the on log.html link under Artifacts



This link opens a detailed log.html report ("Jobfile Log") for all the procedures executed in the Job File.



From the Job File log:


2. Click on TEST 1. CONNECT TO DEVICES to expand
3. Click on KEYWORD ${status} = BuiltIn.Run Keyword And Ignore Error connect to device "${DEVICES}" to expand
4. Click on KEYWORD CXTA.robot.DeviceCli.connect to device "${DEVICES}" to expand



In this expanded section, you can see that CXTM connects to the C8Kv via ssh, using the iosxe Unicon plugin.


5. Click on TEST 1. CONNECT TO DEVICES again to collapse this output
6. Click on TEST 2. VERIFY SOFTWARE VERSION to expand
7. Click on KEYWORD ${VERSION_OUTPUT} = genie.libs.robot.GenieRobot.parse "show version" on device "${DEVICES}" to expand



In this expanded section, notice that the 'show version' command is run on the C8Kv device and the output is displayed here. Notice the value for version displayed in this output.

Step 18 - Change the Report Log Level to Trace

From the Report Log Level dropdown in the upper right-hand corner of the log.html report:

1. Select TRACE to change the Log Level from INFO to TRACE




2. Scroll down and click on KEYWORD ${CFG_VERSION} = Collections.Get From Dictionary ${VERSION_OUTPUT['version']}, version to expand



Notice the output of 'show version' is parsed according to the aforementioned Genie parser schema. Notice the value for the 'version' key, and notice that it's the same value stored in the CFG_VERSION variable.


3. Close the log.html tab and return to the Job File Results page.
4. Click on the Job File Verify Software Version on Devices tab in the breadcrumbs to return to the Job File page.

Continue to the next section to edit this test case to iterate over all the devices in your testbed and verify the software versions.