4.4.2.7. Troubleshooting
To troubleshoot the Network Device Compliance Use Case, we have outlined the following sections. Please review these topics carefully to determine if they can help resolve your issue.
4.4.2.7.1. NDC flow diagrams
This section includes a detailed diagram illustrating the execution flow of a Network Device Compliance test. It corresponds to a single job, representing the test conducted for one device from the complete list managed by the scheduler.
The execution flow can help you in troubleshooting this use case:
The rule execution flow happens as follows:
4.4.2.7.2. Compliance Logs
The log and analytics management model for Network Device Compliance is based on the ELK stack (Elasticsearch, Logstash, and Kibana). Together, these tools create an information pipeline: Logstash collects and transforms data, Elasticsearch serves as the storage database, and Kibana provides a graphical interface to visualize the data.
During the execution of the Network Device Compliance tests, logs are generated from collected information, including events, errors, test results, and tags for network devices. As data is added to the log files, it is sent to the Analytics server via Filebeat. Logstash processes this incoming data based on predefined log types, indexing and storing it in Elasticsearch in JSON format for querying.
The Network Device Compliance execution generates the opennac-netdev-compliance.log log file.
This log is in JSON format and contains information related to all tests conducted on the devices. It is transmitted to the Analytics server via Beats.
Network Device Compliance log path:
/var/log/opennac/opennac-netdev-compliance.log
Example of a Network Device Compliance log :
2020-09-15T09:58:43.195888+02:00 on-dc-dev opennac-netdev-compliance[7329]:
{"deviceId":"6852218816923963698","deviceIp":"10.10.36.49","deviceIpManagement":"0.0.0.0",
"deviceBrand":"Cisco","deviceModel":"Generic","deviceHostname":"CiscoDemoCenter",
"deviceLocation":"","deviceVersion":"15.0(2)SE5","deviceMacaddress":"38:1C:1A:AA:F9:80",
"deviceSerialNumber":"FCW1828A5M2","devicePurchaseDate":"2020-09-25T00:00:00+02:00",
"devicePurchaseOrder":"","deviceWarranty":"2020-09-16T00:00:00+02:00",
"deviceMaintenance":"2020-09-18T00:00:00+02:00","deviceEol":"2020-09-21T00:00:00+02:00",
"deviceTags":"CGF_IOS_LAN_PORTS, CGF_IOS_SWITCH_IMPORTANCE_2, CGF_IOS_SWITCH_IMPORTANCE_3,
CGF_IOS_SWITCH_IMPORTANCE_4, CGF_IOS_SWITCH_IMPORTANCE_5, CGP_SWITCH_INFO,
CRF_IOS_AAA_SESSION_ID_UNIQUE, CRF_IOS_ACCESS_CLASS_LINE_VTY, CRF_IOS_BANNER_MOTD,
CRF_IOS_BOOT_SYSTEM, CRF_IOS_CDP_DISABLED, CRF_IOS_CHECK_VERSION,
CRF_IOS_CLOCK_SUMMER_TIME, CRF_IOS_DOMAIN_LOOKUP_DISABLED, CRF_IOS_EXEC_TIMEOUT_LINE,
CRF_IOS_GRATUITOUS_ARP_DISABLED, CRF_IOS_IP_BOOTP_SERVER_DISABLED, CRF_IOS_IP_CEF,
CRF_IOS_IP_HTTP_SERVER_DISABLED, ","complianceTestGroupId":"6790331328198873089",
"complianceTestGroup":"IOS_SWITCH_IMPORTANCE_5","complianceTestId":"6789876735128113163",
"complianceTest":"IOS_LINE_CONFIGURATION","complianceRule":["IOS_EXEC_TIMEOUT_LINE",
"IOS_LOGIN_AUTHENTICATION_LINE","IOS_ACCESS_CLASS_LINE_VTY",
"IOS_TRANSPORT_INPUT_LINE_VTY"],"complianceTestConditionFullfiled":"all",
"complianceStatus":0,"complianceResultTag":"CTF_IOS_LINE_CONFIGURATION"}
The log includes information about the device itself, the tags assigned to it, and the details of the tests performed.
Once the log file reaches the Analytics server, it will pass through the Logstash pipeline, where it will be processed using input, filter (for specific file types), and output files. These configurations are located in the logstash conf.d file.
After the NDC logs have been processed by Logstash and stored in Elasticsearch, they will appear in a format similar to the one below, reflecting the modifications made during the pipeline:
These logs can be accessed in Analytics > Discover, by selecting the NDC data in the opennac_nd-* index.
Finally, this information is stored in Elasticsearch, allowing Kibana to utilize it for generating graphs in the test results dashboards at Analytics > Network Device Compliance.