Question

How to troubleshoot Splunk performance issues?

Answer 1

The answer to this question would be very wide, but mostly an interviewer would be looking for the following keywords:

• Check splunkd.log for errors
• Check server performance issues, i.e., CPU, memory usage, disk I/O, etc.
• Install the SOS (Splunk on Splunk) app and check for warnings and errors in its dashboard
• Check the number of saved searches currently running and their consumption of system resources
• Install and enable Firebug, a Firefox extension. Log into Splunk (using Firefox) and open Firebug’s panels. Then, switch to the ‘Net’ panel (we will have to enable it). The Net panel will show us the HTTP requests and responses, along with the time spent in each. This will give us a lot of information quickly such as which requests are hanging Splunk, which requests are blameless, etc.