Previous Topic Next Topic

Universal splunk forwarder as sidecar not showing internal s

hemanth5 Posts 2 Registration date Saturday June 1, 2024 Status Member Last seen September 28, 2024 - Sep 28, 2024 at 09:11 AM
evabrown2508 Posts 12 Registration date Saturday July 20, 2024 Status Member Last seen October 7, 2024 - Sep 30, 2024 at 09:07 AM

I have implemented a sidecar container to forward my main application logs to splunk. Have used universalsplunkforwarder image. After I deploy both my main application and forwarder seems up and running. But anyway not recieving any logs in splunk index specified. To troubleshoot splunkd log or any specific splunk internal logs are not found in /var/log path. Can someone please help how we enable this splunk internal logs?

1 response

Answer 1 / 1
evabrown2508 Posts 12 Registration date Saturday July 20, 2024 Status Member Last seen October 7, 2024
Updated on Oct 2, 2024 at 05:15 AM

It sounds like there may be a few potential issues with your Splunk Universal Forwarder configuration. Here are some steps you can take to troubleshoot and enable internal Splunk logs:

  1. Check Splunk Configuration Files:
    Ensure that the Universal Forwarder is properly configured. The main configuration files to check are:

    • inputs.conf: Verify that your log forwarding settings are correctly defined. Ensure the correct file paths and log sources are specified.
    • outputs.conf: Check that the destination (Splunk indexer) is correctly configured, including the IP address, port, and any required authentication.

  2. Verify Logs Forwarding:
    Run the following command inside your Universal Forwarder container to check if logs are being forwarded correctly:

    /opt/splunkforwarder/bin/splunk list forward-server

    This will show the forwarder’s status and whether it is successfully connecting to the Splunk indexer.

  3. Check Splunkd Logs Path:
    Internal logs (e.g., splunkd.log) are usually found under /opt/splunkforwarder/var/log/splunk/. If you cannot see any logs under /var/log, you should check this path in the forwarder container:

    cd /opt/splunkforwarder/var/log/splunk/

    If the directory doesn’t exist or is empty, Splunk may not be starting correctly or might not have proper permissions to write logs.

  4. Enable Debug Logging:
    If you're not seeing any logs at all, you can enable debug-level logging for troubleshooting by modifying the log.cfg file located at /opt/splunkforwarder/etc/log.cfg. Set the logging level to debug for more detailed output:

    [splunkd] level = debug

  5. Check for Container Permissions:
    Ensure that the sidecar container running the Universal Forwarder has the correct permissions to access the log files from your main application. You may need to mount the log file directories correctly between the containers.

  6. Restart the Splunk Forwarder:
    If you have made configuration changes, restart the Splunk Forwarder to apply them:

    /opt/splunkforwarder/bin/splunk restart

  7. Check Connectivity to the Splunk Indexer:
    Ensure that the forwarder can reach the Splunk indexer (check firewall rules, network connectivity, and the correct port).

By following these steps, you should be able to find where the issue is occurring and see if logs are being generated or sent to the Splunk index.

Reply generated by AI
0