The article below describes one of the ways to implement vRealize Log Insight into an environment where a (security departments) syslog server is already available and where this server ideally only receives filtered events and auditing information around changes to the filters.
Let’s start of by saying that with the introduction of ESXi6, VMware makes it possible to do Log Filtering directly on the syslog service. This however is not recommended as it will make troubleshooting potential future issues impossible.
The overview below shows you a basic setup where Log Insight is used by the operations team and the security departments syslog server only receives filtered events relevant to its purpose. This filtering can be setup in numerous ways like “everything, except …” or “only these specific matches …”. So you could easily filter out all vpxa, vMotion, scsi, etc events that are not relevant for the security department but are useful for the operations team.
- Devices log towards Log Insight;
- Log Insight applies filtering;
- Filtered events get forwarded to the security departments syslog server.
Now the biggest security concern with this setup is the fact that filters can be altered, potentially preventing events from reaching the security departments syslog server. Unfortunately Log Insight has not got a simplified audit trail log yet (this is on the feature request list, thanks to Raymond de Jong) but it does have its own internal logging available (thanks to Leon Scheltema for delivering me information around this)
Log Insights Logs are described here and one of them is the /storage/var/loginsight/runtime.log, which is used to track all run time information related to Log Insight. So if you take a closer look at this log file you can actually find two important things there:
- Login details to Log Insight;
- Registration whenever the Event Forwarding Config is changed.
With this information you can track if the filters are changed and what has been changed (more on this later), which is key information in safeguarding the configuration. The only thing you need to do is to get this information passed on towards the security departments syslog server as well.
Now let’s take a closer look at the Log Insight appliance itself as this has its own Log Insight Linux agent running which can be validated by running: pgrep liagent
Now this might sound a bit confusing but the Log Insight Appliance itself is based on Linux OS and comes with the Log Insight Linux Agent preinstalled.
Configuring this local Log Insight Linux agent gives you the possibility to forward its own internal log files towards a syslog destination, which in this case can be the security departments syslog server.
Data flow then looks like this:
- Log Insight forwards filtered events;
- Log Insight sends its own internal log(‘s), which include information about logins and modifications to the filters;
- Log Insight sends its configuration file for auditing purposes.
So by enabling these data flows the security department does not get overloaded with useless events as the operations team can filter out on their request and at the same time the security department does get information on changes to the filters.
Configuring the Log Insight Linux Agent
The configuration of the Log Insight Linux Agent is described here. As an example I will show you what I’ve added to the /var/lib/loginsight-agent/liagent.ini in order to get two files forwarded to the security departments syslog server:
[filelog|runtime-log] directory=/storage/var/loginsight include=runtime.log;runtime.log.* [filelog|loginsight-config] directory=/storage/core/loginsight/config include=loginsight-config.xml*
As described earlier on, the /storage/var/loginsight/runtime.log contains information around login details and changes to the Event Forwarding Config. Secondly the /storage/core/loginsight/config/loginsight-config.xml#xx contains Log Insights configuration, which also contains the definition of the filters.
By sending both files we can exactly see what has been changed on the filters by comparing the new configuration against the previous configuration. As an example I’ve created an filter within Log Insight that shows these differences. (do note that in my test-setup I’m using two Log Insight appliances: Appliance1 is forwarding/filtering to Appliance2)
Last but not least I will show some sample events within the /storage/var/loginsight/runtime.log regarding login attempts and it’s source:
Local User Login
[2016-12-28 11:08:29.954+0000] ["https-jsse-nio-443-exec-6"/x.x.x.x INFO] [com.vmware.loginsight.aaa.local.LocalAndActiveDirectoryAuthenticator] [Successful local user login: Local User: Name=admin] [2016-12-28 11:08:33.109+0000] ["LogSearchMaster-thread-1567"/x.x.x.x INFO] [com.vmware.loginsight.analytics.distributed.DistributedLogSearcher] [Master returned last result: token: 1cb29ba805a10904, type: AQ, ClientInfo(origin:UI_DASHBOARD, userName:admin, userHost:x.x.x.x), last-response-time: 78 msec progress: 1.0]
AD User Login to Log Insight
[2016-12-28 11:19:59.075+0000] ["https-jsse-nio-443-exec-2"/x.x.x.x INFO] [com.vmware.loginsight.aaa.krb5.KrbAuthenticator] [Attempting Kerberos login: [[ user=test01 ], [ domain=test.local ]]] [2016-12-28 11:19:59.175+0000] ["https-jsse-nio-443-exec-2"/x.x.x.x INFO] [com.vmware.loginsight.aaa.ad.ActiveDirectoryAuthenticator] [Logged in user [test01 ] : [ sam: test01, domain test.local ] in 103ms] [2016-12-28 11:20:02.480+0000] ["LogSearchMaster-thread-1572"/x.x.x.x INFO] [com.vmware.loginsight.analytics.distributed.DistributedLogSearcher] [Master returned first result: token: 60fff58804cb291c, type: AQ, ClientInfo(origin:UI_DASHBOARD, userName:test01, userHost:x.x.x.x), first-response-time: 61 msec progress: 1.0]
Whenever the Filters are changed
[2016-12-28 10:20:22.466+0000] ["DaemonCommands-thread-1253"/x.x.x.x INFO] [com.vmware.loginsight.daemon.DaemonCommandsHandler] [Going to apply configuration change #19, changedSections = EventForwardingConfig]
The post Filtering and Auditing within vRealize Log Insight appeared first on VirtualKenneth's Blog - hqVirtual | hire quality.
