topic Query: Location of the Log Files in the HPE Data Fabric in HPE Ezmeral Software platform

Location of the Log Files in the HPE Data Fabric

ChiragIBM — Wed, 10 Jul 2024 06:17:03 GMT

We are trying out the HPE ezmeral Platform for out requirements. We have setup the cluster and other services too. Currently, we are stuck at finding the location of the log files which are written by the Auditing.

By following this guide, I have enabld the Auditing for my cluster at all three levels, Cluster, Volume and Tables. https://docs.ezmeral.hpe.com/datafabric-customer-managed/76/SecurityGuide/Auditing-Prerequisites.html

I took the way of configuring it through CLI (maprcli). Even I enabled all the required flags, I am not able to find the location of the actual log files. I am mainly intersted in the logs of the Database tables.

Looking forward for the help.

Re: Location of the Log Files in the HPE Data Fabric

thomas945 — Mon, 16 Sep 2024 09:19:13 GMT

@ChiragIBM gm-socrates wrote:
We are trying out the HPE ezmeral Platform for out requirements. We have setup the cluster and other services too. Currently, we are stuck at finding the location of the log files which are written by the Auditing.
By following this guide, I have enabld the Auditing for my cluster at all three levels, Cluster, Volume and Tables. https://docs.ezmeral.hpe.com/datafabric-customer-managed/76/SecurityGuide/Auditing-Prerequisites.html
I took the way of configuring it through CLI (maprcli). Even I enabled all the required flags, I am not able to find the location of the actual log files. I am mainly intersted in the logs of the Database tables.
Looking forward for the help.

Hello,

Certainly! When it comes to auditing in HPE Ezmeral Unified Analytics Software, the audit logs provide a chronological record of events within your cluster. Let’s focus on locating the logs related to database tables:

Audit Log Locations:
The audit logs are stored on the MAPR-FS (MapR File System).
You can find them at the following path:
/var/mapr/local/<hostname>/audit/<port>/FSAudit.log-<date>.json
Replace <hostname> with your actual hostname and <port> with the appropriate port number1.
Specifically for Database Tables:
If you’re interested in logs related to database tables, check the audit logs in the specified location.
These logs capture actions such as table modifications, access, and other relevant events.
Additional Resources:
Refer to the official documentation for more details on accessing and interpreting audit logs: Audit Logging - HPE Ezmeral Docs.

Hope this work for you.
Best regards,
thomas945

Re: Location of the Log Files in the HPE Data Fabric

ChiragIBM — Wed, 10 Jul 2024 09:21:46 GMT

Thanks for the reply @thomas945

but I could only see one file created at that location with the name in the pattern you mentioned. The date of that file is of June. No new file is being created.

Somthing is stopping to write in that file?

Query: Location of the Log Files in the HPE Data Fabric

support_s — Mon, 16 Sep 2024 09:19:10 GMT

System recommended content:

1. HPE Ezmeral Data Fabric �� Customer-Managed 7.6.1 Documentation | HPE EZMERAL DATA FABRIC SOFTWARE LICENSING

2. HPE Ezmeral Runtime Enterprise 5.6 Documentation | Managing HPE Ezmeral Data Fabric on Kubernetes

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

Re: Location of the Log Files in the HPE Data Fabric

Shishir_Prakash — Mon, 16 Sep 2024 09:19:05 GMT

The location mentioned by Thomas is a location on maprfs/hdfs. You will need to run command like hadoop fs -ls /var/../../

[1] https://docs.ezmeral.hpe.com/datafabric-customer-managed/77/SecurityGuide/Auditing-Prerequisites.html

Audit gets stored on master node of primary container. You first need to make sure that audit at volume and data level is enabled or not. Use above doc [1] for reference.

Next you need to check the master node of primary container. In below example host1 is that node.

$ hadoop mfs -ls /tmp/t1 |head -3
Found 1 items
tr-------- Z E U U 3 mapr mapr 3 2024-07-10 06:17 0 /tmp/t1
p 2050.348.3837624 <host1>:5660 <host2>:5660 <host3>:5660

Now the relavent audit logs for this can be found /var/mapr/local/<host1>/audit/5660/* on hdfs. You can use expandaudit command doc [2] to read the audit in meaningful format.

[2] https://docs.ezmeral.hpe.com/datafabric-customer-managed/77/ReferenceGuide/r_expand_audit_logs.html

There is a btter way of storing audit in stream and reading that stream from external application. Can use below doc[3] for the same.

[3] https://docs.ezmeral.hpe.com/datafabric-customer-managed/77/SecurityGuide/StreamingAuditLogs.html

Note: If audit is enabled on every attributes of volume, it can cause huge ammount of autid logging. Make sure to enable audit only on selected attributes. Follow doc[4]

[4] https://docs.ezmeral.hpe.com/datafabric-customer-managed/77/SecurityGuide/Auditing-FilesystemTableOperations.html

Query: Location of the Log Files in the HPE Data Fabric

support_s — Tue, 16 Jul 2024 13:59:31 GMT

Hello,

Let us know if you were able to resolve the issue.

If you have no further query, and you are satisfied with the answer then kindly mark the topic as Solved so that it is helpful for all community members.

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.