Updated for version: 0.2412.10

Logging and Auditing

Convier produces two different types of logs: Service logs and Audit logs. While the service log is produced by default, the service log has to be explicitly enabled. Convier does not produce audit logs by default, because that means starting to store some customer data (queries), which requires additional considerations.

Service logs

The service logs contains a mix of access log and other internal service related log entries. It is produced using log4j. An excerpt from an access log:

2024-12-18T07:29:11: [INFO]  INFO - 123.123.123.123 - user@acme.com [18/Dec/2024:07:29:11 +0000] "PUT /convier/api/read/aml_platform_develop/main/count HTTP/1.1" 200 53 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0)"
2024-12-18T07:29:11: [INFO]  INFO - Post-filtering 157 found links
2024-12-18T07:29:11: [INFO]  INFO - 123.123.123.123 - user@acme.com [18/Dec/2024:07:29:11 +0000] "POST /convier/api/async-read/aml_platform_develop/main/request HTTP/1.1" 200 18406 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0)"

The access log format may be modified using the LOG_FORMAT environment setting. See the Undertow documentation for more information.

Audit logs

The audit log is written to a SQL database of your choice. It is enabled by setting environment setting INTERNAL_AUDIT_LOG=true (the default is false, as explained above).

By default, the audit log is written to the same internal database containing project configuration (environment setting JDBC_CONNECTION_STRING). To write to another database, use the configuration settings:

  • INTERNAL_AUDIT_LOG_JDBC_CONNECTION_STRING
  • INTERNAL_AUDIT_LOG_JDBC_USERNAME
  • INTERNAL_AUDIT_LOG_JDBC_PASSWORD

An audit log entry contains the following fields:

  • UTC Timestamp
  • User ID
  • Project ID
  • Project Branch Name
  • Topic
  • Request payload (JSON string)
  • Response status (JSON string)

We currently do not include raw response playloads in the audit log, as they may be quite large.

These are the topics being logged

  • connect_to_source
  • search
  • load_object
  • get_linked_objects
  • count
  • compute_analytics
  • load_report_data
  • report_save
  • report_other_action

An example log entry on the search topic:

field value
project_id aml
branch main
timestamp 2024-12-10T10:00:00Z
user_id user@acme.com
topic search
payload {"query":"test","fields":["CUSTOMER_NAME"],"sourceIds":["dwh"]}
response {"status": "OK"}