Logging
Logging and Auditing
Convier produces two different types of logs: Service logs and Audit logs.
Service logs
The service logs contains a mix of access log and other internal service related log entries. It is produced using log4j. An excerpt from an access log:
2024-12-18T07:29:11: [INFO] INFO - 123.123.123.123 - [email protected] [18/Dec/2024:07:29:11 +0000] "PUT /convier/api/read/aml_platform_develop/main/count HTTP/1.1" 200 53 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0)"
2024-12-18T07:29:11: [INFO] INFO - Post-filtering 157 found links
2024-12-18T07:29:11: [INFO] INFO - 123.123.123.123 - [email protected] [18/Dec/2024:07:29:11 +0000] "POST /convier/api/async-read/aml_platform_develop/main/request HTTP/1.1" 200 18406 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0)"
The access log format may be modified using the LOG_FORMAT environment setting. See the Undertow documentation for more information.
Audit logs
While Convier supports writing audit logs to multiple sources, we recommend Azure Log Service with DCR automatically provisioned during deployment. See Installation
Audit log contents
An audit log entry contains the following fields:
- UTC Timestamp
- OIDC client ID (Azure Log Service only)
- User ID
- Project ID
- Project Branch Name
- Topic
- Request payload (JSON string)
- Response status (JSON string)
We currently do not include raw response playloads in the audit log, as they may be quite large. Audit log entries are immediately written to the database, there are not written in batches.
These are the topics being logged
connect_to_sourcesearchbrowse_dataload_objectget_linked_objectscountcompute_analyticsload_report_datareport_savereport_other_action
An example log entry on the search topic:
| field | value |
|---|---|
| project_id | aml |
| branch | main |
| timestamp | 2024-12-10T10:00:00Z |
| user_id | [email protected] |
| topic | search |
| payload | {"query":"test","fields":["CUSTOMER_NAME"],"sourceIds":["dwh"]} |
| response | {"status": "OK", "primary_related_object_ids": []} |
The field primary_related_object_ids contains references to loaded objects if any. It is as of version 0.2601.7 populated for topics
load_object: loaded objectsload_report_data: the report object and all “connected objects” (e.g. customer associated with a task)
It is not populated for topics search and compute_analytics as they are covered by previous or subsequent load_object audit entries.
Last updated on