# Field-Level Change Tracking

## Overview

Field-Level Change Tracking records the old and new values for each individual field that changes within a business operation, enabling precise before/after comparison at the most granular level. Each change detail is linked to a parent audit entry and captures the field name, previous value, and new value as serialized strings. This approach follows the common ERP pattern of a ChangeDetail entity associated with a parent AuditEntry by auditEntryId.

Not all field types are eligible for audit capture. Scalar fields (string, number, boolean, date, enum) and short text fields (≤ 4,000 characters) are serialized as strings and stored as-is. Long text fields (> 4,000 characters) are truncated to 4,000 characters with a `[truncated]` suffix. Rich text / HTML fields are stripped to plain text, then truncated to 4,000 characters if needed. Binary / BLOB fields are excluded from audit capture entirely and are not eligible for policy field rules. File attachment references capture only the reference metadata (file ID, filename, size), not the file content. Relation fields (foreign keys) capture the foreign key ID as a string. Collection / array fields are serialized as JSON strings and truncated to 4,000 characters if the serialized form exceeds the limit. Computed / derived fields are excluded from audit capture because they are not persisted state. Fields not listed in the entity's `auditableFields` registration are never captured regardless of type. The 4,000-character truncation limit applies after sensitivity mode processing (MASK/HASH/CAPTURE).

By storing values as serialized strings regardless of the original data type, the system maintains a uniform storage model while supporting any eligible field type. For CREATE operations the old value is null and the new value contains the initial value; for DELETE operations the old value contains the last known value and the new value is null; for UPDATE operations both values are populated but only for fields that actually changed. This design keeps the audit trail compact while providing full traceability.

Field values are subject to the sensitivity mode defined in the applicable audit policy. Fields configured as MASK store a masked representation, HASH fields store a one-way hash, and EXCLUDE fields record that a change occurred but store null for both old and new values. This ensures that change tracking does not become a secondary store of sensitive data.

## Business Purpose

- **Regulatory compliance**: Auditors and regulators require evidence of exactly what changed, when, and by whom — field-level tracking satisfies SOX, GDPR, and similar mandates that demand detailed change history
- **Sensitive data protection**: Field values are processed through sensitivity modes (CAPTURE, MASK, HASH, EXCLUDE) defined in audit policies before storage, preventing the audit trail from containing raw personal data, credentials, or payment information
- **Dispute resolution**: When discrepancies arise (e.g., pricing, quantities, addresses), field-level history provides an authoritative record of the before and after state for each affected field
- **Operational debugging**: Support teams can quickly identify which field change caused a downstream issue without reviewing entire record snapshots
- **Rollback analysis**: Knowing the previous value of each changed field enables informed decisions about whether and how to revert a change
- **Change auditing across modules**: Every transactional module (accounting, sales, purchasing, inventory) and global modules (user-management, primitives) benefit from a single, consistent mechanism for capturing field-level mutations

## Process Flow

```mermaid
flowchart TD
    A[Business Operation Occurs] --> B[Determine Operation Type]
    B --> C{CREATE / UPDATE / DELETE?}
    C -->|CREATE| D[Capture all initial field values]
    C -->|UPDATE| E[Compare old and new record states]
    C -->|DELETE| F[Capture all last-known field values]
    D --> G[Generate ChangeDetail entries\noldValue = null, newValue = initial value]
    E --> H[Generate ChangeDetail entries\nonly for fields that differ]
    F --> I[Generate ChangeDetail entries\noldValue = last value, newValue = null]
    G --> J[Apply sensitivity modes from policy]
    H --> J
    I --> J
    J --> K[Link ChangeDetail records to parent AuditEntry]
    K --> L[Persist AuditEntry + ChangeDetails]
```

```mermaid
flowchart TD
    A[Query Audit History for Record] --> B[Retrieve AuditEntry list]
    B --> C[Select AuditEntry of interest]
    C --> D[Retrieve linked ChangeDetail records]
    D --> E[Display field-by-field before/after comparison]
```

## Scenario Patterns

- **Price Change Audit**: A sales manager updates the unit price on a quotation line. The system records a ChangeDetail with fieldName = unitPrice, oldValue = "100.00", and newValue = "95.00", providing a clear trail for margin analysis and approval workflows
- **Address Correction**: A customer service representative corrects a shipping address. Each modified address field (street, city, postalCode) produces its own ChangeDetail entry, allowing reviewers to see exactly which parts of the address were changed
- **Sensitive Field Masking**: An HR administrator updates an employee's salary. The policy defines salary with sensitivity mode MASK, so the ChangeDetail stores oldValue = "***" and newValue = "***", recording that the field changed without exposing the actual values
- **Excluded Field**: A system updates a user's password hash. The policy defines passwordHash with sensitivity mode EXCLUDE, so the ChangeDetail stores fieldName = "passwordHash" with both oldValue = null and newValue = null, recording that the field changed without storing any value
- **Record Creation Baseline**: When a new purchase order is created, ChangeDetail entries are generated for every populated field with oldValue = null, establishing the initial baseline for all future change comparisons
- **Record Deletion Snapshot**: When a draft invoice is deleted, ChangeDetail entries capture the last known value of every field with newValue = null, preserving a complete snapshot of the record at the time of deletion
- **Bulk Field Update**: An administrator updates the cost center on 50 inventory items. Each item's audit entry contains a single ChangeDetail for the costCenter field, keeping the audit trail compact by recording only the field that changed rather than full record snapshots
- **No-Op Detection**: A user saves a record without modifying any fields. No ChangeDetail entries are created because no field values differ, avoiding unnecessary audit noise

## Test Cases

- Each ChangeDetail record is linked to exactly one parent AuditEntry via auditEntryId
- ChangeDetail captures fieldName, oldValue, and newValue as serialized strings
- For a CREATE operation, oldValue is null and newValue contains the initial field value
- For a DELETE operation, oldValue contains the last known value and newValue is null
- For an UPDATE operation, both oldValue and newValue are populated
- UPDATE operations generate ChangeDetail entries only for fields whose values actually changed
- Fields that did not change during an UPDATE are not recorded as ChangeDetail entries
- A save operation with no field changes produces zero ChangeDetail entries
- Multiple field changes within a single operation each produce a separate ChangeDetail record under the same AuditEntry
- fieldName is required and must be non-empty on every ChangeDetail record
- Values are stored as serialized strings regardless of the original data type (integer, boolean, date, etc.)
- Querying ChangeDetails by auditEntryId returns all field-level changes for that audit entry
- ChangeDetail records are immutable once persisted — they cannot be updated or deleted
- A CREATE operation on a record with N populated fields produces N ChangeDetail entries
- When the policy sensitivity mode for a field is MASK, oldValue and newValue store masked representations
- When the policy sensitivity mode for a field is HASH, oldValue and newValue store one-way hashes
- When the policy sensitivity mode for a field is EXCLUDE, both oldValue and newValue are null, but the ChangeDetail record is still created to indicate the field changed
- Fields without an explicit sensitivity mode in the policy default to CAPTURE (raw values stored)
- Binary / BLOB fields are excluded from audit capture and cannot be added to policy field rules
- Computed / derived fields are excluded from audit capture
- Long text values exceeding 4,000 characters are truncated with a `[truncated]` suffix
- Rich text / HTML field values are stripped to plain text before storage
- Collection / array field values are serialized as JSON strings
- Serialized values exceeding 4,000 characters are truncated with a `[truncated]` suffix
- Relation field values capture the foreign key ID as a string
- File attachment fields capture the reference metadata (file ID, filename, size), not the file content
- Truncation is applied after sensitivity mode processing

## Reference Links

- [SAP Change Documents (CDHDR/CDPOS)](https://help.sap.com/doc/saphelp_nw75/7.5.5/en-US/4d/b2ba6fbb571e57e10000000a42189b/frameset.htm)
- [Odoo Mail Tracking Values](https://www.odoo.com/documentation/19.0/developer/reference/backend/logging.html)
- [Dynamics 365 Auditing Overview](https://learn.microsoft.com/en-us/power-platform/admin/manage-dataverse-auditing)
- [AWS CloudTrail Log Event Reference](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference.html)
