Skip to main content
Unlisted page
This page is unlisted. Search engines will not index it, and only users having a direct link can access it.
Version: 11.3.0

Data Management

HYPR AgentPass™ records the agent traffic that flows through the gateway. That record covers the requests and responses your agents exchange with LLM providers and MCP tools, the policy decisions taken on them, and the administrative changes made around them.

The Data Management page shows that record in full: every event AgentPass emits, the attributes each one carries, where those events are sent, and how long they are kept. It answers three questions, and keeps them separate because they have different answers:

  • What is recorded: the event catalog. Every event type, its attributes, and which of them identify a person.
  • Where it goes: the destinations each category of event is sent to.
  • How it leaves: per attribute, whether it is sent as clear text, hashed or not sent at all.
note

Data Management is an administrator control. Administrators and operators can read the configuration, and administrators can change it. See Permissions.

The Data Management page: four destination cards, Database, Export files, Log stream and a connected Forensics Store, above the category tree with Activity expanded, beside the detail panel for the PROXIED_REQUEST event with each attribute set to Default

Where Your Data Goes​

Open Advanced → Data Management. The strip across the top of the page names the four destinations, and shows their current state.

DestinationWhat it isRetention
DatabaseThe store behind the Request Log, the agent Activity tab and the connector request views. Always onSet by the platform, per category
Export filesJSON Lines files written on each gateway node. A write-only export for your own compliance or analytics storeA window you set: how long a file waits to be collected
Log streamOne line per event in the gateway's logs, for deployments that already collect container logsNone. See Log stream
Forensics StoreLong-term storage for your organization's events, when your deployment has one connected. See Forensics StoreSet by your deployment

Each destination follows its own schedule, so no single retention figure appears on this page. The log stream has no retention setting. A line belongs to your collector once it is written.

The database cannot be turned off. The console is built from it. The retention window shown against each category bounds it.

Export Files​

Export files are JSON Lines files written on each gateway node, one file per organization per UTC hour. AgentPass never reads these files back, and the console cannot display them. They exist for you to ship into your own systems. The Export files card shows whether they are On or Off.

To turn them on:

  1. On the Export files card, click Directory and retention settings.
  2. Select Write export files on each gateway node.
  3. Click Save changes.

The same panel holds the export settings:

  • Directory is where the files are written on each gateway node. Leave it blank to use the directory configured when the gateway was deployed. The directory must already exist on every node, because AgentPass does not create it.
  • Keep files for is how many days a file waits for a collector to fetch it, from 1 to 365. The default is 14. This is the one retention number on the page you can change. It is not a window on the data itself. Deleting a file a collector has not yet fetched would destroy the only copy, so this clock belongs to your collector, not the record.
  • Write test writes a server-generated test line to the target directory. The page reports the file it wrote, or why the test write failed. Use it to verify the destination before you save.

Inside the directory, each organization's files sit in a folder named for its organization ID. Each file name carries the hour and the node that wrote it, so nodes that share a volume never write to the same file.

Each run of records in a file ends with a footer line, {"shard_footer": {...}}, that is not an event. It carries the node, the record count and the sequence range the run covered. Have your collector skip it, or use it to check that no records are missing.

Collecting the files is an operations task

Each gateway node writes its own files. Collecting them, with a mounted volume or a log shipper for example, is a task for your deployment team.

When Export Files Stop Being Written​

When a gateway node cannot write its export files, a banner reads Event export files are not being written. It appears on this page and on the Request Log, for anyone who can view Data Management. The banner names the cause, such as a directory the gateway could not create or a file it could not write. It also gives the number of failed writes, how many nodes are affected and when the failures started.

The export files for that period are incomplete

Records still reach the database while the banner shows, so the console stays complete. The export files for the same period are missing records. Check the export directory on the affected nodes, including its free disk space and write permissions.

The banner clears once the node writes its export files again.

Log Stream​

Turn on Log stream to write one line per event in the gateway's logs. Each line is the complete event, in the same format the export files use. Anything that already collects your container logs can consume it, with no volume or cleanup job.

Both destinations can record an event twice

If you turn on both Export files and Log stream, events routed to both are recorded twice. This suits a setup such as files for a SIEM and logs for a cloud collector. The Log stream card tells you how many categories are affected.

Forensics Store​

The Forensics Store keeps your organization's events for the long term. Your deployment decides whether one is connected, and how long it keeps records. When one is connected, its card reads Connected. When none is, the card says so, and no records are sent there.

When a Forensics Store is connected:

  • Every category is sent to it by default. Clear a category's Forensics Store box to stop sending that category.
  • It can keep what the console shows. An attribute left at Default can be kept as clear text there, so its records match the console. See Leaves the gateway as.
  • It can keep whole bodies. The database keeps only the first part of a very large request or response body. When your deployment keeps whole bodies, the Request Log can fetch the whole body when you ask. See When a Body Was Cut Short.

Whole bodies are kept only while the Activity category is sent to the Forensics Store. For the PROXIED_REQUEST event, arguments_summary governs the request body and response_body the response body. A whole body is kept only while its attribute leaves as Clear text there.

Categories and Events​

Events are grouped into six categories. A category decides two things: how long its events are kept in the database, and which destinations they are sent to.

CategoryWhat it recordsKept in the database
ActivityRequests your agents made through the gateway30 days
EnforcementPolicy decisions, access denials and the approval workflow90 days
DiscoveryAI tools and SaaS usage observed across your estate30 days
AuditAdministrative changes: Data Management and platform retention settings, policy mode changes, directory provisioning and access reviews. Also export-file failures and recoveriesKept indefinitely, by design
OperationsWhat the gateway did to keep a connector working30 days
Session evaluationThe session digests Session Watch sends its evaluator agents, and their replies30 days

The platform sets database retention, not your organization. Each category in the tree shows its window, such as 30 days, marked · set by the platform. Audit records are never dropped.

Expand a category to route it. Each destination has its own checkbox. The Log stream checkbox stays disabled while the log stream is off. Hover over it to see where to turn the log stream on. The Forensics Store checkbox is selected by default when your deployment has one connected.

The category tree with Operations expanded: its 30-day database retention, the Export files, Log stream and Forensics Store checkboxes, and its six event types

What an Event Carries​

Click an event type to see what it records:

  • Published as: the event's identifier
  • Schema version: the version of its record
  • In export files: the type and event_type values the event carries in the export files
  • The attributes it carries. Click Show all to list every attribute
  • Sample record · illustrative: an example of the record a destination receives. Copy sample record copies it

Each attribute shows what it holds. Its Tags column marks whether it is always present, and whether it is tagged PII. The PII tag marks an attribute that identifies a person directly, such as an email address or a login name. It is not a judgment about user content. An attribute that may contain anything a user typed says so through its treatment instead.

The detail panel for the PROXIED_REQUEST event: its published identifier and schema version, then its attributes with Always present and PII tags, each set to Default under Leaves the gateway as

Leaves the gateway as​

Each attribute has one setting, which governs every destination outside the database.

SettingWhat it does
DefaultNo choice of your own. Each destination applies its own default for the attribute
Clear textThe attribute is sent as recorded
HashedThe attribute is replaced by a one-way digest. The same input always produces the same digest, so records can still be correlated without carrying the value
Not sentThe attribute is omitted entirely

Every attribute starts at Default. The export files and the log stream then apply the setting AgentPass gives the attribute. The Forensics Store can keep the attribute as clear text instead, so that its records match what the console shows. Hover over Default to see what each destination applies.

The Default setting on supervisor_email, with its tooltip: the Forensics Store keeps clear text, and the exports hash it

Choosing any other setting stores your choice, even one that matches a default, and it then applies to every destination. Choose Default to remove it.

This setting does not affect the database. The database holds what the gateway saw, for the category's retention window. Not sent means the attribute does not leave the gateway. It is still recorded in the database.

An attribute that is always present cannot be Not sent, because there is nowhere to put an absent value. Only text attributes can be Hashed. Where a choice does not exist, the page states the setting, instead of offering a control that would be refused.

Changes apply from now on

Changing a setting governs what is written from that point on. Records already written to a destination are not changed.

Applying One Attribute Everywhere​

The same attribute name often appears on several event types. When it does, the row offers to apply your choice to every event type that carries it, and says how many that is. "Hash supervisor email everywhere" is therefore a single action.

The count is the number of event types that carry the attribute today. An event type added in a later release starts from its own default, which is the safe one.

Save Your Changes​

Changes on this page apply when you save them. While a change is pending, the page header reads Unsaved changes. Click Save changes to apply them.

The page confirms that the data management configuration was saved, and the header reads No unsaved changes. Only administrators can save. See Permissions.

Defaults​

Until you change anything, every attribute is set to Default:

  • In the export files and the log stream, every attribute tagged PII leaves the gateway Hashed.
  • Attributes that carry user content, such as request and response bodies and tool-call arguments, are Not sent there.
  • Everything else leaves as Clear text.
  • The Forensics Store can keep attributes as clear text. Hover over Default to see what it applies.
  • Events are written to the database, and to the Forensics Store when your deployment has one connected. Export files and the log stream are off.

AgentPass sets these defaults per attribute, not your organization, so a newly added event carries a safe setting before anyone configures it. You can loosen or tighten any of them.

Platform Retention Limit​

Platform administrators set the platform retention limit. On the control plane, open Platform Data, and set the Default retention limit, from 1 to 365 days. The default is 90.

note

Editing the platform limit is a control-plane task for platform administrators, who need permission to manage organizations across the platform. Organization administrators never see this screen.

Permissions​

Data Management has its own permissions, separate from the general settings permissions.

CapabilityDefault roles
See the Data Management page and read the configurationAgentPassAdmin, AgentPassOperator
Change and save the configuration, and test the export destinationAgentPassAdmin

Users who hold neither see no Data Management entry in the navigation. Permission to edit gateway settings does not grant access to this control. Platform-administration permissions govern the platform retention limit. See Permissions by Role.