Skip to main content
Version: 11.5.0

Session Watch

Agent session forensics lets you read what an agent did. Session Watch has an AI agent read it for you.

You tag one of your existing cloud agents as an evaluator, and attach it to a session. While the session runs, HYPR AgentPass™ sends the evaluator batched summaries of its activity, called digests. The evaluator reviews each digest, and returns findings: short observations, each pointing at the request it concerns. It never blocks the session, and never interacts with the person running it.

Use Session Watch when reading every session yourself does not scale. Fifty people running coding agents produce more activity than any reviewer can follow. Session Watch has something look at all of it, and tells you what is worth your attention.

Review is unannounced

Attaching an evaluator begins continuous, unannounced review of a named colleague's activity. AgentPass does not notify the person running the session. Whether that is appropriate is your organization's decision. Consult whoever owns your acceptable-use and employee-monitoring policies before you enable it broadly.

How Session Watch Works​

  1. Tag an agent as an evaluator. Any Anthropic cloud agent or AWS Bedrock AgentCore agent can be one. See What an Evaluator Is.
  2. Choose which sessions it reviews — one session by hand, or every session that matches a rule. See Attaching an Evaluator to a Session and Watching Sessions Automatically.
  3. AgentPass sends digests. While the session runs, AgentPass batches its activity into digests and sends one to the evaluator at a regular interval.
  4. The evaluator replies with findings. AgentPass reads each reply on the next delivery and records the findings it contains.
  5. You triage. Findings appear on the Session Watch page and on the session itself. You open the request a finding cites, act on it, and dismiss it.

How often, how much and for how long are all governed by Session Watch Settings.

What an Evaluator Is​

An evaluator is an existing cloud agent that you have tagged, not a special kind of agent. It keeps its own model, system prompt and connector tools. Its system prompt must ask for the evaluator reply format, so AgentPass can record its findings. See Adding the Evaluator Prompt.

Those tools are how a finding reaches a person. AgentPass sends no notifications of its own. An evaluator that can post to Slack, open a Jira ticket or send mail is what turns "a finding was recorded" into "somebody was told."

Tagging changes one thing, and only for Anthropic cloud agents: the agent no longer receives the provider's built-in sandbox tools (bash, file operations and web fetch). Those run inside the provider, where AgentPass cannot see or govern them, and an evaluator reads content that a repository or a fetched web page can influence. Its MCP connector tools are unaffected, because they route back through the gateway, where your policies apply.

An agent tagged as both an evaluator and a policy author loses its connector tools as well. It keeps reviewing sessions, but can no longer post to Slack, open a ticket or send mail.

Tagging an Agent​

You need Agents: Edit, which Admin and Operator hold.

For a step-by-step setup, including deploying an AgentCore evaluator in AWS and importing it, see Setting Up an Evaluator Agent.

  1. Open Agents, and choose Cloud in the location filter above the list.
  2. In the agent's row, click ⋮, then Manage. The agent's page opens.
  3. In the page header, turn on the Evaluator switch.
  4. Read Tag this agent as a session evaluator?, and click Tag as evaluator.

The confirmation differs by provider, because the consequence does. Both remind you to make sure the agent's prompt asks for the evaluator reply format.

Agent typeWhat changes
Anthropic cloud agentLoses the provider's sandbox tools. Connector tools are unaffected
AWS Bedrock AgentCore agentNothing changes at AWS. But AgentCore enforces no spend limit of its own, so the gateway's limits are the only ceiling

A tagged agent shows an Evaluator badge. To find evaluators in a long list, choose Evaluator in the Roles filter on Agents. You can reverse tagging at any time. Removing the tag does not stop reviews already running.

Adding the Evaluator Prompt​

AgentPass records only findings written in its reply format, so the evaluator agent's own system prompt must ask for that format. Each digest also lists the fields it expects, but the agent's prompt decides how it answers. The console offers a sample prompt that includes the format. For an AWS Bedrock AgentCore agent deployed from the AgentCore evaluator agent template, the template's prompt already includes it; ask your HYPR representative for the template. See Setting Up an Evaluator Agent.

  1. Open Agents, and choose Evaluator in the Roles filter. In the evaluator's row, click ⋮, then Manage.
  2. Go to its system prompt:
    • For an Anthropic cloud agent, click Edit configuration, and expand Advanced.
    • For an AWS Bedrock AgentCore agent, go to Harness configuration.
  3. Above the System prompt box, in Start from a sample prompt, choose Security session evaluator. This sample reviews a red-team or penetration-testing agent's session, reports findings in the evaluator reply format, and flags out-of-scope targets.
  4. If the box already holds a prompt, Replace the system prompt? asks to confirm. Click Replace.
  5. Edit the prompt for your own review, and keep the part that asks for the reply format.
  6. Save the agent. For an Anthropic cloud agent, click Save. For an AgentCore agent, click Save to AWS.

A watch that is already running keeps the evaluator session it started with, so it may not pick up the new prompt. Detach the evaluator, and attach it again.

Start from a sample prompt above the System prompt field in the Advanced settings of a cloud agent

The Session Watch Page​

Activity › Session watch lists every coding-agent session in the window you choose, whether each one is being reviewed, and every finding evaluators have reported.

The Session Watch page: the Last 24h time range button, Refresh and Settings in the header; tiles for people watched, live watched sessions, open findings with the highest severity, and watches needing attention; and the Sessions tab grouped by session, with the All and Not watched pills, a search box, and a table of sessions showing person, watch status and open findings

The header holds three controls:

  • Time range: the period the page covers, such as Last 24h. See Choosing a Time Range.
  • Refresh: reloads the tiles and the list you are looking at.
  • Settings: opens Session Watch Settings. Shown only to roles that manage Session Watch.

Below the header, the Sessions and Findings tabs sit above the tiles.

The page keeps the tiles and their counts current while it is open, but it never rearranges the list you are reading. Select Refresh to bring the list up to date.

Choosing a Time Range​

The time range button in the header shows the period the page covers. It starts at Last 24h. You can choose any range inside the last 30 days.

  1. In the page header, click the time range button.
  2. Under Last, pick a window: 1h, 6h, 24h, 3d, 7d or 30d.
  3. Optional: Under Refine, narrow the range inside that window. Drag across the bar to select a range, drag either edge to adjust it, or drag the middle to move it.
  4. Click Apply.

Beneath the bar, a readout shows the start and end of your selection, and a chip shows how long it is. To refine from the keyboard, focus either edge and use the left and right arrow keys. Hold Shift for larger steps. Cancel, or Esc, closes the selector without changing anything.

What you apply decides how the range behaves:

  • A whole window stays rolling. Last 7d covers the seven days up to the moment you apply it, and moves forward each time you select Refresh.
  • A narrower selection is fixed. The button shows its start and end, such as Oct 20, 8:25 PM – Oct 21, 2:10 PM.

The Findings tab and its entity graph use the whole range. The tiles and the Sessions tab use only its start, and run to now. For a fixed range, the Sessions tab says so: Sessions from … to now.

The range stays as you switch tabs, and it is part of the page's address, so a copied link opens the same period.

Tiles​

Four tiles summarize the window. Selecting a tile opens the rows behind it; selecting it again clears it.

TileShowsSelecting it
People watchedHow many people have at least one watched session, out of everyone with sessions. Its label adds how many sessions are watched, such as People watched · 12 of 40 sessionsGroups the Sessions list by person
Live watched sessionsHow many watched sessions were active in the last five minutes. While any are live, the label names the evaluators watching them, such as Live · Security reviewer watchingShows live watched sessions only
Open findingsHow many findings nobody has dismissed. When there are any, the label adds the highest severity, such as Open findings · highest High, and the tile takes that severity's colorOpens the Findings tab
Watches needing attentionHow many watches stopped for a reason an administrator should act on. When there are any, the label reads Need attention followed by the statuses, such as Need attention · FailedShows those sessions only

Sessions​

The Sessions tab is the default. It lists every coding-agent session in the window — watched or not — so you can see what is reviewed and what is not.

Use Group by to choose the shape of the list:

Two pills narrow the list, each with its count: All and Not watched. The tiles cover live sessions and watches that need attention. The Open findings tile opens the Findings tab. To find one person's or one agent's sessions, type an email address or agent name in Filter by email or agent and press Enter.

When grouped by session, each row shows:

ColumnShows
SessionThe agent's session name, with a green dot when it was active in the last five minutes. Beneath it, a short session identifier and how long ago it was active
PersonThe person who ran the session. Select the address to list only that person's sessions. For traffic with no identifiable person, Unattributed and the connector's owner
WatchThe watch's status and the evaluator, with +1, +2 and so on when more than one is watching. Hover the status for why it is in that state. An unwatched session shows Not watched, and roles that manage Session Watch also see a Watch link that opens the session's Evaluator agents tab
FindingsThe highest open severity and the number of open findings. When there are none, which of the four states in If No Findings Arrive the session is in

Watched sessions with open findings come first, then live sessions, then the most recent. Select Load more at the bottom to see more.

The Session Drawer​

Select a session row to open a summary of it in a drawer beside the list:

  • the session identifier, how long ago it was active, the person, and who is watching it;
  • Open findings, highest severity first, each linking to the session's Evaluator agents tab;
  • Digests, the most recent ones sent, each with its time, how many events it carried and what came back;
  • Open session, which opens the full session page.

Roles that manage Session Watch also see Watch on an unwatched session, and Detach on a watched one. Detaching asks you to confirm with Yes, detach evaluator, or No, keep evaluator to cancel. It stops the evaluator reviewing the session, and keeps the findings it has already reported.

The session drawer opened from a Sessions row: the session's person and evaluator, its open findings, its latest digests, and the Open session and Detach buttons

Watch Statuses​

Each evaluator on a session shows one of these statuses. Only Watching is live. Every other status is final, and says why the watch stopped.

StatusMeaning
WatchingReceiving the session's activity.
DetachedAn administrator stopped the watch. Activity already sent cannot be recalled.
EndedThe session went quiet for long enough that the watch closed, and its final activity was sent. See Close after quiet in Limits.
Timed outThe watch reached its age, digest or event limit. See Limits.
Not startedA tenant limit was already reached, so no evaluator was attached. It is recorded so an unwatched session does not look like one with nothing to report.
FailedDelivery failed repeatedly and the watch stopped. The evaluator's row on the session shows the recorded error. Needs attention.
HaltedTwo people's activity appeared under the same session identifier, so watching stopped rather than risk reporting one person's work to another. Needs attention.
Spend limit reachedThe evaluator reached its spend cap. It is not raised automatically. Needs attention.
Agent disabledThe evaluator agent was disabled or archived while it was watching. Needs attention.

Grouped by Person​

One row per person, answering is my organization's agent activity actually being reviewed? Each row shows the person, their coverage, how many of their sessions are watched (and how many are live), and their open findings. Select a row to list that person's sessions.

CoverageMeaning
WatchedEvery session in the window has an evaluator.
PartlySome do, some do not — for example, a rule created after earlier sessions had started.
Not watchedNo session has an evaluator.
Cannot watchThese sessions cannot be reviewed as things stand. Hover for why; the reason often names the setting that would change it.

People with sessions nobody is watching are listed first. Traffic with no identifiable person appears as its own row per connector, labeled Unattributed with its owner. Such a row is not an individual, and nothing on it should be read as one person's activity.

The Sessions tab grouped by person: an unattributed connector that cannot be watched, one with a named owner that is not watched, a person who is not watched, a person who is partly watched and one who is fully watched, each with session counts and open findings

Grouped by Evaluator​

One row per evaluator that watched a session in the window. Select a row to list the sessions it watched.

The Sessions tab grouped by evaluator: one row per evaluator showing how its watches started, how many sessions and people it is watching, the digests it was sent and when the last was sent, its open findings with the highest severity, and the watches that need attention, above the page footer with the concurrent watch count, the evaluator spend for the window and the time of the last update
ColumnShows
EvaluatorThe evaluator's name, linked to the agent, and how its watches started — by rule, by hand, or both. If your role cannot view that kind of agent, or the agent no longer exists, a short identifier appears instead
WatchingHow many sessions and people it is watching, and how many of those sessions are live
DigestsHow many digests it was sent, and when the last one went
Open findingsThe highest open severity and the number of open findings
Needs attentionAny of its watches that stopped for a reason you should act on, or replies it sent that were not in the expected format

Findings​

The Findings tab lists every finding from every watched session. See Reviewing Findings Across Sessions.

Beneath both tabs, a line shows how many watches are running against the concurrent limit, the evaluator spend for the window, and when the page last updated. If the session list is running behind recent activity, the footer says by how much.

Who Can Do What​

Session Watch capabilities follow these permissions.

CapabilityPermission
See sessions, findings and digestsSession watch: View, with Request log: View at the same or a wider scope
Attach, detach, dismiss findings, create rules and change settings, including owners and limitsSession watch: Manage
Tag an agent as an evaluatorAgents: Edit
Draft a policy from an entity in the findings graphSession watch: View and Policies: Add

The built-in roles have this access.

Built-in roleSession Watch
AdminSees every session and finding, and manages Session Watch
Operator, Security analyst and AuditorSee every session and finding. Attaching, detaching, dismissing and settings stay with Admin
Supervisor and Model governance ownerNo access. The page is not in their navigation

Reading findings needs both the Session Watch and the Request Log permission, because a finding describes request content. If you hold only one, AgentPass tells you which is missing, instead of showing an empty list. See Permissions by Role.

Supervisor holds no Session Watch permission, and holds Request log: View only for its own records. A supervisor's own sessions may be under review, and showing them the review attached to their work would defeat its purpose.

A custom role can be given Session Watch at own scope. Such a role sees every session, but only the watches it attached itself and the findings those watches reported. The Open findings and Needs attention views count only its own watches.