Tracking Agent docs

Tracking Agent data and keys: what Datalyze sees

Datalyze sees that your workspace key was used and when, and which repo asked for a run's GitHub App token. It never keeps your code or your event names, and never sees your analytics data or what Claude said. Claude runs on your GitHub runner or laptop with your own Anthropic key, and your keys stay in your GitHub secrets. Your GitHub sign-in token stays on Datalyze's server, encrypted, and never reaches your browser.

By Ansh AgrawalUpdated

What does Datalyze see when the Tracking Agent runs?

WhatWhere it goesDoes Datalyze see it?
Your codeYour GitHub runner or laptop, and Anthropic, as Claude reads itNo
Claude's workAnthropic, with your keyNo
Analytics dataRead from your tool with your read-only key, into a snapshot on the runner, and to Anthropic with the rest of what Claude readsNo
The knowledge callDatalyze: your workspace key in, the five knowledge files out. On GitHub the plugin also sends the repo's nameThat the key was used, and when. The repo's name isn't read or kept
A run's GitHub App tokenDatalyze's server makes a token for one repo, valid for an hour, when the run proves which repo it is with GitHub's OIDC tokenWhich repo asked, counted per hour. The token isn't stored or logged
Your Datalyze workflow's versionsDatalyze's database: each version's git hash, with the repo's numeric idThe hashes only, never the file. They decide which runs get the App token
What the app reads and writes on GitHub (proposals, audits, pull requests)Read on Datalyze's server with a read-only Datalyze GitHub App token (or your own GitHub sign-in where the App isn't installed), and written with your own sign-in, between your browser and GitHubIt passes through and isn't stored or logged. To save GitHub calls, a server may keep an unchanged answer in memory for up to ten minutes
Your linked repos' last known stateDatalyze's database: each repo's setup stage, yes-or-no facts (set up, has a plan, live), counts, dates, pull request and run numbers and their states, and the latest audit's date, health and number of findingsYes, and only those facts. Never a repo's name, a title, an author, an event name, a file's text, a finding or a commit message. For repos with the Datalyze App, it's what the dashboard shows, marked with when, if GitHub doesn't answer
Secrets typed into onboarding or SetupEncrypted in your browser with the repo's key, then sent to GitHub through Datalyze's serverNo: only GitHub can open them
Google Sheet updatesYour Sheet, through Datalyze's relay or the Apps Script receiverThrough the relay, the rows pass through and aren't stored. With Apps Script, no

The dashboard and Setup read each repo's state, its runs included, on Datalyze's server with a read-only Datalyze GitHub App token, and through your own GitHub sign-in where the App isn't installed. Datalyze keeps only the facts in the row above, so a repo still shows where it stands, marked with when, if GitHub doesn't answer. The onboarding, Setup, review and sign-in pages load no analytics or third-party scripts, ours included, so nothing typed or shown on them reaches a tracking tool.

What does the Tracking Agent's licence check send?

At the start of every run, the plugin's knowledge tool sends your workspace key to Datalyze and gets back the five knowledge files: what to track, event structure, client versus server, good and bad tracking, and Datalyze's method. No code, analytics data or model output is sent. The playbook chapter on analytics in the age of AI explains why an agent needs written rules to be trusted.

With a lapsed subscription or a revoked key, no knowledge comes back, so the skills stop. In GitHub Actions the run then posts "Tracking Agent inactive, not checked" and passes, so a lapse never blocks a merge. If Datalyze's database is down, the run gets a token that can only post that passing status, and does. If Datalyze can't be reached at all, the run gets no App token, so on a repo that has gone live the pull request waits until the job is re-run.

Where do the Tracking Agent's keys live?

KeyWhere it livesWhat reads it
Workspace key (DATALYZE_KEY)A secret in each repo, made by setup and never shown; on a laptop, a key you made on the Setup page. Datalyze keeps only a SHA-256 hash of each, and several can be live at onceEach run's knowledge call
Anthropic API keyA repo or organisation secret. On a laptop, Claude Code's own sign-inClaude, on your runner
Analytics keysRead-only Mixpanel, PostHog or Amplitude keys, as repo secretsThe snapshot tool, on your runner
The Datalyze GitHub App's private keyDatalyze's server only, never in your repoDatalyze, to make a run's token for its own repo
Your GitHub sign-in tokenDatalyze's server: on your Datalyze account, and in an HttpOnly cookie your browser holds but its pages can't read, both encrypted with a key only the server has. Kept until you sign out of GitHub, GitHub refuses it, or six months pass without you using the appDatalyze's server, to make the app's GitHub calls for you

GitHub explains how Actions secrets are encrypted and passed to workflows.

What can the Datalyze GitHub App do in my repo?

It pushes the agent's commits and posts its checks, so your CI runs on those commits. Each run sends Datalyze the OIDC token GitHub gives the job; Datalyze checks the repo is linked to a workspace and the workflow running is a .github/workflows/datalyze.yml it trusts, then answers with a token for that one repo, valid for an hour.

With an active subscription the token can write code, pull requests, statuses and comments, and read checks, Actions runs and team membership (so @org/team reviewers count). It can't read or write your repo secrets or change settings. Without a subscription, or while Datalyze's database is down, the token can only post the two checks as passing. Uninstalling the App on GitHub stops it at once.

For the dashboard and Setup, Datalyze's server reads your linked repos with a second token that can only read: code, pull requests, statuses, Actions runs, and repository settings (read-only administration, to see whether the go-live ruleset is on). Until your installation accepts administration, that one read uses your own saved GitHub sign-in instead. A page uses the token only for someone in your workspace who could see the repo on GitHub when Datalyze last checked with their own sign-in: every hour, or within the last day when GitHub can't be asked. Datalyze also uses it, with no one signed in, to keep the facts it saves about your linked repos current.

The App also tells Datalyze when something changes in a linked repo: a push to the default branch or a datalyze/ branch, a pull request, a check or a workflow run. Datalyze reads the repo again then, so the facts it keeps stay current.

What can Claude do on the GitHub runner?

Claude runs through Anthropic's Claude Code action with file edits allowed and these tools: read, search, edit and write files, subagents, shell commands, and the plugin's own tools. Like any step in your workflow, its shell commands run on the runner with the job's environment, which includes your Anthropic and analytics keys.

Before Claude starts, a branch's .claude/ and .mcp.json are replaced by the base branch's, so a pull request can't change Claude's settings. After it stops, the action undoes any edit to _conventions.yaml, _baseline.json, .datalyze/config.yaml, .datalyze/context.md, .claude/, .github/, CLAUDE.md, .husky/, .gitmodules and .ripgreprc before committing.

Can anyone forge an approval on a pull request?

Not without write access to the repo. Both ways of approving are checked on the run, against the base branch's reviewers:

ApprovalCounts when
Build it in DatalyzeThe commit is pushed with the reviewer's own GitHub sign-in, changes only the proposal file, and the person who pushed it has write access and is in reviewers when it's set. The commit under it must be Datalyze's own, so nothing unreviewed is built
An @datalyze apply commentThe commenter has write access and is in reviewers when it's set, under the same rule about unreviewed commits
Anything elseNever: another comment, a review on GitHub, or a commit faked to look like a decision starts no build

A refused build posts "Needs a person" and fails the run, so it shows on the pull request.

Can anyone fake the datalyze/tracking check?

Go live makes both checks required with a repository ruleset that accepts datalyze/tracking and datalyze/rules only when the Datalyze GitHub App posted them, so no other token can turn them green.

One risk is left. Anyone who can push a branch that changes .github/workflows/ can run code with the repo's secrets on that pull request. Datalyze gives the App token only to a datalyze.yml it trusts: your default branch's copy, a copy it had in the last seven days, or the newest one Setup wrote for an admin. Give write access only to people you'd trust with your secrets, or add a CODEOWNERS entry for .github/ and .datalyze/.

Questions about data and keys

Does Datalyze store our code or tracking plan?

No. Both stay in your repo. Datalyze stores your account, your workspace and which repos are linked to it, a hash of each workspace key with when it was last used, how far you got through onboarding, each Sheet's name and last sync, the workflow hashes, and short-lived rate-limit counters.

Which model reads our code?

Claude, through your own Anthropic account in GitHub Actions, or whatever Claude Code uses on a laptop. Datalyze doesn't call a model with your code.

What happens if Datalyze is down?

Runs can't fetch the knowledge, so nothing is checked. While only Datalyze's database is down, pull requests pass with "not checked". When Datalyze can't be reached at all, a repo that has gone live holds its pull requests until the job is re-run after Datalyze is back.

How is this different from hiring Datalyze?

The Tracking Agent is software that runs in your accounts, and Datalyze sees only the key check. When you hire us for an analytics audit, people on our team work with the access you choose to give them.