Tracking Agent docs

Tracking Agent troubleshooting: statuses and fixes

Most Tracking Agent problems show up as a status on the pull request or a line in the run's log, and each one says what to do next. This page lists the ones people ask about, grouped by where you see them. Every run's full log is on GitHub Actions, and the dashboard links to it.

By Ansh AgrawalUpdated

Why does my pull request's datalyze/tracking status say that?

What you seeWhyWhat to do
Tracking Agent inactive, not checkedThe workspace's subscription lapsed, the repo's DATALYZE_KEY was revoked, or Datalyze's database couldn't be readCheck Billing on the dashboard. If it's active, re-run the job; the log says which it was
Needs a person: …The rules still fail, the verifier's blockers are open after two rounds, or Claude stopped early. Nothing from that run was keptRead the report comment, then fix the branch and push, or answer the questions and approve again
Needs a person: commits after the proposal haven't been reviewed yetA build was asked for before the newest commits had their reviewWait for the review of the new push, then approve again
Needs a person: the review's proposal didn't pass its checkThe proposal file Claude wrote didn't pass check_proposalRe-run the job from the Actions tab
Awaiting approval doesn't change after ApproveThe approver lacks write access or isn't in reviewers, or the run is still queuedOpen the run from the dashboard. A refusal shows as "Needs a person" with the reason
No status at all, on a fork's pull requestGitHub gives a fork's pull request no secrets and no OIDC token, so the run can't get the App's token to post a statusPush the branch to the repo itself to have it reviewed
No status at all, or a status that doesn't countThe workflow didn't run (missing on the base branch, or Actions are off), or Datalyze couldn't be reached, so the run had no App token and its status came from GitHub Actions insteadCheck .github/workflows/datalyze.yml is on the default branch and Actions are allowed; otherwise re-run the job once Datalyze answers

Why did datalyze/rules fail?

What you seeWhyWhat to do
N problems: …The plan breaks one of its hard rules, or the code and the plan disagree for what this change touchedFix the plan or the call. Never edit _conventions.yaml or _baseline.json to make it pass
Couldn't run: …The base branch or its history wasn't checked outKeep fetch-depth: 0 on the workflow's checkout step
No tracking plan yetThe first plan hasn't mergedMerge the first plan's pull request

Why didn't the first audit start?

It starts by itself only on the push that brings the workflow to your default branch while there's no plan, never on a later push or a force-push. Use Run the first audit on the dashboard or the Setup page. If its run failed, the dashboard shows Run it again with a link to the log.

Why didn't @datalyze apply build anything?

  • The comment must start with @datalyze apply, on a pull request, not an issue.
  • The commenter needs write access, and to be in reviewers when it's set.
  • There must be a proposal on the branch, and the commits under it must have had their review.

Why doesn't our CI run on the agent's commits?

The commit was pushed without the Datalyze GitHub App's token, and GitHub doesn't start workflows for commits pushed with the default Actions token. Check the App is installed on the repo (Setup page, GitHub row). A free-form @datalyze request pushes with the workflow's own token by design; its commit still gets datalyze/rules and a review.

Why does the audit say our analytics key was refused?

The analytics tool turned the read-only key down, so the audit compared the plan and the code only. Make a new read-only key, then use Update your Mixpanel key (or PostHog, or Amplitude) on the dashboard, or Change on the Setup page's Analytics row. Setup says which key each tool needs, and the playbook chapter on event verification covers checking events arrive as planned once it works.

Why doesn't the plugin work in Claude Code?

What you seeWhat to do
"DATALYZE_KEY isn't set"Make a key on the Setup page, export it in your shell profile, and start Claude Code from a new terminal
"The Datalyze key is unknown or was revoked"Make a new key on the Setup page
The /datalyze: commands don't appearAccept the marketplace and plugin when Claude Code offers them, or add them by hand: see the plugin on a laptop
The plugin's tools don't startInstall Node.js 22 or later
A plain code review instead of the tracking reviewCheck the repo's CLAUDE.md has the Datalyze block, or run /datalyze:review directly

Why does the dashboard say a repo still runs v1?

Its workflow still uses the v1 action. v1 runs pass with "not checked" until the repo moves. Use Update the workflow on the Setup page, and save an Anthropic API key first if the repo has none. Setup covers the move.

Questions about troubleshooting

Where do I find a run's log?

On GitHub, under Actions, then Datalyze. The dashboard's Runs list links each run's log, and a failed run's row shows why it failed.

Will a broken Datalyze run block our merges?

A lapsed subscription passes with "not checked", and a failed Sheet sync never touches a pull request. On a repo that has gone live, two things hold a merge without a tracking problem: a fork's pull request, which gets no status, and a run while Datalyze can't be reached at all, which can't post the App's status; re-run that one once Datalyze is back.

Who do we contact when we're stuck?

Email ansh@joindatalyze.com with the pull request's link and the run's link. Our analytics audit is there if the tracking itself needs a person.