Monitoring, retests and alerts
Scheduled monitoring, retest comparison and alert rules.
On this page
A diagnosis is a single point in time. Retest & monitoring keeps sampling with the same set of questions: a monitoring plan runs automatically on a schedule, retests compare results before and after an optimization, and alerts tell you when a metric drops noticeably or an answer gets your brand information wrong. This page explains how each part works and how to read the results.
Before you start, the brand needs a completed first diagnosis with valid measurement questions and model answers, and the question library needs measurement questions that have passed the automatic checks.
Scheduled monitoring plan
On the Retest & monitoring page, open "Scheduled monitoring" and click "Edit plan". There are two tabs:
Questions and models
- Choose the questions and models to monitor. When you save, the plan freezes the text of those questions as it is at that moment.
- If the question library changes later, the plan notes "The question library has changed; this plan still uses the questions saved earlier." Only when you tick "Update to the questions that currently pass review" does it switch to the latest formal question set.
- Repeats per question: how many times each question is asked on each model per run (1–5). More repeats narrow the confidence interval and make changes easier to tell apart; model calls scale accordingly.
Schedule and alerts
- Frequency (daily or weekly), weekday, time and time zone. The schedule takes effect once scheduled monitoring is enabled and saved; saving does not trigger a catch-up run.
- In-app alerts, and webhook notifications (see below).
For a one-off sample, use "Start measurement › Monitor now". It only samples and records answers — it doesn't change the scheduled plan or compare against earlier runs.
Run history
Every monitoring run appears in "Run history" with its scope, status and answers. You can filter by model, search questions or answers, and see the evidence behind each mention and recommendation judgement. If something failed, "Retry failed answers" retries only the failed part and keeps existing results.
Latest run changes
When a monitoring run finishes, PeakGEO compares each model's organic mention rate and organic recommendation rate with the previous run. Only answers with the same questions, models and measurement conditions, and with valid results in both runs, are compared — see Comparing runs.
Possible labels:
| Label | Meaning |
|---|---|
| N percentage points higher / lower than last round | The two runs' confidence intervals don't overlap; the change goes beyond normal fluctuation |
| About the same as last round (normal fluctuation) | The intervals overlap; the difference is within sampling error |
| Dropped two rounds in a row | See the consecutive-decline rule below |
| No longer dropping | It had been declining; this run is back at or above the previous one |
| Can't compare with the last round | No answers, or there is only one run so far |
| The monitored questions changed, so comparison restarts from this round | This run's question set differs from the previous one |
Question set changes: PeakGEO records the combination of question texts used in each run. Whenever that combination changes, the run becomes a new starting point, and numbers from before and after the change are never chained together.
Retests
A retest samples the same questions again and compares them with a baseline. Results appear in "Retest comparison".
- Start retest: choose questions and models, and confirm the baseline. You can link optimization actions or leave them out (a regular retest doesn't change action status). The default question scope comes from the "Core retest set".
- Result source: start a new retest, or link an existing monitoring run. Linking an existing run makes no new model calls, but it still has to meet the comparison conditions for questions, models, measurement conditions and sampling time.
- Automatic retest after publishing: when you register a published link in Content, a retest is created automatically for the questions linked to that action — it doesn't expand to unrelated questions.
Each metric is marked Improved, Worsened, Unchanged, Within sampling error or Can't compare yet, with an overall verdict. Retests observe changes on the same basis; they don't promise a causal improvement.
Alert rules
All alerts are collected under Alerts and risks, in three groups.
Monitor alerts: a clear drop in a retest
When a retest finishes, PeakGEO raises an alert only if:
- the recommendation rate is checked first, then the mention rate;
- the rate dropped by at least 10 percentage points compared with the baseline;
- and the confidence intervals on the two sides do not overlap.
Both conditions must hold, and one retest raises at most one alert. A drop of a few points, or one whose intervals overlap, doesn't trigger anything.
Example: the recommendation rate was 70% before the retest (28 of 40) and 30% after (12 of 40) — a 40-point drop. The intervals are about 54.6%–81.9% and 18.1%–45.4% and don't overlap, so a "Recommendation decline" alert is raised.
Consecutive declines
Monitoring tracks consecutive declines per plan × model × metric:
- It looks at the latest three runs that share the same question set, all of which must have completed normally;
- if, on the answers that are valid in all three runs, the rate is lower each time — two declines in a row — an alert opens;
- further declines are logged as "Consecutive decline still ongoing"; a run that can't be compared is logged as a gap, and results on either side of a gap are never bridged;
- once a run is back at or above the previous one, the alert ends. The console distinguishes "Back to where it was before the drop" from "It stopped dropping but hasn't recovered yet".
A consecutive-decline alert is a signal worth checking, not statistical proof of a decline. Open the answers and look: are competitors being recommended more, or has AI started citing different pages?
Factual risks
When monitoring finds that a model got your brand information wrong, or confused you with another brand, product or company, a fact-error alert opens. It runs independently of decline alerts and only ends when later answers confirm the information was corrected.
Webhook notifications
Under "Schedule and alerts", turn on "Send notifications to a webhook", pick a channel and enter the address:
- Generic webhook (JSON)
- Feishu bot
- WeCom bot
- DingTalk bot
Each brand has one webhook address, and only public http(s) URLs are accepted. In-app alerts and webhook notifications work independently.
Events that are pushed:
| Event | Priority |
|---|---|
| Answer has incorrect brand info (first detected) | P0 |
| Retest recommendation or mention rate down 15 points or more | P0 |
| Retest recommendation or mention rate down 10–15 points | P1 |
| Down two runs in a row (alert opened) | P1 |
| Consecutive-decline alert closed | P2 |
| Incorrect brand info corrected | P2 |
"Still ongoing" updates and gaps are only recorded in the timeline and aren't pushed again; the same retest is never pushed twice. Feishu, WeCom and DingTalk receive text messages in each bot's own format. The generic webhook receives JSON with the alert type, brand, plan, model, metric, priority, title, description and time, so you can route alerts by type or priority in your own systems. Messages are written in the interface language used when the settings were saved.
Where trends come from
The trend in Visibility, the per-question trends in Tracking, and the trends on the monitoring page are all calculated from the same set of real measured answers — there is no separate trend dataset. Each point on a chart is an actual measurement; gaps aren't filled with zeroes and no history is invented. Scopes can differ between measurements, so read trends together with sample counts and comparison details, and use same-scope comparisons to judge whether a change is reliable.
Related: Comparing runs · Metric definitions · Actions · Content