chore: implement telemetry persistence and daily active logging#769
Merged
chore: implement telemetry persistence and daily active logging#769
Conversation
- Add `FilePersistence` to manage local telemetry state (last active date). - Integrate persistence into `ClearcutLogger` to track and log daily active users. - Implement `logDailyActiveIfNeeded` to handle Daily Active events. - Explicitly call `logDailyActiveIfNeeded` in `main.ts` during server startup. - Update `ClearcutLogger` to use the shared `logger` module for error reporting. - Add comprehensive tests for `FilePersistence` and `ClearcutLogger` logic.
OrKoN
reviewed
Jan 14, 2026
OrKoN
reviewed
Jan 14, 2026
OrKoN
approved these changes
Jan 14, 2026
OrKoN
reviewed
Jan 14, 2026
OrKoN
reviewed
Jan 14, 2026
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 27, 2026
… by default (#805) This PR completes the telemetry system by implementing the transport layer for `ClearcutSender`. It enables actual HTTP communication with the Clearcut backend, handling event batching, rate limiting, and reliable delivery, including robust shutdown handling. **Key Changes:** * **HTTP Transport**: Implemented `fetch`-based transport sending `POST` requests to the Clearcut HTTP server. * **Event Batching**: Events are now buffered and flushed periodically (default: 15 minutes) or on shutdown. * **Reliability & Rate Limiting**: * **Server-Side Backoff**: Respects `next_request_wait_millis` from server responses to handle rate limiting dynamically. * **Transient Error Retries**: Failed requests (5xx, 429) result in events being requeued for the next flush. * **Request Timeouts**: Enforced 30s timeout on requests to prevent hanging processes. * **Session Rotation**: Automatically rotates session IDs every 24 hours. * **Safety & Stability**: * **Buffer Overflow Protection**: Caps the buffer at 1000 events to prevent memory leaks, dropping oldest events if necessary. * **Optimistic Removal**: Prevents race conditions and duplicate events during shutdown by optimistically removing events from the buffer before sending. * **Testing Improvements**: * **E2E Robustness**: Updated E2E tests to use a mock web server instead of relying on the logger to log specific lines. **Implementation Roadmap:** These changes finalize the planned telemetry architecture: 1. **CLI & Opt-out Mechanism ([Merged](#757 2. **Logger Scaffolding & Integration ([Merged](#758 3. **Persistence Layer ([Merged](#766 4. **Watchdog Process Architecture ([Merged](#769 5. **Transport, Batching & Retries (This PR):** * Finalized `ClearcutSender` with HTTP transport, batching, and server-directed backoff strategies. --------- Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements the persistence layer for the telemetry system. It introduces
FilePersistencefor local state management and integrates it withClearcutLoggerto support "Daily Active" metric. I have decided not to sendfirst_time_installationevents since we can deduce them fromdaily activeevents where thedays_since_last_activewill be-1for that case.Implementation Roadmap:
This is the third in a series of PRs designed to implement the telemetry system:
--usage-statisticsflag and transparency logging.ClearcutLogger: Implemented the main logging entry point.logToolInvocationandlogServerStarthooks intomain.tsto capture events.ClearcutSender: Introduced a transport abstraction.FilePersistence: Implemented a local file-based state manager to persist thelastActivetimestamp.ClearcutLoggerto automatically detect and logdaily_activeevents (withdays_since_last_activecalculation) vialogDailyActiveIfNeeded.ClearcutSenderexecution to a dedicated watchdog process to ensure reliable event transmission even during abrupt server shutdowns.ClearcutSenderwith actual HTTP transport logic, including event batching and exponential backoff retries.