Skip to content

Improve text log view performance with cached indexing and virtualized rendering#12707

Open
vaishanth-rmrj wants to merge 1 commit into
rerun-io:mainfrom
vaishanth-rmrj:fix/text_log_view_lag
Open

Improve text log view performance with cached indexing and virtualized rendering#12707
vaishanth-rmrj wants to merge 1 commit into
rerun-io:mainfrom
vaishanth-rmrj:fix/text_log_view_lag

Conversation

@vaishanth-rmrj
Copy link
Copy Markdown

Related

  • No linked issue yet.

What

Improve text log view performance for large numbers of log entries by replacing the eager per-frame expansion path with a cached, virtualized one.

Previously, the text log view eagerly expanded all matching TextLog data into a flat in-memory list, sorted the entire list, recalculated row heights from full text bodies, and rendered through the old heterogeneous row path. That approach works for small recordings, but it becomes expensive when a recording contains many log rows or large multi-line messages.

This PR introduces a store-level TextLogCache plus a per-view TextLogProjectionState so the view can work from lightweight row metadata instead of eagerly materializing every visible row body on every frame.

Main changes:

  • Add TextLogCache, which indexes text-log chunks once and stores lightweight per-row metadata:

    • row id / instance id
    • timepoint
    • log level
    • color
    • explicit line count
  • Support incremental append-only updates in the cache, while falling back to a rebuild for non-additive store edits.

  • Add a per-view projection that:

    • tracks the currently included entities and active timeline
    • preserves the legacy stable row ordering
    • applies log-level filtering over cached row handles
    • caches prefix sums for multi-line row heights
    • reuses cached discovered levels for blueprint fallback UI
  • Switch rendering to egui_table with a table delegate that:

    • resolves text bodies lazily for visible rows only
    • uses cached prefix sums for row positioning
    • preserves current-time autoscroll behavior
    • preserves current-time marker rendering
  • Simplify TextLogSystem so it remains responsible for discoverability of TextLog entities, while the view owns cached loading/projection logic.

  • Add unit tests covering:

    • sparse carry-forward behavior for level/color metadata
    • lazy body resolution from indexed chunks
    • additive vs non-additive cache revisions
    • explicit newline counting
    • filtered projection prefix sums
    • sorted merge behavior
    • scroll target behavior
    • current-time marker placement

This is intended to preserve existing user-visible behavior while making large text-log views much more responsive.

Local validation:

  • cargo check -p re_view_text_log
  • cargo test -p re_view_text_log

Both passed locally.

Breaking changes:

  • None.

Migration guide:

  • Not needed.

@oxkitsune oxkitsune self-requested a review March 30, 2026 20:00
@vaishanth-rmrj
Copy link
Copy Markdown
Author

Hi @oxkitsune, any updates on this PR? Let me know if there are any issues. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant