Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

High memory consumption from trace engine #16111

Open
3 tasks done
mattzeunert opened this issue Jul 11, 2024 · 2 comments
Open
3 tasks done

High memory consumption from trace engine #16111

mattzeunert opened this issue Jul 11, 2024 · 2 comments
Assignees
@adamraine
Labels
bug P1

Comments

@mattzeunert
Copy link
Contributor

FAQ

URL

No shareable repro

What happened?

Node ran out of memory when running Lighthouse.

The memory profiler shows that invalidationsForEvent uses 328 MB.

Screenshot 2024-07-11 at 22 44 16

The invalidationsForEvent map has around 900 entries, but some of them have 10,000 or so SyntheticInvalidation events.

Screenshot 2024-07-11 at 22 43 58

What did you expect?

Lighthouse should run without crashing the Node process.

What have you tried?

No response

How were you running Lighthouse?

node

Lighthouse Version

12.1.0

Chrome Version

No response

Node Version

No response

OS

No response

Relevant log output

No response
@adamraine
Copy link
Member

I think your assessment is correct. Invalidation events are extremely noisy and have led to problems in the past like #15869.

The invalidationsForEvent object is in a part of the trace engine Lighthouse doesn't use so we should consider turning it off somehow if we can't optimize the memory usage.

@adamraine adamraine assigned adamraine and unassigned brendankenny Jul 12, 2024
@jackfranklin
Copy link

Hey @mattzeunert thanks for the report. Is it possible for you to share the website you were using that caused this high memory? Just to aid our debugging. I absolutely understand if it's private and you are unable to though. Thanks!

copybara-service bot pushed a commit to ChromeDevTools/devtools-frontend that referenced this issue Jul 12, 2024
@jackfranklin
RPP: remove SyntheticInvalidation
80d1944 
I was investigating this bug in LH
[GoogleChrome/lighthouse#16111] where the
trace engine OOMs, and the reporter notes that the InvalidationsHandler
map is 328MB in size.

During my initial investigation I realised we were creating synthetic
invalidations for no real purpose; all the information exists in the
events as is, and instead we can just create a type to group them and
everything gets much more simple.

I don't expect this to resolve the problem in that bug report, but it is
firstly cleaner to not create synthetic events if we don't need to, and
it does also mean we are now not going to allocate a new object per
every invalidation we want to store, so I expect this to be a positive
change.

Bug: 352680159
Change-Id: I4817774ce692c0dd73eb2156389148d7b756208c
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/5702081
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Andres Olivares <andoli@chromium.org>
Reviewed-by: Andres Olivares <andoli@chromium.org>
@adamraine adamraine mentioned this issue Aug 5, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adamraine adamraine

Labels
bug P1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jackfranklin @brendankenny @mattzeunert @adamraine @devtools-bot