gh-100870: A per-class type cache

[Fidget-Spinner]

See faster-cpython/ideas#532.

The main benefit is shrinking the inline cache for LOAD_ATTR.

One design flaw is that there is no way to reclaim space occupied by methods that no longer exist (e.g if a method is deleted). My assumption is that most normal code doesn't continuously delete methods though.

• Issue: Per-class type cache #100870

[Fidget-Spinner]

@markshannon could you please run this on the Faster CPython pyperformance machine?

[Fidget-Spinner]

We can shrink this even further by reducing type_version and keys_version in the future.

[Fidget-Spinner]

An alternative approach is to store the index into the global type cache instead. That will save us some space since we will no longer need to allocate a per-class cache. We can also make it effectively clean up dead methods id they're all in there.

[gvanrossum]

One design flaw is that there is no way to reclaim space occupied by methods that no longer exist (e.g if a method is deleted). My assumption is that most normal code doesn't continuously delete methods though.

But what about systems that repeatedly reload a module? That doesn't strike me as totally abnormal.

[Fidget-Spinner]

I can overcome that with a little more space by attaching the type cache index to the method descriptor. Then during method cleanup I set that index to NULL.

[encukou]

@Fidget-Spinner, what's the status of this PR?

[Fidget-Spinner]

@encukou I think we might need this for free-threaded eventually. But I'd rather open a new PR at that point. So time to close this,