Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-95754: Better error for shadowing stdlib module #113769

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

hauntsaninja
Copy link
Contributor

@hauntsaninja hauntsaninja commented Jan 6, 2024

This is a very common mistake for beginners. This PR tries to detect the most common case: where a file in the current directory ends up on sys.path and shadows a standard library module. (My previous PR #112577 was only marginally helpful and wouldn't help with many such errors)

The first commit in this PR is just refactoring. I'm a little nervous about running this much Python and the potential for recursion in import machinery. It's also easy to do evil things that will make bad things happen. Please let me know if you have suggestions (should I do the path munging in C?) for additional test cases or sanity checks!


📚 Documentation preview 📚: https://cpython-previews--113769.org.readthedocs.build/

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @hauntsaninja for commit d4111e5 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Jan 6, 2024
@hauntsaninja
Copy link
Contributor Author

Oh actually looks like I can use stdlib_module_names.h do get sys.stdlib_module_names in C. And I can of course do the path munging in C as well.

@hauntsaninja hauntsaninja marked this pull request as ready for review January 7, 2024 03:39
@hauntsaninja hauntsaninja force-pushed the stdlib-error branch 2 times, most recently from 4c09d8e to 9cb7291 Compare January 7, 2024 03:56
@hauntsaninja
Copy link
Contributor Author

hauntsaninja commented Jan 7, 2024

Should I disable this if sys.flags.safe_path? Also this won't currently work for python nested_folder/script.py where nested_folder contains the shadowing module... I could base this around Py_GetPath()[0] instead.

Oh hmm, Py_GetPath is deprecated, so I guess I have to get sys.path[0] (and assume it hasn't changed). This means we have to call back into Python and risk the bad recursion again. See the code from before 7f24b99 , curious if people have suggestions that are more bulletproof than module name checks.

@pablogsal
Copy link
Member

See the code from before 7f24b99 , curious if people have suggestions that are more bulletproof than module name checks.

I think sacrificing a bit of accuracy to avoid bad recursion errors and nasty conditions is very much worth it so my advice is to target maintainability without making the errors plainly worng. In other words: false negatives are a good tradeoff for maintenance.

#include "pycore_interp.h" // PyInterpreterState.importlib
#include "pycore_modsupport.h" // _PyModule_CreateInitialized()
#include "pycore_moduleobject.h" // _PyModule_GetDef()
#include "pycore_object.h" // _PyType_AllocNoTrack
#include "pycore_pyerrors.h" // _PyErr_FormatFromCause()
#include "pycore_pystate.h" // _PyInterpreterState_GET()

#include "osdefs.h" // MAXPATHLEN
#include "../Python/stdlib_module_names.h" // _Py_stdlib_module_names
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be a relative include? (I don't like relative includes.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants