Add a way to determine if the current thread has the import lock

[gvanrossum]

BPO	23068
Nosy	@gvanrossum, @brettcannon, @ncoghlan, @ericsnowcurrently

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-12-17.02:14:37.809>
labels = ['type-feature', 'library']
title = 'Add a way to determine if the current thread has the import lock'
updated_at = <Date 2014-12-17.23:45:12.546>
user = 'https://github.com/gvanrossum'

bugs.python.org fields:

activity = <Date 2014-12-17.23:45:12.546>
actor = 'gvanrossum'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-12-17.02:14:37.809>
creator = 'gvanrossum'
dependencies = []
files = []
hgrepos = []
issue_num = 23068
keywords = []
message_count = 8.0
messages = ['232792', '232802', '232811', '232819', '232824', '232831', '232832', '232837']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'brett.cannon', 'ncoghlan', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23068'
versions = ['Python 3.5']

[gvanrossum]

imp.lock_held() returns True if *any* thread is currently importing something. I'd like to have an API that can tell whether the *current* thread has the import lock. The implementation keeps track of this, but doesn't make the info available.

Use case: print a warning/error if a network call is made during import.

[ericsnowcurrently]

Keep in mind that the global import lock is only held long enough to create a module-level lock. The cache of module locks is found at line 166 of Lib/importlib/_bootstrap.py and the code related to module-level locking follows. So unfortunately it won't be as simple as just getting the thread info from the global import lock.

Regardless, would it be useful to expose a function in importlib.util that indicates whether or not the current thread is importing a module (actually the names of the modules it is importing)? Something like that would probably check both the global import lock and the module level locks, but make it unnecessary to expose the locks directly. Given the situation with module-level locks I image such a helper is going to be desirable.

[ncoghlan]

Perhaps an API like "importlib.util.locks_held()" that returns a list of module names?

We'd then just iterate over the _module_locks() dictionary, looking for locks where the owner matched the current thread id (alternatively, if speed was critical for Guido's use case, add a separate reverse mapping from tid to locks held)

(Unless I've missed something, we don't run user code with the global import lock held any more)

[ericsnowcurrently]

(Unless I've missed something, we don't run user code with the global
import lock held any more)

Ah. You are correct.

[gvanrossum]

Oh. I'd forgotten the PY3 situation is completely different from the PY2
situation. We're still stuck here in PY2 land where there's just the one
import lock. I guess not even ctypes can help us find out whether the
current thread is holding the import lock (which is the condition we're
trying to test for so we can print the warning). Our current hack is to
check lock_held() and if it is held wait for up to 5 second (in 10 msec
increments) to see if it goes away -- if it doesn't go away it would seem
we are holding it in the current thread and then we fail with a rude
assert. If it does go away within that time we assume some other thread was
doing some lazy importing and we hope they'll be done soon because the
whole world is waiting for them.

On Wed, Dec 17, 2014 at 6:36 AM, Eric Snow <report@bugs.python.org> wrote:

Eric Snow added the comment:

> (Unless I've missed something, we don't run user code with the global
import lock held any more)

Ah. You are correct.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue23068\>

---

[ncoghlan]

Looking at the implementation of PyImport_ImportModuleNoBlock, you should be able to invoke that via ctypes with a nonsense module name to probe for whether or not the current thread has the import lock.

A call like 'PyImport_ImportModuleNoBlock("this-is-not-a-legal-module-name")' will always fail with ImportError, but the exception message will be different if another thread holds the import lock. Specifically, it will end with "because the import lock is held by another thread".

[ncoghlan]

You'd still need to check lock_held() to see if *some* thread is holding the import lock. The non-blocking import API should then let you determine if that thread is the current one.

[gvanrossum]

Interesting. The Dropbox server team thanks you for the suggestion!

[kumaraditya303]

This seems like a Python 2 issue. @gvanrossum Can we close it now?