Mar 25, 2020

Python GIL, lock, mutex, semaphore

Lock
#####
A lock allows only one thread to enter the part that's locked and the lock is not shared with any other processes.

Mutex
#######
Mutex as the name means mutual exclusion.
A mutex is used to guard shared data such as a list, dict.
A mutex allows only a single thread to access a resource or critical section.
A mutex is the same as a lock but it can be system wide (shared by multiple processes).
Same thread can acquire and release lock
Owned by thread

Semaphore
##########
Semaphore, is used for limiting access to a collection of resources.
A semaphore does the same as a mutex but allows x number of threads to enter, this can be used
E.g., to limit the number of cpu, io or ram intensive tasks running at the same time.
      Pool of DB connections to be handed out to requesting threads
Different threads can call acquire and release on the semaphore
No ownership

GIL
####
Ref: https://realpython.com/python-gil/

The Python interpreter can only execute a single thread at a time
If your machine has one or hundred processors, the Python interpreter is only able to run a single thread at a time using a single processor.
Two threads on a machine with two available processors can't be executed in parallel each running on a single CPU.
This lock is known as the Global Interpreter Lock using CPython (to avoid deadlocks)
Python implementations which overcome the GIL altogether. E.g., Jython, IronPython and pypy-stm.

Python uses reference counting for memory management. (unlike garbage collection in other languages)
It means that objects created in Python have a reference count variable that keeps track of the number of references that point to the object. When this count reaches zero, the memory occupied by the object is released.

This reference count variable can be kept safe by adding locks to all data structures that are shared across threads so that they are not modified inconsistently.

But adding a lock to each object or groups of objects means multiple locks will exist which can cause another problem—Deadlocks (deadlocks can only happen if there is more than one lock). Another side effect would be decreased performance caused by the repeated acquisition and release of locks.

The GIL is a single lock on the interpreter itself which adds a rule that execution of any Python bytecode requires acquiring the interpreter lock. This prevents deadlocks (as there is only one lock) and doesn’t introduce much performance overhead. But it effectively makes any CPU-bound Python program single-threaded.


No comments:

Post a Comment