WRITELOOP

PYTHON CONCURRENCY OPTIONS

Describes possible concurrency options that can be used on python programs

2022 July 5
  • ASYNCIO: For input/output. Cooperative, on the same process. Useful on blocking tasks e.g. get data on a database, make requests, read a file etc.

  • MULTIPROCESSING: When you need to use all available resources on a machine. It makes copies of the original process, but that needs more resources from a machine. Useful when you need to use the machine at its' limits to have faster processing.

  • THREADING: Useful when you need to run jobs that must be isolated and/or will spend most of their time idle. They are not cooperative. Then can talk among them. You can have blocking and race conditions. Useful e.g. if you need to leave something on stand-by/infinite loop.

  • CONCURRENT.FUTURES: Use that instead of multiprocessing, it is the right way to use resources when you need heavy computation.