Skip to content

Develop-Python/procrastinate

Procrastinate: PostgreSQL-based Task Queue for Python

Deployed to PyPIDeployed to PyPIGitHub RepositoryContinuous IntegrationDocumentationCoverage badgeMIT LicenseContributor CovenantDiscord

Procrastinate is looking foradditional maintainers!

Procrastinate is an open-source Python 3.8+ distributed task processing library, leveraging PostgreSQL to store task definitions, manage locks and dispatch tasks. It can be used within both sync and async code, has Django integration, and is easy to use with ASGI frameworks. It supports periodic tasks, retries, arbitrary task locks etc.

In other words, from your main code, you call specific functions (tasks) in a special way and instead of being run on the spot, they're scheduled to be run elsewhere, now or in the future.

Here's an example (if you want to run the code yourself, head to Quickstart):

# mycode.pyimportprocrastinate# Make an app in your codeapp=procrastinate.App(connector=procrastinate.SyncPsycopgConnector()) # Then define tasks@app.task(queue="sums")defsum(a, b): withopen("myfile", "w") asf: f.write(str(a+b)) withapp.open(): # Launch a jobsum.defer(a=3, b=5) # Somewhere in your program, run a worker (actually, it's usually a# different program than the one deferring jobs for execution)app.run_worker(queues=["sums"])

The worker will run the job, which will create a text file named myfile with the result of the sum 3 + 5 (that's 8).

Similarly, from the command line:

export PROCRASTINATE_APP="mycode.app"# Launch a job procrastinate defer mycode.sum '{"a": 3, "b": 5}'# Run a worker procrastinate worker -q sums

Lastly, you can use Procrastinate asynchronously too (actually, it's the recommended way to use it):

importasyncioimportprocrastinate# Make an app in your codeapp=procrastinate.App(connector=procrastinate.PsycopgConnector()) # Define tasks using coroutine functions@app.task(queue="sums")asyncdefsum(a, b): awaitasyncio.sleep(a+b) asyncwithapp.open_async(): # Launch a jobawaitsum.defer_async(a=3, b=5) # Somewhere in your program, run a worker (actually, it's often a# different program than the one deferring jobs for execution)awaitapp.run_worker_async(queues=["sums"])

There are quite a few interesting features that Procrastinate adds to the mix. You can head to the Quickstart section for a general tour or to the How-To sections for specific features. The Discussion section should hopefully answer your questions. Otherwise, feel free to open an issue.

Note to my future self: add a quick note here on why this project is named "Procrastinate" ;) .

Where to go from here

The complete docs is probably the best place to learn about the project.

If you encounter a bug, or want to get in touch, you're always welcome to open a ticket.

About

PostgreSQL-based Task Queue for Python

Resources

License

Code of conduct

Contributing

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python89.6%
  • PLpgSQL9.8%
  • Other0.6%