This article explains , how to use celery in production and also follows best practices of celery.
Hello all. Today we will see how to run celery in production (daemon mode).
In local system we used to run with command like this
celery -A celery_app worker -l INFO
In production we need to run celery in daemon mode and celery need to be started automatically whenever system reboots.
we are going to use celeryd
to run celery in production.
$ sudo apt-get install python-celery celeryd
Prerequistes:
save this code as testapp.py
from __future__ import absolute_import
from celery import Celery
import sys
import time
celery_url = "amqp://%s:%s@%s//" % ("guest", "guest", "0.0.0.0")
celery_app = Celery("testapp", broker=celery_url,
backend='rpc://')
@celery_app.task()
def add(x,y):
print "hi from task add", x, y
return x+y
def main(x,y):
"""
This is main function gets called initially while running
"""
print "Let's add %s and %s" % (x, y)
res = add.apply_async(args=[x, y])
time.sleep(3)
if res.ready():
print "Result from task %s" % res.result
if __name__ == '__main__':
if len(sys.argv) < 3:
print "Usage: python testapp.py 1 2"
sys.exit(0)
x = int(sys.argv[1])
y = int(sys.argv[2])
main(x,y)
/etc/default/celeryd
If you have installed celeryd. it created daemon script in /etc/default/celeryd.
change/add these lines in /etc/default/celeryd
ENABLED="true" # enable daemon mode
CELERYD_CHDIR="/home/bala/personal/python/celerytest/main"
# change above line to the directory which contains our testapp.py
CELERYD_OPTS="--app=testapp --loglevel=info --time-limit=300 --concurrency=8"
$ sudo service celeryd restart
Note: change CELERYD_CHDIR
to your directory which has our testapp.py
Now to run open two tabs in terminal
sudo service celeryd start
which results like this
celery multi v3.1.6 (Cipater)
> Starting nodes...
> w1@bala: OK
# you can check the logs in /var/log/celery/w1.log
Output
Let's add 2 and 3
Result from task 5
You can find complete source code in my repository here
That's it . Thanks for reading . Happy coding !!!
Your comment
Your comment has been saved and will be subject to review before it is displayed to other visitors of the website. Thank you for your comment!
Leave a comment
(Note: Comments are moderated)