Cover Image
#laravel #linux

Setting up laravel queue workers using supervisor

~ 3 MINS READ
7th Dec 2021
Mohammed Omer By Mohammed Omer

Laravel queuing system provides a very easy way to process heavy work or long-running operations (sending emails for example) in the background, keeping the actual request processing time as small as possible, providing a better user experience.

In this post, we are going to setup queue workers to handle background jobs in a production environment.

Our queue driver

we are going to use the database as our queue driver since it is the simplest to configure, but feel free to use any queue driver you want. To specify that we want to use the database as our queue driver we have to change the default QUEUE_CONNECTION value in .env file from sync to database

- QUEUE_CONNECTION=sync
+ QUEUE_CONNECTION=database

Then we will create and migrate a database table for queue jobs

php artisan queue:table
php artisan migrate

Now our laravel configuration is ready for processing queues.

Creating and running queue workers using supervisor

At this point, our queue is ready but there is no process that is watching our queue and processing incoming jobs. This is what we are going to create now, a number of processes to handle our queue jobs. To achieve this, we are going to use supervisor which is a process manager that monitors and controls processes.

To install supervisor, simply run

sudo apt install supervisor

Then check if it is running as a service

sudo systemctl status supervisor.service

Now we need to create a supervisor configuration file in /etc/supervisor/conf.d directory, I will call it queue-worker.conf

sudo nano /etc/supervisor/conf.d/queue-worker.conf

add the following content to instruct supervisor to run our queue workers

make sure to replace the path /var/www/project to match your project path

[program:queue-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/project/artisan queue:work --tries=3
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=5
redirect_stderr=true
stdout_logfile=/var/www/project/storage/logs/queue.log
stopwaitsecs=3600

The above configuration will instruct supervisor to create 5 processes (you can change this number to your liking) to handle our laravel queue. And it will try to auto restart the process if they somehow stopped running, and rerun them after system reboot.

To avoid permission issues, If you are not using www-data group as owner of your project files, you have to change www-data to the user owning your project files.

To instruct supervisor to read our new changes we run:

sudo supervisorctl reread

Then we tell supervisor to run our queue workers

sudo supervisorctl update

To check if our queue workers are running we do

sudo supervisorctl status

The above command should output something like this

queue-worker:queue-worker_00     RUNNING   pid 35589, uptime 0:00:20
queue-worker:queue-worker_01     RUNNING   pid 35590, uptime 0:00:20
queue-worker:queue-worker_02     RUNNING   pid 35591, uptime 0:00:20
queue-worker:queue-worker_03     RUNNING   pid 35593, uptime 0:00:20
queue-worker:queue-worker_04     RUNNING   pid 35595, uptime 0:00:20

Now we have several processes processing our queue jobs in the background, and supervisor is making sure that they are always up!


If you liked this post consider sharing it :

You may also like

stopwatch6 MINS 
cover
By Mohammed Omer | 15th Jan 2022
#laravel #testing
stopwatch4 MINS 
cover
By Mohammed Omer | 21st Dec 2021
#laravel #performance
stopwatch4 MINS 
cover
By Mohammed Omer | 1st Dec 2021
#laravel #security #testing
stopwatch6 MINS 
cover
By Mohammed Omer | 15th Mar 2022
#laravel #tooling #php
stopwatch2 MINS 
cover
By Mohammed Omer | 27th Mar 2022
#laravel #productivity #tooling #shorts 🔥
stopwatch14 MINS 
cover
By Mohammed Omer | 22nd Jan 2022
#laravel #linux #nginx