Laravel queuing system provides a very easy way to process heavy work or long-running operations (sending emails for example) in the background, keeping the actual request processing time as small as possible, providing a better user experience.

In this post, we are going to setup queue workers to handle background jobs in a production environment.

Our queue driver

we are going to use the database as our queue driver since it is the simplest to configure, but feel free to use any queue driver you want. To specify that we want to use the database as our queue driver we have to change the default QUEUE_CONNECTION value in .env file from sync to database

- QUEUE_CONNECTION=sync
+ QUEUE_CONNECTION=database

Then we will create and migrate a database table for queue jobs

php artisan queue:table
php artisan migrate

Now our laravel configuration is ready for processing queues.

Creating and running queue workers using supervisor

At this point, our queue is ready but there is no process that is watching our queue and processing incoming jobs. This is what we are going to create now, a number of processes to handle our queue jobs. To achieve this, we are going to use supervisor which is a process manager that monitors and controls processes.

To install supervisor, simply run

sudo apt install supervisor

Then check if it is running as a service

sudo systemctl status supervisor.service

Now we need to create a supervisor configuration file in /etc/supervisor/conf.d directory, I will call it queue-worker.conf

sudo nano /etc/supervisor/conf.d/queue-worker.conf

add the following content to instruct supervisor to run our queue workers

make sure to replace the path /var/www/project to match your project path

[program:queue-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/project/artisan queue:work --tries=3
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=5
redirect_stderr=true
stdout_logfile=/var/www/project/storage/logs/queue.log
stopwaitsecs=3600

The above configuration will instruct supervisor to create 5 processes (you can change this number to your liking) to handle our laravel queue. And it will try to auto restart the process if they somehow stopped running, and rerun them after system reboot.

To avoid permission issues, If you are not using www-data group as owner of your project files, you have to change www-data to the user owning your project files.

To instruct supervisor to read our new changes we run:

sudo supervisorctl reread

Then we tell supervisor to run our queue workers

sudo supervisorctl update

To check if our queue workers are running we do

sudo supervisorctl status

The above command should output something like this

queue-worker:queue-worker_00     RUNNING   pid 35589, uptime 0:00:20
queue-worker:queue-worker_01     RUNNING   pid 35590, uptime 0:00:20
queue-worker:queue-worker_02     RUNNING   pid 35591, uptime 0:00:20
queue-worker:queue-worker_03     RUNNING   pid 35593, uptime 0:00:20
queue-worker:queue-worker_04     RUNNING   pid 35595, uptime 0:00:20

Now we have several processes processing our queue jobs in the background, and supervisor is making sure that they are always up!