Laravel: Synchronisch queue system

I am trying to set up an API which uses a queue system on another server to handle the requests. Let me start what I am trying to accomplish without the queue system (no authorization to keep it simple): Using Postman for example making a GET request to the URL http://ift.tt/2qJb75V would return a JSON string like

[
    {
        "id": 1,
        "name": "some name",
        ...
    },
    {
        "id": 2,
        "name": "some other name",
        ...
    }.
    ...
]

The code in routes/api.php would be something like:

<php

Route::get('/products', ProductController@index');

And the code in app/Http/Controllers/ProductController.php:

<php

namespace App\Http\Controllers;

class ProductController extends Controller
{
    /**
     * Return the products.
     *
     * @return \Illuminate\Http\Response
     */
    public function index()
    {
        // Logic to get the products.
        return $products->toJson();
    }
}

What I would like to accomplish is that all the business logic is processed on another server which runs multiple workers. What follows is my reasoning behind this.

• Security: In the case we would get hacked, it will most probably be the client server and not the worker server. Since the last one has all the business logic, the hacker will, in worst case scenario, only be able to get incoming and outgoing request data.
• Multiple workers: Getting the products will probably not take a long time, but there might be other requests which need more time to process. The user making the request will have to wait for the result in most situations. However, other users making a call should not have to wait for this. Therefore, another worker can take this request and handle the job.

This would be the workflow how I see it:

• All free workers are constantly polling for a job on the queue

  1. User makes request
  2. Client Server takes request data and put it on the queue
  3. A worker takes a job from the queue and handles it
  4. The worker returns the result to the client server
  5. The client server returns the result to the user

Below A small drawing to clear things out.

  User 1
      _   \
     |     \
       \    \   1.
        \    \  request
         \    \                                  -------------
  result  \    \                                /             \
  5.       \    \                               |  Worker     |
            \    _|                             |  Server     |
             \                                  |  ---------  |
                   -------------                | /         \ |
                  /             \               | | Worker1 | |
                  |  Client     |            /  | |         | |  \
                  |  Server  2. |           /   | \         / |   \
                  |  ---------  |          /    |  ---------  |    \
                  | /         \ |         /     |  ---------  |     \
                  | | Queue   | |        /      | /         \ |      \     ---------
                  | |         | |      |_       | | Worker2 | |       _|  /         \
                  | | Job A   | |               | |         | |           | DB      |
                  | | Job B   | |   3.  <-----  | \         / |  ----->   | Server  |
                  | |         | |       _       |  ---------  |       _   |         |
                  | \         / |      |        |  ...        |        |  \         /
                  |  ---------  |        \      |  ---------  |      /     ---------
                  \             /         \     | /         \ |     /      ---------
                   -------------           \    | | WorkerN | |    /      /         \
              _               4. ?          \   | |         | |   /       | Other   |
               |                                | \         / |           | Servers |
             /    /                             |  ---------  |           |         |
  1.        /    /                              \             /           \         /
  request  /    /                                -------------             ---------
          /    /
         /    /  result
        /    /   5.
       /    /
          |_
   User 2

In the documentation of Laravel I came across queues, which I thought would easily do the trick. I started experimenting with Beanstalkd, but I assume any queue driver would do. The issue I stumbled upon is that the queue system works asynchronously. As a consequence, the Client server simply carries on without waiting for a result. Unless I am missing something, there seems to be no option to make the queue system work synchronously.

When looking further into the Laravel documentation I came across broadcasting. I am not sure if I understand the concept of broadcasting a 100%, but from what I do understand is that the receiving seems to happen in Javascript. I am a backend developer and would like to stay clear from Javascript. For some reason it feels wrong to me use javascript here, however I am not sure if that feeling is justifiable.

Looking further in the documentation I came across Redis. I was mainly intrigued by the Pub/Sub functionality. I was thinking that the Client server could generate a unique value, send it with the request data to the queue and subscribe to it. Once the worker is finished it can publish the result with this unique value. I was thinking that this could cover the missing part for step 4. I am still not sure how this would work in code, if this logic would work in the first place. I am mainly stuck with the part where the client should listen and receive the data from Redis.

I might be missing something very simple. Know that I am relatively new to programming with PHP and to the concept of programming over the world wide web. Therefore, if you find that the logic is flawed, or it is all too farfetched, please give me some pointers on other/better methods.

An extra FYI, I have heard of Gearman, which seem to able to work both synchronously and asynchronously. I would like to stay clear from that, however, since my aim is to use the tools provided by Laravel to the fullest. I am still learning and am not confident enough to use too many external plugins.

via Chebli Mohamed