Laravel Grouping with eager loading

I want to achieve some grouping with eager loading

When I do something like this:

$usersWithDepartments = User::with('department')->get();

this comes out:

[
    {
        "userID": 491,
        "userName": "gigi"
        "deparment": 
        {
            "departmentID": 1,
            "name": "Economy"
        }
    },
    {
        "userID": 444,
        "userName": "gigi2"
        "deparment": 
        {
            "departmentID": 1,
            "name": "Economy"
        }
    }
]

But I want to know if it's possible to do something like this:

$usersWithDepartments = User::with('department')->groupBy('deparment')->get();

in order to get a JSON like this:

[
    {
        "departmentID": 1,
        "name": "Economy",
        "users":
        [
            {
                "userID": 491,
                "userName": "gigi"            
            },
            {
                "userID": 444,
                "userName": "gigi2"            
            }            
        ]
    }
]

I know I could do Department::with('users') but I want to keep the same query above in order not to do another query call

if it's not possible. Are any other work arounds?

via Chebli Mohamed