Eloquent Laravel sort paginate query by related model

I have the following Eloquent query:

Company::has('cars')->with(['cars', 'location' => function ($q) {
   $q->nearBy(); // *** See futher
}])->take(40)->paginate(10);

How can I achieve that the Companies are ordered by distance?

*** This is the NearBy Scope in the Location-model:

public function scopeNearBy($query)
{
    $location = json_decode(request()->cookie('location')); 
    $query->distance($location->lat, $location->lng); // ** Using package
    return $query->orderBy('distance');
}

** Which uses this package.

The calculation of the distances is OK and works when I call \App\Address::nearBy()->get().

via Chebli Mohamed