Laravel: Column not found when using "WithCount"

I am trying to order by using following priority:

1. Show casts first having given city
2. Show casts first having given prefecture
3. Display remaining casts which does not have provided city or prefecture

My database table structure is:

Casts:

id
...
...
shop_id

Shop:

id
..
..

Location

id
city
prefecture

Available Location

id
location_id
shop_id
..

In order to filter the cast from city/district,

1. It first needs a shop
2. Shop needs available_location
3. Available_location is then filtered from the location's city or district

For that scenario, I have used the following condition:

$casts = Cast::filterNonBusy()->attendancePresent()->where('is_deleted', 0)->with('review', 'castImage', 'shop', 'shop.location');
        if (isset($request->city) || isset($request->prefecture)) {
            $city = $request->city;
            $prefecture = $request->prefecture;

            $locationHavingCityIds = Location::where('city', $city)->pluck('id')->toArray();
            $locationHavingPrefectureIds = Location::where('prefecture', $prefecture)->pluck('id')->toArray();

            if (!empty($locationHavingCityIds)) {
                $casts = $casts->withCount(['shop', 'shop AS has_filtered_city' => function ($shopQuery) use ($locationHavingCityIds) {
                    return $shopQuery->whereHas('availableLocations', function ($availableLocationQuery) use ($locationHavingCityIds) {
                        $availableLocationQuery->whereIn('location_id', $locationHavingCityIds);
                    });
                }])->orderBy('has_filtered_city', 'DESC');
            }

            if (!empty($locationHavingPrefectureIds)) {
                $casts = $casts->withCount(['shop', 'shop AS has_filtered_prefecture' => function ($shopQuery) use ($locationHavingPrefectureIds) {
                    return $shopQuery->whereHas('availableLocations', function ($availableLocationQuery) use ($locationHavingPrefectureIds) {
                        $availableLocationQuery->whereIn('location_id', $locationHavingPrefectureIds);
                    });
                }])->orderBy('has_filtered_prefecture', 'DESC');
            }
        }

If the cast has related city, it gives me has_filtered_city as 1. Else it gives me has_filtered_city as 0

And I ordered according to the has_filtered_city.

The problem occurred when the cast has no associated city, it gives me an error with:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'has_filtered_city' in 'order clause' (SQL: select casts., cp.cast_id, cp.depreciated_price from casts inner join cast_prices as cp on cp.cast_id = casts.id and cp.id = (select id from cast_prices cp1 WHERE cp1.cast_id = casts.id ORDER BY depreciated_price DESC LIMIT 1) where is_published = 1 and (busy_till is null or busy_till < 2020-12-09 04:26:15) and not exists (select * from reservations where casts.id = reservations.cast_id and (status_id = 1 or status_id = 2)) and exists (select * from attendances where casts.id = attendances.cast_id and attendance_status = 1) and is_deleted = 0 and exists (select * from cast_prices where casts.id = cast_prices.cast_id order by name asc) and (not exists (select * from reservations where casts.id = reservations.cast_id) or (select count() from reservations where casts.id = reservations.cast_id and status_id = 5) < 3) and is_published = 1 order by has_filtered_city desc, has_filtered_prefecture desc, depreciated_price desc limit 20 offset 0)

Where did I go wrong in has_filtered_city? It sometimes work perfectly but sometimes throws as error saying "Unknown Column: has_fiiltered_city".

via Chebli Mohamed