How to implement eloquent where has relationship with specific condition?

i have a case where i need to filter results by condition in 1-n relationship

so i have this 1-n relationship in User.php

public function permits()
{
    return $this->hasMany('App\Permit');
}

result i want to retrieve is users where all permits are expired (expiry on permits table = 0)

my current query is like this, this pulled several hundred wrong data because i didn't understand yet how to implement where all permits expiry = 0

$users = User::where('club_id',$this->id)->whereHas('permits',function($q){
        $q->where('expiry','<=','0');
    })->get();

current solution which is bad solution (longer time to execute), just so you guys clearer with what i want to achieve

foreach($users as $key => $user){
        foreach($user->permits as $permit){
            if($permit->expiry > 0){
                $users->forget($key); // unset from the users collection
                break;
            }
        }
    }

any help would appreciated thank you!

via Chebli Mohamed