Laravel Eloquent - get count non-repetitive columns according to another column

I have a table contains appointments, every appointment has provider_id and user_id foreign keys.

I want to get counts of non-repetitive users for each provider. Here is what I did,

return DB::table('appointments')
    ->where('provider_id', 1)
    ->groupBy('user_id')
    ->count();

It returns 2 when I have two appointments with the same provider_id and user_id, but what I want is 1.

How should I solve this?

via Chebli Mohamed