Laravel 5.2 Authorization Policy Fails Always Unauthorized to Destroy

I can't see why I've done wrong with this policy. It is a copy of a controller action and policy with the only difference being the required classes and variables have been changed, but it always fails even though the original action passes. I've registered the policies in AuthServiceProvider, each of the different types of contacts are all owned by the same user in the DB through user_id. The removeDirectoryContact action works and removeMobileContact does not work. Composer autoload has been run after added the new policy, and their is no errors in laravel.log.

Can anyone see what the difference is or why it always fails even when I return true from the policy?

Registered Policies

protected $policies = [
    Event::class => EventPolicy::class,
    UserContact::class => UserContactPolicy::class,
    ExternalContact::class => ExternalContactPolicy::class,
];

Remove Mobile Contact Action - Always Fails

public function removeMobileContact($id)
{
    $externalContact = ExternalContact::where('id', $id)->firstOrFail(); // 404

    $this->authorize('destroy', $externalContact); // 403

    $externalContact->delete();

    return response()->json([
        'message' => trans('user.contact.removed'),
    ]);
}

ExternalContactPolicy- Always Fails

public function destroy(User $user, ExternalContact $externalContact)
{
    return $user->id === $externalContact->user_id;
    //return true; <-- also fails?
}

Remove Contact Action - Works

public function removeDirectoryContact($id)
{
    $userContact = UserContact::where('contact_id', $id)->firstOrFail(); // 404

    $this->authorize('destroy', $userContact); // 403

    $userContact->delete();

    return response()->json([
        'message' => trans('user.contact.removed'),
    ]);
}

UserContactPolicy - Works

public function destroy(User $user, UserContact $userContact)
{
    return $user->id === $userContact->user_id;
}

via Chebli Mohamed