sortBy not working in my case in laravel5.1

I'm trying to use the sortBy() of laravel but it has no effect for my situation. I want to sort the question_number field in the question_numbers table. I'm using belongsToMany relationship here. Please see my code for reference:

    $undeleted = function($query){
        return $query->where('deleted', 0);
    };

    $quiz = Quiz::with([
        'multiple_choices.question_numbers' => $undeleted
    ])->find($id);

    $quiz_items = collect($quiz->multiple_choices);

    $quiz_items->sort(function($quiz_item){
        return $quiz_item['question_numbers.question_number'];
    });

    dd($quiz_items);

The result of the code above is just list of multiple choices without sorting.

via Chebli Mohamed