Modifying the collection of related models after executing query

I've loaded a model along with a belongsToMany relationship from my database:

$author = Author::with('publications')->first();

/** Returns something like:

{
    "id": 3457,
    "email": "alex@trump.edu",
    "publications": {
        "1": {
            "id": 240897,
            "title": "Food left by other people at a restaurant - is it safe to eat?  A comparative review.",
            "journal": "Journal of Scrounging and Gleaning",
            "year": 2007,
            "pivot": {
                "author_id": 3457,
                "publication_id": 240897
            }
        },
        "2": {
            "id": 249196,
            "title": "Stop picking at it - you'll leave a scar!",
            "journal": "Proceedings of the International Conference on Nagging",
            "year": 2008,
            "pivot": {
                "author_id": 3457,
                "publication_id": 249196
            }
        }
    }
}

*/

I then fetch some additional data for each publication from a third-party API and merge it into my collection, which works fine.

However, I then want to sort the publications based on this third-party data, so I use a callback in sortByDesc:

$sorted = $author->publications->sortByDesc(function ($publication, $key) {
    // Blah blah blah not important how I sort
    return $blah;
});

$author->publications = $sorted->values();

According to the docs for sortBy, I need to use values if I want to renumber my results after sorting. $sorted->values() does indeed seem to be a re-keyed, sorted list, but even after I assign it to $author->publications, $author->publications still has the old keys.

Even more strange is that just running sortByDesc seems to sort the list in-place, even if I don't assign the result back to $author->publications. Why can't I assign my re-keyed collection back to $author->publications? I have a feeling that this has something to do with the nuances of relations versus attributes, but I don't know how to address this.

via Chebli Mohamed