Laravel - Creating a relationship from Pivot Table Field

I cannot find a solution, likely to how I am phrasing the question so here I am. I have a model called Invoice and it has the following relationship:

public function manifests(){
    return $this->morphedByMany(carrier_manifest::class, 'invoiceable')->withPivot(['amount','rate_id','notes']);
}

As you can see, in the pivot, I have a table called rate_id. I would like to be able to add a relationship to another model based on the value of the rate_id (the model just being called ChargeRates). Is there a way I can do this in order to access a field in the ChargeRates model called label?

Thanks!

via Chebli Mohamed