dimanche 25 juin 2017

Best way to retrieve huge data amount then sort them using php

In my DB I have about 350k data with many fields and related tables. My users can search these data and the average amount of results is between 50 to about 130k.

Actually I just retrieve the data and doing a pagination system and ordering the data for the best with sql.

But I'm working on a much better sorting and display system. For example for a user getting about 10k of result: In a first time I retrieve the data in my DB. Then I need to sort all of these data with a custom algorithm in order to have the best display. For about 10k it takes to much time for the response to come.

I mean that I can not just retrieve the 50 first data, sort them, load the 50 next data, sort them, etc. If I do it, my sorting system becomes inefficient because for example my 64th data might be in 27th position if I had looped on all the data to sort them.

Hope I have been clear enough. If you have any idea or suggestion, I'll be glad to hear it. Thank you in advance.



via Chebli Mohamed

Aucun commentaire:

Enregistrer un commentaire