lundi 29 mai 2017

Laravel factories failing upon random but unique element generation with many elements

I'm developing the backend of a service in which users can set a "like" (in my case called "check") on multiple other entities (multimedia, activities, other users).

Those checks must be unique per checkable entity and performer of the check, as shown here:

Schema::create('checks', function (Blueprint $table) {
        // Artificial auto-increment primary key (PM is set by default using increments() method)
        $table->increments('id');

        $table->unsignedInteger('user_id');

        // Automatically adds checkable_id and checkable_type
        // checkable_type represents the table in which search for the checkable_id
        //  and can be equal to 'multimedia', 'users', 'activities' or 'tags'
        $table->morphs('checkable');

        // Setup creation timestamp functionality
        // updated_at is not useful here, that's why we are not using timestamps()
        $table->timestamp('created_at')->nullable();

        $table->unique(['user_id', 'checkable_id', 'checkable_type']);
    });

Everything works fine when I test the system, but I have some problems when I try to fill the database with random data using factories. I enforced the uniqueness constraint in the factory, but sometimes the code I used do not catch the duplicate selection and I receive a PDO exception for violating the uniqueness contraint. Here the enforcement code:

$factory->define(Check::class, function (Faker\Generator $faker) {
    // Get the checker user
    $checkUserId = User::withTrashed()->get()->random()->id;

    // Chose where the check have been done
    $checkableType = $faker->randomElement(['multimedia', 'users', 'activities']);
    // Get all checkables of the specified type
    $checkables = call_user_func('App\\' . studly_case(str_singular($checkableType)) .
        '::withTrashed');

    // Apply type-related restrictions
    switch ($checkableType) {
        // Attachments multimedia and categories avatars cannot be checked
        case 'multimedia':
            $checkables = $checkables->where('model_type', '!=', 'messages')
                ->where('model_type', '!=', 'categories');
            break;
        // An user cannot check himself
        case 'users':
            $checkables = $checkables->where('id', '!=', $checkUserId);
            break;
    }

    // Retrieves data from database
    $checkables = $checkables->get();

    // Picks a random checkable id until it finds one not already checked by the current user
    do {
        $checkableId = $checkables->random()->id;
        $check = DB::table('checks')->where([
            ['checkable_id', $checkableId],
            ['checkable_type', $checkableType],
            ['user_id', $checkUserId]
        ])->first();
    } while ($check != null);

    return [
        'user_id' => $checkUserId,
        'checkable_id' => $checkableId,
        'checkable_type' => $checkableType,
        'created_at' => $faker->dateTime,
    ];
});

I tried in two different ways and got two different problems.

First option:

for ($i = 0; $i < self::CHECK_NUM; $i++) {
    factory(Check::class)->create();
}

In this way everything seems to work fine for the uniqueness constraint, but the generation just stop at some point if there are too much elements to create (currently it tries to generate 1900 random checks) without saying a thing somewhere after 1300 elements (it's never the same number). I discovered this by logging in with a second SSH connection and directly monitoring the checks number on the database when the seeding was occuring: it kept growing and at some point it just stops, but the seeder is still running.

Second option:

factory(Check::class, self::CHECK_NUM)->create();

I liked this the best, but at some point it just throws a PDO error like this

Integrity constraint violation: 1062 Duplicate entry '22-44-activities' for key 'checks_user_id_checkable_id_checkable_type_unique'

and stops the seeder. Repeating the same procedure as before, I saw that the number of checks in the DB stays to 0 until the error occurs, then it gets to a variable number (around 150). I guess that internally Laravel does not really write records on the DB until it finished generating them, but with some debugging I found that the constraint enforcing code I put in place actually works some times preventing duplicates, which means that the DB facade can see in some way the records created but not yet written and this confuses me even more about why the check does not work all the time (given that the generation script doesn't run on multiple thread, as far as I know).

Any ideas/insight on how Laravel manage these things low level or on how to tackle those problems? I'm expecially interested in the second option, but if I can understand the problem also with the first one I'd be happier :D



via Chebli Mohamed

Aucun commentaire:

Enregistrer un commentaire