Symfony2中的Doctrine多次插入超时

Question

I am trying to create many of the same object in my database, to insert some users, more exactly. First, I tried flushing the entity manager after every loop, but it didn't work. After some research, I found out that I can do it with batches, so I tried flushing every 15 loops, but I still get timed out after 30 seconds(around 180 queries). Shouldn't these be fast, or almost instant for a number as small as 200-250? How can I make this not time out and insert them faster?

What my function does is make a request to randomuser.me and retrieves a json of users. I retrieve that data into an array and use it to fill my users info with it. I also have a $fields array somewhere that has certain medical fields(won't post it here as it is just an array of plain text). It also adds a related settings entry for the user entry(containing the field in which the medic works and his picture).

 public function generateAction()
    {
        $em = $this->getDoctrine()->getManager();
        $generated = 0;

        $url = "https://randomuser.me/api/?results=250&nat=us";
        $str = file_get_contents($url);
        $medics = json_decode($str);


   $medicsgruop = $this->getDoctrine()->getRepository('MedAppBundle:Group')->findOneBy(array('name' => 'medics'));

    if (!$medicsgruop) {
        $medicsgruop = new Group('medics');
        $medicsgruop->addRole('ROLE_MEDIC');
        $em->persist($medicsgruop);
        $em->flush();
    }
        $batchSize = 15;
        foreach ($medics->results as $i => $medic) {

            if ($this->fields[$i]) {
                $field = $this->fields[$i];
            } else {
                $field = array_rand($this->fields);
            }

            $fname = $medic->user->name->first;
            $lname = $medic->user->name->last;

            $image = $medic->user->picture->large;
            $email = $medic->user->email;

            $user = new User();
            //user
            $user->setUsername($lname.$fname);
            $user->setFirstname($fname);
            $user->setLastname($lname);
            $user->setEmail($email);
            $user->setEnabled(true);
            $user->setPlainPassword($fname.$lname);
            //group



            $user->addGroup($medicsgruop);
            $em->persist($user);
            //MEDIC SETTINGS
            $medsett = new MedicSettings();
            $medsett->setField($field);
            $medsett->setProfile($image);
            $medsett->setMedic($user);
            $em->persist($medsett);


            if (($i % $batchSize) == 0) {
                $em->flush();
                $em->clear();
            }
        }
        $em->flush();
        $em->clear();
        return $this->render(
            '@MedApp/Admin/generatemedics.html.twig',
            array('generated' => $generated, 'medics' => 'asd')
        );
    }

When using the batch method, around 40 users are being added to my database, but it stops after the maximum execution time is reached. What am I doing wrong?

Answer 1

Using malcolm's suggestion to use the Client command from symfony. The command still runs slow, though. Like ~1 second per user. And they are inserted in the DB 15 at a time, per batch.

class GenMedicsCommand extends ContainerAwareCommand
{
    /**
     * {@inheritdoc}
     */
    protected function configure()
    {
        $this
            ->setName('doctrine:generate:medics')
            ->setDescription('Generate medics group, users, group-users, medic settings')
            ->addArgument(
                'number',
                InputArgument::REQUIRED,
                'How many medics to generate?'
            );
    }

    /**
     * {@inheritdoc}
     */
    protected function execute(InputInterface $input, OutputInterface $output)
    {
        $number = $input->getArgument('number');
        $this->generate($number,$output);

    }


    protected $fields = array(
        'Abdominal Radiology Radiology-Diagnostic',
        'Addiction Psychiatry Psychiatry',
        'Adolescent Medicine Pediatrics'    
    );

    //generate random medics for the win
    public function generate($nr,   OutputInterface $output)
    {


        $em = $this->getContainer()->get('doctrine.orm.entity_manager');
        $doctrine = $this->getContainer()->get('doctrine');

        $generated = 0;

        $url = "https://randomuser.me/api/?results=".$nr."&nat=us";
        $str = file_get_contents($url);
        $medics = json_decode($str);

        $medicsgruop = $doctrine->getRepository('MedAppBundle:Group')->findOneBy(array('name' => 'medics'));

        if (!$medicsgruop) {
            $medicsgruop = new Group('medics');
            $medicsgruop->addRole('ROLE_MEDIC');
            $em->persist($medicsgruop);
            $em->flush();
        }

        $batchSize = 15;
        foreach ($medics->results as $i => $medic) {
         //   set_time_limit(2);


            $randkey = array_rand($this->fields);
            $field = $this->fields[$randkey];

            $fname = $medic->user->name->first;
            $lname = $medic->user->name->last;

            $image = $medic->user->picture->large;
            $email = $medic->user->email;

            $user = new User();
            //user
            $user->setUsername($lname.$fname);
            $user->setFirstname($fname);
            $user->setLastname($lname);
            $user->setEmail($email);
            $user->setEnabled(true);
            $user->setPlainPassword($fname.$lname);
            //group


            $user->addGroup($medicsgruop);

            $em->persist($user);
            //MEDIC SETTINGS
            $medsett = new MedicSettings();
            $medsett->setField($field);
            $medsett->setProfile($image);
            $medsett->setMedic($user);
            $em->persist($medsett);
            $output->write(array($i+1,'|',$fname,'|',$lname,'|',$field,'|',$email,'|',$image));
            $output->writeln('');
            if (($i % $batchSize) == 0) {
                $em->flush();
                //  $em->clear();
            }
        }
        $em->flush();
        $em->clear();


        $output->writeln('done');
    }

}