This is a problem I'm having for quite some time now. I work for a banking institute: our service provider lets us access our data via ODBC through a proprietary DB engine. Since I need almost a hundred tables for our internal procedures and whatnot, I set up some "replication" scripts, put em in a cron and basically reloading from scratch the tables I need every morning.
When the number of records is small (approx. 50.000 records and 100 columns or so) everything goes smooth, but whenever I get a medium-big table (approx. 700.000 records), more often than not the script restarts itself (I look at my MySQL tables while the import scripts are running and I see them going 400k, 500k... and back from 1).
This is an example of one of my import scripts:
<?php
ini_set('max_execution_time', '0');
$connect = odbc_connect('XXXXX', '', '') or die ('0');
$empty_query = "TRUNCATE TABLE SADAS.".$NOME_SCRIPT;
$SQL->query($empty_query);
$select_query = " SELECT...
FROM ...";
$result = odbc_exec($connect, $select_query);
while($dati = odbc_fetch_object($result)) {
$insert_query = " INSERT INTO ...
VALUES ...";
$SQL->Query($insert_query);
}
// Close ODBC
odbc_close($connect);
?>
Any ideas?