如何将非常大的文本文件存储到数组中？

Question

I have a very large text file with more than 10.000 entries. I need to store the data into an array, because I need to check and validate all the entries before storing everything in the mySQL database.

$file_path = $filename;
$linesArray = file($file_path);    
$properties = array();

foreach ($linesArray AS $line) {
    if (strlen($line) && $line[0] == '#') {
        $pdate = substr($line, 1);
        $date = rtrim($pdate);
        $formatted = DateTime::createFromFormat('* M d H:i:s T Y',$date);
    }
    if (false !== ($pos = strpos($line, '='))) {
        $prop=array();    
        $prop[trim(substr($line, 0, $pos))] = trim(substr($line, $pos + 1));
        $lineContArray = explode("=", $line);
        $identArray = explode(".", $lineContArray[0]);
        $ident = $identArray[0];
        $type = $identArray[1];     
        $value = trim($lineContArray[1]);
        $found = 0;
        for ($i=0; $i<count($properties); $i++) {
            if ($properties[$i]['number'] == $ident) {
                $properties[$i][$type]= $value;
                $found=1;
                break;
            }
        }
        if ($found == 0) { 
            if (!empty($type)) { 
                $properties[] = array('number' => $ident, $type => $value); 
            } else { 
                $properties[] = array($ident => $value); 
            } 
        }
    }
}
var_dump($properties);

This works very well with files up to 3000 entries, but with larger files I get a blanc page.

Answer 1

Instead of dumping the array after the whole file has been processed, you could dump in each iteration the element you are adding to that array. That way there is a flow of output:

        ...
        if ($found == 0) { 
            if (!empty($type)) { 
                $property = array('number' => $ident, $type => $value); 
            } else { 
                $property = array($ident => $value); 
            }
            // dump this line's result and a linebreak:
            var_dump($property);
            echo '<br>';
            $properties[] = $property;
        }
    }
}
// not needed anymore:
//var_dump($properties);
//

If the output still does not come through, add flush() in your code.

Some comments on your code

The last of the following lines in your code will produce a run-time error when processing a line that has an equal sign, but no preceding dot:

    $identArray = explode(".", $lineContArray[0]);
    $ident = $identArray[0];
    $type = $identArray[1];

The inner loop (for ($i=0; $i<count($properties); $i++)) is very inefficient. It would be better to make an associative array keyed by $ident, containing references to your $properties array. That way your look-up does not need a loop.