I spent the last 2 hours trying to find my memory leak.
- Optimized the doctrine bulk processing
- Optimized my detach and all that doctrine annotation stuff
- Optimized the SQL Logger
- The script was still leaking
- Decided to comment out the logging because there wasn't much I could do anyway
Turns out that
- Over 40k iterations without logging at each n but at modulus 50, start mem: 28 mb end mem: 30mb
- Over 5k iterations with logging at each n, no modulus, start mem:28mb, end mem 38mb.
Example
# this leaks
# start mem: 28 mb end mem: 38mb, n = 5k
foreach ($this->queryData->iterate() as $j => $data):
declare(ticks = 1);
self::$currentAd++;
$this->em->detach($data[0]);
$this->logger->info(LogUtility::getMemoryUsage() . " (" . self::$currentAd .")");
if(self::$currentAd === 40000 ):
break(2);
endif;
endforeach;
# this doesn't leak
# start mem: 28 mb end mem: 30mb, n = 40k
foreach ($this->queryData->iterate() as $j => $data):
declare(ticks = 1);
self::$currentAd++;
$this->em->detach($data[0]);
if(self::$currentAd % 50 == 0):
$this->logger->info(LogUtility::getMemoryUsage() . " (" . self::$currentAd .")");
endif;
if(self::$currentAd === 40000 ):
break(2);
endif;
endforeach;
my monolog config:
handlers:
test:
type: stream
path: "%kernel.logs_dir%/immobilier/test.log"
level: debug
channels: test
console:
type: console
bubble: false
verbosity_levels:
VERBOSITY_VERBOSE: INFO
VERBOSITY_VERY_VERBOSE: DEBUG
channels: [test]
Any suggestions to correct this ?