I've got a few huge XML files, and I cut a few rows out, so I could have a manageable-sized file on which to test my parsing script, written in php. There is a lot of nesting in the XML file, there are a lot of columns, and there are a lot of blanks, so writing the script was this huge ordeal. Now, I'm hitting my php memory limit on the full-sized XML files I want to parse.
Now, one thing I've considered is temporarily upping the php memory limit, but I need to rerun this script every well... week or so. Also, I don't have the best system. Running it hot and setting it melt is an all-to-real possibility and one of my "perfect storms".
I also considered attempting to learn a new language, such as perl or python. I probably could use to know one of these languages, anyway. I would prefer to stick with what I have, though, if only in the interest of time.
Isn't there some way to have php break the XML file up into manageable chunks that won't push my machine to its limit? Because every row in the XML file is wrapped by an ID column, it seems like I should be able to cut to the nth row closure, parse what was sliced, and then sleep, or something?
Any ideas?
