Python 内存占用高

Question

Python 调用Tensorflow模型，开启一个Session预测完，执行Sess.close ， 但是内存始终没有释放

 

请问用哪种方式可以查看Python中所有变量占用内存的情况？

 

    with tf.Session(config=config) as sess:

        show_memory_info('Session')
        saver.restore(sess, ckpt_file)
        show_memory_info('restore')
        if seq == '' or seq.isspace():
            logger.info('please check sentence')
        else:
            demo_sent = list(seq.strip())
            demo_data = [(demo_sent, ['O'] * len(demo_sent))]

            show_memory_info('demo_one-')
            tag = model.demo_one(sess, demo_data)
            show_memory_info('demo_one+')
            result = get_entity_test(tag, demo_sent)
        logger.info(result)

    print('sess = '+str(sys.getsizeof(sess)))

    sess.close()

    print('sess = ' +str(sys.getsizeof(sess)))

    show_memory_info('close')

    tf.reset_default_graph()

    show_memory_info('default')

    del sess
    del word2id
    del embeddings
    del model
    del saver
    del tag2label

    print(dir())
    print(globals().keys() )
    gc.collect()
    return result

内存总体情况：

initial memory used: 295.5703125 MB
56
56
image memory used: 295.5703125 MB
ocr memory used: 296.578125 MB
分类 memory used: 296.578125 MB
解析 memory used: 296.578125 MB
ImageReport memory used: 296.5859375 MB
超参数设置 memory used: 296.5859375 MB
使用最新模型 memory used: 296.5859375 MB
build char embeddings memory used: 296.59375 MB
paths setting memory used: 297.62890625 MB
reset memory used: 297.62890625 MB
build_graph memory used: 332.00390625 MB
saver memory used: 332.66015625 MB
Session memory used: 765.99609375 MB
restore memory used: 818.26953125 MB
demo_one- memory used: 818.26953125 MB
demo_one1 memory used: 818.26953125 MB
demo_one2 memory used: 985.04296875 MB
demo_one+ memory used: 985.04296875 MB
sess = 56
sess = 56
close memory used: 985.015625 MB
default memory used: 985.015625 MB
['args', 'ckpt_file', 'ckpt_prefix', 'demo_data', 'demo_sent', 'files', 'fr', 'log_pre', 'model_path', 'modelpath', 'output_path', 'paths', 'result', 'result_path', 'seq', 'summary_path', 'tag', 'timestamp']
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__file__', '__cached__', '__builtins__', 'tf', 'os', 'time', 'json', 'BiLSTM_CRF', 'get_entity_test', 'Log', 'vocab_build', 'read_dictionary', 'random_embedding', 'tag2label_mapping', 'sys', 'gc', 'psutil', 'config', 'logger', 'predict', 'show_memory_info'])
result memory used: 981.98828125 MB
after memory used: 981.98828125 MB

 

 

Answer 1

不知道你这个问题是否已经解决, 如果还没有解决的话:

• 你可以参考下这篇文章：Python内存模型

如果你已经解决了该问题, 非常希望你能够分享一下解决方案, 以帮助更多的人 ^-^