From on September 10, 2015 21:57
IronPython seems to leak memory every time I set a variable via compiled Python scripts. The memory seems to get collected if I let the CompiledCode object go out of scope, but I want to be able to reuse one CompiledCode many times because of the speed benefits. The leak does not happen when I execute the ScriptSource instead of CompiledCode. It also doesn't happen when I set the variables in the scope via ScriptScope.SetVariable instead of setting them in Python.
Setting values this way:
d = dict()
d[1] = 1
d[2] = 1
d[3] = 1
# ......
d[100000] = 1
leaks much faster than:
d = dict()
for i in range(1, 100000):
d[i] = 1
I have reproduced the issue using the below program in 64-bit .NET 4.5 with both IronPython 2.7.4 and 2.7.5. The Script.py I used is similar to the first Python script above: I set 100000 dictionary entries individually. Other variations of the C# program I have tried that do not help with the leak: - Using LightweightScopes = false - Using ExecuteAndWrap instead of Execute - Creating a new ScriptScope for each execution - Setting the dictionary variable to be a .NET dictionary in in the program instead of declaring it in Python - Building the script from a string rather than a file
public static class Program
{
public static void Main()
{
var engine = Python.CreateEngine(
new Dictionary<string object> { { "Debug", false }, { "LightweightScopes", true } });
var scope = engine.CreateScope();
var script = engine.CreateScriptSourceFromFile(@"Script.py").Compile();
for (var i = 0; i < 100000; i++)
{
script.Execute(scope);
GC.Collect();
}
Console.ReadKey();
}
}
</string>If it helps, I ran the above leaking program in windbg and after many loop iterations, here are the biggest heap stats given by "!dumpheap -stat":
MT Count TotalSize Class Name
000007fee85b0178 326 8627 System.Boolean[]
000007fee85b0a50 125 10000 System.Signature
000007fee859b0b8 20 11424 System.Collections.Hashtable+bucket[]
000007feefa481a0 299 11960 Microsoft.CSharp.RuntimeBinder.Syntax.NameTable+Entry
000007fee858c7e8 94 12032 System.Text.SBCSCodePageEncoding
000007fe8f7ee7a8 305 12200 IronPython.Runtime.Operations.StringOps+EncodingInfoWrapper
000007feefa49788 121 12584 Microsoft.CSharp.RuntimeBinder.Semantics.AggregateType
000007fee85bc370 226 12656 System.Reflection.Emit.OpCode
000007feefa49500 62 16368 Microsoft.CSharp.RuntimeBinder.Semantics.MethodSymbol
000007feefa48730 80 17920 Microsoft.CSharp.RuntimeBinder.Semantics.AggregateSymbol
000007fe8f7ef7d0 1 22080 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[IronPython.Runtime.Operations.StringOps+EncodingInfoWrapper, IronPython]][]
000007fee85b0e98 192 23040 System.Reflection.RuntimeParameterInfo
000007fee8592690 1467 164304 System.Reflection.RuntimeMethodInfo
000007fee859d610 2992 167552 System.RuntimeType
000007fee859b320 3764 193132 System.String
000007fee859e070 240 420368 System.Int32[]
000007fe8f8a55c8 99999 2399976 Microsoft.Scripting.Interpreter.DynamicInstruction`4[[System.Object, mscorlib],[System.Object, mscorlib],[System.Object, mscorlib],[System.Object, mscorlib]]
000007feed613088 100000 2400000 System.Linq.Expressions.ClearDebugInfoExpression
000007fee859d960 100051 2401224 System.RuntimeTypeHandle
000007fee85a1300 100064 2401536 System.RuntimeMethodHandle
000007fe8f867e58 100000 3200000 IronPython.Compiler.LookupGlobalInstruction
000007feed61e560 100003 3200096 System.Linq.Expressions.AssignBinaryExpression
000007feed61e208 100009 3200288 System.Linq.Expressions.ListArgumentProvider
000007fee85ef938 100030 3200960 System.Reflection.Emit.GenericMethodInfo
000007feed61bc18 100000 4000000 System.Linq.Expressions.SpanDebugInfoExpression
000007feed61f0a8 100002 4000080 System.Linq.Expressions.MethodCallExpression2
000007fe8f80f3c8 100008 4000320 IronPython.Compiler.LookupGlobalVariable
000007fe8f95d130 99999 4799952 System.Runtime.CompilerServices.CallSite`1[[System.Func`5[[System.Runtime.CompilerServices.CallSite, System.Core],[System.Object, mscorlib],[System.Int32, mscorlib],[System.Int32, mscorlib],[System.Object, mscorlib]], mscorlib]]
000007fe8f8a8720 99999 4799952 System.Runtime.CompilerServices.CallSite`1[[System.Func`5[[System.Runtime.CompilerServices.CallSite, System.Core],[System.Object, mscorlib],[System.Object, mscorlib],[System.Object, mscorlib],[System.Object, mscorlib]], mscorlib]]
000007fe8f82b058 99999 4799952 IronPython.Compiler.PythonDynamicExpressionN
000007fe8f80a5c0 99999 4799952 IronPython.Compiler.Ast.IndexExpression
000007fe8f80a998 100000 4800000 IronPython.Compiler.Ast.AssignmentStatement
000007feed60dbe8 99999 5599944 System.Linq.Expressions.DynamicExpression3
000007fe8f809aa0 100001 5600056 IronPython.Compiler.Ast.NameExpression
000007fe8f8a85d0 100001 6400064 System.Func`5[[System.Runtime.CompilerServices.CallSite, System.Core],[System.Object, mscorlib],[System.Object, mscorlib],[System.Object, mscorlib],[System.Object, mscorlib]]
000007fee85cade0 200004 6400128 System.Reflection.Emit.GenericFieldInfo
000007feed61e3a0 200008 6400256 System.Linq.Expressions.Block2
000007fee859e0d0 299405 7185720 System.Int32
000007fe8f84c958 299552 7189248 Microsoft.Scripting.Interpreter.LoadObjectInstruction
000007fe8f801ea8 199999 7999960 IronPython.Compiler.Ast.ConstantExpression
000007feed61e720 200000 8000000 System.Linq.Expressions.Block3
000007fe8f8428b0 200000 8000000 Microsoft.Scripting.Interpreter.DebugInfo
000007feed60c040 400022 9600528 System.Linq.Expressions.ConstantExpression
000007fe8f7712d0 14 10073568 IronPython.Runtime.CommonDictionaryStorage+Bucket[]
000007feed61e458 400013 12800416 System.Linq.Expressions.BlockExpressionList
000007fee859f498 99 14199148 System.Byte[]
000007feed2c99b0 500023 16000736 System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Linq.Expressions.Expression, System.Core]]
0000000000736640 15294 19583260 Free
000007fee857f5e8 301116 35046696 System.Object[]
Copied from original issue: IronLanguages/main#1228
该提问来源于开源项目：IronLanguages/ironpython2
