Hi Mike,
Thank you for the script and test data. I have carried out several tests on Linux and Windows 8.1, with similar results. Memory consumption is somewhat worse on Windows but it tends to stabilize in my tests after several executions. Here is a sequence of free memory measurements (in MiB) after running your script on a Windows 8.1 Professional virtual machine (VMware WS 10.0.4) with 8 GiB of RAM:
7027.027
6558.781
6798.816
6250.934
6217.543
6282.281
6251.137
6306.367
Each value has been measured after closing all images and clearing the console. There are variations but the values are more or less consistent with what you see on your machine. However it seems to stabilize at about 6200-6300 MiB. On this machine, the amount of free memory can decrease by 400 MiB if I simply open Windows Explorer. After a while, the wasted memory is normally returned to the free amount.
I have traced deallocations of Matrix objects in the core application while your script is running, and it is quite curious how the JS engine performs garbage collections. The sequence of deallocations is always: 2048, 2560, 2048, 2560, ... For some reason the engine decides to call Matrix finalizing routines this way, but I have no idea why.
I don't think we have a memory leak here. It is just that your script allocates many large matrices and *lots* of variables. Here are several things that you can try to reduce memory consumption:
1. As a general rule: As soon as you are done with a large Matrix (or similar objects associated with large data blocks, such as Image), deallocate it. For example:
function foo()
{
...
{
...
let M = new Matrix( 1000, 1000 );
...
// Force deallocation. Now we don't depend on the garbage collector.
M.assign( 0, 0, 0 );
}
...
}In the next version I'm going to add a new clear() method to force deallocation of Matrix and Vector objects. Image already has a free() method that you can use when necessary.
2. In the incoming version 6 of the ECMAScript specification (ES6),
the let statement allows defining variables with block scope. Unlike
var, which defines a function scope variable, objects declared with
let get out of scope exactly as automatic variables do in C and C++. For example, instead of:
for ( var i = 0; i < ... )we can typically use:
for ( let i = 0; i < ... )The difference is that
i is now local to the for loop body, and hence unknown outside the loop. Just the same as in C++:
for ( int i = 0; i < ... )For functions that use many variables and objects with associated large native data blocks, the let keyword may save memory because the garbage collector has more chances to release memory when variables get successively out of scope:
for ( let i = 0; i < 100; ++i )
{
let M = new Matrix( ... );
...
}In this example, M will be unreferenced as soon as the current iteration of the for loop terminates. Declared with var, the M variable would be accessible within the whole function where it is declared.
Fortunately, we can use let in PJSR because SpiderMonkey 24 already implements this ES6 feature (actually, the Mozilla engine implements this keyword since version 1.7).
I know it can be a lot of work, but it may be interesting to use let instead of var in your script, at least for some critical routines, and see if this helps memory consumption.
Let me know if this helps somewhat. The JavaScript engine is a complex and difficult beast, especially regarding memory resources. I'm still concerned with this problem though, so I'll continue doing tests and trying to devise ways to improve it.
