Memory leak

[pierreinglebert]

I ran a batch using sharp on ~1000 files and it ended using >300MB ram.

I created a little gist to "demonstrate" it, it download a large image and creates 800 thumbs at different sizes.
On my mbp, when finished, it takes 120MB ram (more if i increase the number of generated thumbs).

[lovell]

Are you able to share the gist, or a gist of the gist?

libvips manages its own internal file and operation cache, defaulting to a soft limit of 100MB. This soft limit can be changed via sharp.cache(limit). To see if this is the source of heap (mis-)usage, try setting it to 0 to see if these numbers reduce.

Are Buffers involved? If so, there may be heap allocated for Buffer objects but now remains in unused virtual memory that v8 has decided it might need in the future. You could try Node's --expose-gc start-up setting that exposes the global.gc() method, which will force it to free/compact unreferenced objects on the heap.

Do you see the same problems with both Node 0.10 and 0.11? If things are better with 0.11 then you could have run into one of the problems targeted by nodejs/node-v0.x-archive#5720.

[pierreinglebert]

I created the gist just for this issue, i forgot to copy paste it :/ (updated my post)

It seems to be buffer related, if i return an empty buffer, it doesn't seem to leak.
I didn't investigate deeply on this, i created the issue if you had some leads.

I faced it on 0.10, i will try on node master.

[pierreinglebert]

Well, it's worse on node master (150MB vs 120MB) after calling global.gc()

I think we need to go deeper.

[lovell]

Thanks for all the extra info Pierre, I've been able to reproduce this locally.

I see the process-level rss climbs approximately in line with the size of the generated Buffers whilst the V8-level heapUsed and heapTotal remain stable.

I'll experiment with the version of NanNewBufferHandle that takes a FreeCallback to discover if the node::Buffer created at sharp.cc:309 is being correctly freed at GC time.

Failing that, valgrind is our friend.

[lovell]

Some Initial findings...

Even when libvips cache limit is set to zero, it always holds on to at least one operation. I'll check with John to see if this is a feature or a bug.

This cache might account for up to half of the Resident Set Size (RSS) value.

I've run memleak,js through valgrind and there are a few small leaks I need to take a closer look at:

==27133== LEAK SUMMARY:
==27133== definitely lost: 5,299 bytes in 43 blocks
==27133== indirectly lost: 60,920 bytes in 87 blocks
==27133== possibly lost: 70,543 bytes in 135 blocks
==27133== still reachable: 7,290,817 bytes in 20,722 blocks

The RSS value can include previously allocated/used memory yet to be returned to the OS, which might not always mean there's a leak. Perhaps try running other memory-intensive processes at the same time to see if the RSS value drops.

[lovell]

I'll attempt to fix the inability to disable the operation cache via https://github.com/jcupitt/libvips/issues/119.

Another thought: the RSS value also includes shared libraries that could be hogging/leaking memory. For example libvips uses Optimized Inner Loop Runtime Compiler, which was updated last week (v0.4.19) to "fix many memory leaks".

[lovell]

The liborc library v0.4.18 is definitely leaking memory.

Valgrind test run with libvips compiled to use ORC v0.4.18:

==22480== LEAK SUMMARY:
==22480== definitely lost: 7,443 bytes in 124 blocks
==22480== indirectly lost: 160,734 bytes in 158 blocks

Valgrind test run with libvips --without-orc:

==19249== LEAK SUMMARY:
==19249== definitely lost: 4,563 bytes in 104 blocks
==19249== indirectly lost: 43,578 bytes in 118 blocks

Disabling, or better upgrading to v0.4.19 is highly recommended.

[jcupitt]

It looks like orc-0.4.19 is freeing too much memory :-( see jcupitt/libvips@d990f5c

apply that patch and live with the small leak, or disable orc

[lovell]

Thanks John, are you planning to file a bug report upstream?

I'm going to recommend disabling liborc for now.

[jcupitt]

Yes, I need to test against their git master and maybe make a tiny test case.