large negative increment in %mprun on ipython

[nekketsuuu]

Problem

The %mprun line profiler reported an unexpected big negative increment values in a for loop. I came across this problem when running it on Google Colaboratory, and also another user said the same problem occurred on IPython on macOS.

I used most recent version of HEAD of master branch (memory_profiler 0.54.0 + PR #202 + something) by pip install -U https://github.com/pythonprofilers/memory_profiler/archive/master.zip, but the problem was reproduced.

Code

The original source code is from https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling.html by Jake VanderPlas.

!pip install -U https://github.com/pythonprofilers/memory_profiler/archive/master.zip
%load_ext memory_profiler

%%file mprun_demo.py
def sum_of_lists(N):
 total = 0
 for i in range(5):
 L = [j ^ (j >> i) for j in range(N)]
 total += sum(L)
 del L # remove reference to L
 return total

from mprun_demo import sum_of_lists
%mprun -f sum_of_lists sum_of_lists(1000000)

See the actual notebook here. You can try and re-run the above example.

Result

Filename: /content/mprun_demo.py

Line # Mem usage Increment Line Contents
================================================
 1 150.3 MiB 150.3 MiB def sum_of_lists(N):
 2 150.3 MiB 0.0 MiB total = 0
 3 150.3 MiB 0.0 MiB for i in range(5):
 4 181.1 MiB -46807065.7 MiB L = [j ^ (j >> i) for j in range(N)]
 5 181.1 MiB 0.0 MiB total += sum(L)
 6 150.3 MiB -123.0 MiB del L # remove reference to L
 7 150.3 MiB 0.0 MiB return total

Environment

• IPython on Google Colaboratory
• memory_profiler 0.54.0 + 4 commits (959d9fc)

Related?

#9, #195, #211

Also the same problem is reported on Stack Overflow: "Strange increment value reported during IPython memory profiling".