Skip to content

Conversation

@dxe4
Copy link

@dxe4dxe4 commented Nov 2, 2019

related to my comment here #236 (comment)
with implementation of guppy3 we can get the right values.

note this implementation does self.prevlines[-1] == self.prev_lineno: for guppy backend only, because repeating [i for i in range(0, 10000000)] makes the guppy backend extremely slow. although this may not be ideal or acceptable i thought it's worth sharing the code.

 @profile def myfn_py(): ones = [1] * NBR_ITEMS twos = [i + 1 for i in ones] return twos @profile def myfn_np(): ones_np = np.ones(NBR_ITEMS) twos_np = ones_np+1 return twos_np myfn_np() myfn_py() python -m memory_profiler --backend guppy prof.py Line # Mem usage Increment Line Contents ================================================ 11 14.191 MiB 14.191 MiB @profile 12 def myfn_py(): 13 18.006 MiB 3.815 MiB ones = [1] * NBR_ITEMS 14 22.098 MiB 4.091 MiB twos = [i + 1 for i in ones] 15 return twos Filename: prof.py Line # Mem usage Increment Line Contents ================================================ 18 14.189 MiB 14.189 MiB @profile 19 def myfn_np(): 20 18.005 MiB 3.817 MiB ones_np = np.ones(NBR_ITEMS) 21 21.820 MiB 3.815 MiB twos_np = ones_np+1 22 21.820 MiB 0.000 MiB return twos_np 

if self.backend == 'guppy':
if self.prevlines[-1] == self.prev_lineno:
# the reason for this is because for code like [i for i in range(0, 1000000)] guppy will be extremely slow
_run = False
Copy link
Author

@dxe4dxe4Nov 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can cause issues if you want to measure the memory of a side effect eg

[foo() for i in range(0, 500)] def foo(): make_big_array = [i for i in random.randint(0, int(1e9))] 

i haven't tested it.
although it will give correct results for this case #236 (comment)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@dxe4