8 Mar
2024
8 Mar
'24
3:24 p.m.
Dear Python experts among the NEST Users!
I just notice a strange interaction between NEST and Ipython. In a freshly started
Ipython, I execute the following three statements
import nest
n = nest.Create('parrot_neuron', positions=nest.spatial.free([(x, 0) for x in
range(10000)]))
n
The last one takes extraordinarily long, over 10 seconds, I think. But if I instead do
print(n)
it executes instantaneously.
Also, if I do the same thing in python directly, just entering
n
shows the NodeCollection immediately.
Can anyone reproduce this and does anyone have an idea what might be going on?
I am on macOS 14.3.1 with Python 3.12.2, Ipython 8.22.2, and have checked with NEST 3.6
and a branch of recent master.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.no<mailto:hans.ekkehard.plesser@nmbu.no>
Home http://arken.nmbu.no/~plesser
8 Mar
8 Mar
6:25 p.m.
I am not a python expert, but same behaviour here ...
NEST 3.6 (git master HEAD from february 12), Python 3.8.12, Rocky Linux 8.5
Xavier
________________________________
From: Hans Ekkehard Plesser <hans.ekkehard.plesser(a)nmbu.no>
Sent: Friday, March 8, 2024 3:24 PM
To: users(a)nest-simulator.org <users(a)nest-simulator.org>
Subject: [NEST Users] Strange interaction between NEST and IPython
Dear Python experts among the NEST Users!
I just notice a strange interaction between NEST and Ipython. In a freshly started
Ipython, I execute the following three statements
import nest
n = nest.Create('parrot_neuron', positions=nest.spatial.free([(x, 0) for x in
range(10000)]))
n
The last one takes extraordinarily long, over 10 seconds, I think. But if I instead do
print(n)
it executes instantaneously.
Also, if I do the same thing in python directly, just entering
n
shows the NodeCollection immediately.
Can anyone reproduce this and does anyone have an idea what might be going on?
I am on macOS 14.3.1 with Python 3.12.2, Ipython 8.22.2, and have checked with NEST 3.6
and a branch of recent master.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.no<mailto:hans.ekkehard.plesser@nmbu.no>
Home http://arken.nmbu.no/~plesser
Computer Vision Center<http://www.cvc.uab.cat>
CONFIDENTIALITY WARNING<http://www.cvc.uab.es/?page_id=7475>
8:19 p.m.
Does IPython offer any profiling? Does regular Python profiling work? I
would recommend to start a profiling session before the last statement, and
to end it immediately after. A tool like snakeviz can them visualize it for
you.
On Fri, Mar 8, 2024, 18:26 Xavier Otazu Porter <xotazu(a)cvc.uab.cat> wrote:
I am not a python expert, but same behaviour here ...
NEST 3.6 (git master HEAD from february 12), Python 3.8.12, Rocky Linux 8.5
Xavier
------------------------------
*From:* Hans Ekkehard Plesser <hans.ekkehard.plesser(a)nmbu.no>
*Sent:* Friday, March 8, 2024 3:24 PM
*To:* users(a)nest-simulator.org <users(a)nest-simulator.org>
*Subject:* [NEST Users] Strange interaction between NEST and IPython
Dear Python experts among the NEST Users!
I just notice a strange interaction between NEST and Ipython. In a freshly
started Ipython, I execute the following three statements
import nest
n = nest.Create('parrot_neuron', positions=nest.spatial.free([(x, 0) *for*
x *in* range(10000)]))
n
The last one takes extraordinarily long, over 10 seconds, I think. But if
I instead do
print(n)
it executes instantaneously.
Also, if I do the same thing in python directly, just entering
n
shows the NodeCollection immediately.
Can anyone reproduce this and does anyone have an idea what might be going
on?
I am on macOS 14.3.1 with Python 3.12.2, Ipython 8.22.2, and have checked
with NEST 3.6 and a branch of recent master.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser(a)nmbu.no
Home http://arken.nmbu.no/~plesser
Computer Vision Center <http://www.cvc.uab.cat>
CONFIDENTIALITY WARNING <http://www.cvc.uab.es/?page_id=7475>
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
11 Mar
11 Mar
10:52 p.m.
Hello Hans,
On Fri, Mar 08, 2024 at 02:24:17PM +0000, Hans Ekkehard Plesser wrote:
Dear Python experts among the NEST Users!
I just notice a strange interaction between NEST and Ipython. In a freshly started
Ipython, I execute the following three statements
import nest
n = nest.Create('parrot_neuron', positions=nest.spatial.free([(x, 0) for x in
range(10000)]))
n
The last one takes extraordinarily long, over 10 seconds, I think. But if I instead do
print(n)
it executes instantaneously.
What happens is that IPython's pretty printing calls (a bit down the
line) the IPythonDisplayFormatter() class. The first thing that does
is test if for the given object there is an _ipython_display_() method
- if that exists, it calls that, otherwise it falls back to a more
generic method. In your case the latter happens, it falls back to the
more generic method and that is what then produces the output.
When looking up if there is an _iython_display_() method, the first
thing IPython does is check for a method called
_ipython_canary_method_should_not_exist_() - and it is precisely this
lookup that takes so long. You can verify this by
getattr(n, "_ipython_canary_method_should_not_exist_")
which will also take rather long. As the output here also contains
some SLI things, one might need to dig deeper to actually speed this
up.
Instead, adding a check specifically for exactly the above mentioned
lookup of _ipython_canary_method_should_not_exist_() also does the
job (for me), see the attachment. With that workaround, your example
returns instantaneously - as one would expect it to.
regards,
Harold
12 Mar
12 Mar
9:40 a.m.
Hi Harold,
Nice to hear from you and thanks a lot for the detailed explanations and the solution for
this mystery 😊!
Would you want to prepare a PR for this, so you get credit assigned for the solution?
Ideally today, since we plan to roll NEST 3.7-RC1 very, very soon now.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.no<mailto:hans.ekkehard.plesser@nmbu.no>
Home http://arken.nmbu.no/~plesser
From: Harold Gutch <nest-hg(a)gutch.de>
Date: Monday, 11 March 2024 at 22:52
To: NEST User Mailing List <users(a)nest-simulator.org>
Subject: [NEST Users] Re: Strange interaction between NEST and IPython
[Du mottar ikke ofte e-post fra nest-hg(a)gutch.de. Finn ut hvorfor dette er viktig p?
https://aka.ms/LearnAboutSenderIdentification ]
Hello Hans,
On Fri, Mar 08, 2024 at 02:24:17PM +0000, Hans Ekkehard Plesser wrote:
Dear Python experts among the NEST Users!
I just notice a strange interaction between NEST and Ipython. In a freshly started
Ipython, I execute the following three statements
import nest
n = nest.Create('parrot_neuron', positions=nest.spatial.free([(x, 0) for x in
range(10000)]))
n
The last one takes extraordinarily long, over 10 seconds, I think. But if I instead do
print(n)
it executes instantaneously.
What happens is that IPython's pretty printing calls (a bit down the
line) the IPythonDisplayFormatter() class. The first thing that does
is test if for the given object there is an _ipython_display_() method
- if that exists, it calls that, otherwise it falls back to a more
generic method. In your case the latter happens, it falls back to the
more generic method and that is what then produces the output.
When looking up if there is an _iython_display_() method, the first
thing IPython does is check for a method called
_ipython_canary_method_should_not_exist_() - and it is precisely this
lookup that takes so long. You can verify this by
getattr(n, "_ipython_canary_method_should_not_exist_")
which will also take rather long. As the output here also contains
some SLI things, one might need to dig deeper to actually speed this
up.
Instead, adding a check specifically for exactly the above mentioned
lookup of _ipython_canary_method_should_not_exist_() also does the
job (for me), see the attachment. With that workaround, your example
returns instantaneously - as one would expect it to.
regards,
Harold
9:56 a.m.
Hi Hans Ekkehard,
On Tue, Mar 12, 2024 at 08:40:50AM +0000, Hans Ekkehard Plesser wrote:
Hi Harold,
Nice to hear from you and thanks a lot for the detailed explanations and the solution for
this mystery 😊!
Would you want to prepare a PR for this, so you get credit assigned for the solution?
Ideally today, since we plan to roll NEST 3.7-RC1 very, very soon now.
I don't have a Github account :). Could you do that? Don't think too
much about how to credit me etc., just do it however is the easiest
for you. From my side, you may see my diff as as close to Public
Domain as possible - so if that's the easiest for you, just put it in
there "as your contribution" or so. I'm happy if it helps :).
regards,
Harold
10:30 a.m.
Hi Harold,
Done, see https://github.com/nest/nest-simulator/pull/3145
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.no<mailto:hans.ekkehard.plesser@nmbu.no>
Home http://arken.nmbu.no/~plesser
From: Harold Gutch <nest-hg(a)gutch.de>
Date: Tuesday, 12 March 2024 at 09:56
To: NEST User Mailing List <users(a)nest-simulator.org>
Subject: [NEST Users] Re: Strange interaction between NEST and IPython
[Du mottar ikke ofte e-post fra nest-hg(a)gutch.de. Finn ut hvorfor dette er viktig på
https://aka.ms/LearnAboutSenderIdentification ]
Hi Hans Ekkehard,
On Tue, Mar 12, 2024 at 08:40:50AM +0000, Hans Ekkehard Plesser wrote:
Hi Harold,
Nice to hear from you and thanks a lot for the detailed explanations and the solution for
this mystery 😊!
Would you want to prepare a PR for this, so you get credit assigned for the solution?
Ideally today, since we plan to roll NEST 3.7-RC1 very, very soon now.
I don't have a Github account :). Could you do that? Don't think too
much about how to credit me etc., just do it however is the easiest
for you. From my side, you may see my diff as as close to Public
Domain as possible - so if that's the easiest for you, just put it in
there "as your contribution" or so. I'm happy if it helps :).
regards,
Harold
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
54
days inactive
58
days old
6 comments
4 participants
participants (4)
-
Hans Ekkehard Plesser -
Harold Gutch -
Robin Gilbert De Schepper -
Xavier Otazu Porter