- NEST Users - NEST Initiative Mail

Problems when using a mask inside the conn_dict on 3 or 4 MPI processes

by Kempter, Miriam

Dear NEST Community, while adapting my model to run on multiple MPI processes, I have been running into some problems connected with the usage of masks inside the connectivity dictionary for a 2D-spatially distributed population. You can find a minimal example containing further details in the attachments in the form of a .txt file. Please just change "txt" with "py" for execution. The following explanation also references to the example. My setup: - NEST version 3.4 - Python version 3.10 - Executed with mpirun inside a conda environment "mpirun -np 4 python3 minEx_4MPIprocesses_problem.py" - System: Ubuntu 22.04 - The same problem occurred when the model was executed on JURECA. Problem description: While using multiple MPI processes: 1. Create circular mask. See line 119 Create a spatial 2D-Population of neurons "neurons". See line 125-132 2. Select some of the neurons as source neurons. See line 154 3. Set up the connection dictionary with a mask inside. See line 157-163 4. For each source neuron: See line 173 Connect the source neurons with the neurons population using the connection dictionary See line 183 Where does the Problem occur?: The error occurs at the point when the nest.Connect(…) call is executed in the loop at line 183 when each source neuron is connected. When does the Problem occur?: Its occurrance depends on whether or not a mask is used inside the connection-dict given to the nest.Connect(...) function. If the mask is removed as in the conn-dict. in line 164-169, no error is produced in none of the used settings (however the result is not as desired). If the mask is used, the way the problem shows itself depends on the number of MPI processes used, and on the setting of the number of neurons, extent, mask radius, number of source neurons and if edge wrap is used. - For 1 and 2 MPI procs the model is running correctly, independent of the used conn-dict and setting. - For 3 MPI procs either the model runs but the distance between connected neurons does not correspond to the given mask dimensions. In other words: The established connections are longer/shorter than the mask should allow. Or execution leads to an error code output ("segmentation fault") from mpirun and job abortion. The terminal output for an example run can be seen in the attached file "minEx_4MPIprocesses_problem_error_output_3_MPI" - For 4 MPI procs either, the model runs correctly, or it leads to an error code output ("segmentation fault") and the model not terminating. (When executed in the terminal the keyboard command "str+c" is needed to stop the execution. The terminal output for an example run can be seen in the attached file "minEx_4MPIprocesses_problem_error_output_4_MPI" If the model is executed with the same setting on 3 and 4 MPI procs there are 3 possible combinations of above described problems. 1. The model runs and terminates on both, however the distances created on 3 MPI procs are wrong. 2. The model works with 4 MPI procs but creates the "segmentation fault" error on 3 MPI processes. 3. The model creates a "segmentation fault" error on both. Please keep in mind that whether an error occurs and in which form highly depends on the used setting. In the minimal example I provide different settings which represent the above described cases. However, it might not cover everything that can occur. Workaround: For my own use-case I found a rather computing-time consuming workaround. It involves applying nest.SelectNodesByMask() on every source neuron, and from the resulting set choosing the targets using the desired probability. This requires multiple additional loops and also communicating the position data of every neuron to every MPI process in the beginning. Using this approach, the connection distances seem to be correct and no error occurs while executing the model. However, while writing this I'm questioning if I actually tested it enough. So there might be some not yet discovered problems. Is there something that I overlooked or approached wrong when using masks in the connectivity dict on multiple MPI processes? Thanks in advance! Best, Miriam Kempter ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Karsten Beneke (stellv. Vorsitzender), Dr. Ir. Pieter Jansens ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------

1 month, 4 weeks

2
4
0 0

Weights assignment error with mpirun

by Xavier Otazu

Hi! I am trying to implement weight normalization (as you show in your documentation). It works well (see below code) when executed "normally" (or when using" mpirun -np 1"), but it hangs (it just do not respond) when executed using mpirun. It hangs when I try to execute nest.Simulate() after weigth assignment. If I remove the "conn.weight = w_nparray" instruction in the following code, it works without hang. I use NEST 3.6 and python3, compiled with python, mpi, sionlib and music. Thanks a lot in advance! import nest import numpy as np pos = nest.spatial.grid(shape = [30,30] ) input = nest.Create('iaf_psc_alpha', positions=pos) layer_0 = nest.Create('iaf_psc_alpha', positions=pos) layer_1 = nest.Create('iaf_psc_alpha', positions=pos) conn_neur = {'rule':'pairwise_bernoulli', 'use_on_source': True, 'mask': {'grid':{'shape':[9,9]}} } nest.CopyModel('static_synapse', 'syn_1_model') syn_0 = {'synapse_model': 'static_synapse'} syn_1 = {'synapse_model': 'syn_1_model'} nest.Connect(input, layer_0, conn_neur, syn_0) nest.Connect(input, layer_1, conn_neur, syn_0) nest.DumpLayerConnections(input, layer_0, 'static_synapse', 'conn.txt') nest.DumpLayerConnections(input, layer_1, 'syn_1_model', 'conn.txt') nest.Simulate(100) for neuron in layer_0: conn = nest.GetConnections(source=input, target=neuron, synapse_model='static_synapse') w = np.array(conn.weight) if (w.size>1): w_normed = w / sum(abs(w)) # L1-norm w_nparray = 2. * w_normed conn.weight = w_nparray nest.Simulate(100)

2 months

2
16
0 0

cannot open shared object file (pynestkernel.so)

by Willem Wybo

I’m trying to run NEST on Jureca, but I get an error: Traceback (most recent call last): File "/p/project/jinm60/users/wybo1/Code/NEAST_models/BBP/tests/run_all_tests.py", line 2, in <module> import nest.lib.hl_api_exceptions as nestexceptions File "/p/home/jusers/wybo1/jureca/nest/nest-install/lib64/python3.11/site-packages/nest/__init__.py", line 64, in <module> from .ll_api_kernel_attributes import KernelAttribute # noqa ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/p/home/jusers/wybo1/jureca/nest/nest-install/lib64/python3.11/site-packages/nest/ll_api_kernel_attributes.py", line 22, in <module> from .ll_api import spp, sps, sr, stack_checker File "/p/home/jusers/wybo1/jureca/nest/nest-install/lib64/python3.11/site-packages/nest/ll_api.py", line 53, in <module> from . import pynestkernel as kernel # noqa pylint: disable=no-name-in-module ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ImportError: libltdl.so.7: cannot open shared object file: No such file or directory Apparently it doesn’t find pynestkernel.so (I think?). However, this file is definitely there. wybo1@jrlogin09 ~/.../site-packages/nest> pwd /p/home/jusers/wybo1/jureca/nest/nest-install/lib64/python3.11/site-packages/nest wybo1@jrlogin09 ~/.../site-packages/nest> ls __init__.py __pycache__ lib ll_api.py ll_api_kernel_attributes.py logic math pynest-init.sli pynestkernel.so random raster_plot.py server spatial spatial_distributions versionchecker.py visualization.py voltage_trace.py The problem also only manifests itself while running from a SLURM script. I followed the install instruction here: https://nest-simulator.readthedocs.io/en/stable/installation/noenv_install.… And source the nest_vars.sh file according to the instructions: `source /p/home/jusers/wybo1/jureca/nest/nest-install/bin/nest_vars.sh` Also, `make installcheck` passes successfully (both from the login node and from a slurm script). Cheers, Willem ---------------------- Dr. Willem Wybo Peter Grünberg Institute 15 - Neuromorphic Software Ecosystems Forschungszentrum Jülich www.fz-juelich.de/pgi/PGI-15<http://www.fz-juelich.de/pgi/PGI-15> ---------------------- ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschäftsführung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Karsten Beneke (stellv. Vorsitzender), Dr. Ir. Pieter Jansens ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------

2 months, 1 week

2
2
0 0

STDP synapse t_spike and t_lastspike clarification

by prao2＠uw.edu

Dear all, I am trying to develop a new synapse model that uses the activity of the postsynaptic neuron. I'm looking at the STDP synapse to understand how it gets the target neuron history, and I wanted to clarify: are t_spike and t_lastspike the time stamps for the last spike of the presynaptic or postsynaptic neuron? I am looking at the send method of stdp_synapse.h (https://github.com/nest/nest-simulator/blob/91a02ed98017224deabe25e8e08bcfd…) Thanks in advance! Best, Priyanka Rao

2 months, 1 week

3
3
0 0

Problems in simulation restore

by Petia Koprinkova

Hi, We've discovered the following problem in restoring network simulation: if the simulation stops for some reason restoring it form the network status kept in files does not yield the same results as in case when the simulation was not interrupted and finishes normally. In our experiment we run a script twice as follows: 1. Create a network and simulate for some period of time. 2. Record the current network status in terms of connections and membrane potentials and store them in files at the end of script.3. End the script 4. Start the script again restoring the network structure from files.5. Run simulation till the end of desired time. The spiking activity of the network is different if we run simulation without interruption or if we run it in the manner described here: https://nest-simulator.readthedocs.io/en/stable/auto_examples/store_restore… i.e. if we run a script with following steps:1. Run simulation for some time.2. Get connections and membrane potentials3. Reset kernel4. Create again the network5. Simulate till the end of desired time. Please help to resolve the problem. Best,Petia Koprinkova-Hristova

2 months, 1 week

1
0
0 0

Question about membrane potential recording

by Petia Koprinkova

Hi, We recently discovered that multimeter returns membrane potential in some unordered way so if we want to record the last state of a group of neurons and then to continue simulation from exactly the same state we were unable to reconstruct it properly. Is there a way to record the membrane potential together with neuron ID? Best,Petia Koprinkova-Hristova

2 months, 2 weeks

3
4
0 0

Assert failed running hpc_benchmark

by Michele Martinelli

Dear NEST Users & Developers, we're currently working on a custom OpenMPI BTL (supporting a custom FPGA-based NIC) at the National Institute for Nuclear Physics in Rome, Italy and we have an error when running hpc_benchmark(this test is currently used as simple validation test) with 2 processes (one on each of 2 hosts), the command we run is like: mpirun -n 2 -H host1:1,host2:1 --bynode --report-bindings -mca btl apelink,self,sm python hpc_benchmark.py (apelink is our custom BTL component) but then we see this error: python: [...]/NEST_with_local_ompi/nest-simulator-3.6/nestkernel/target_table.cpp:107: void nest::TargetTable::add_target(size_t, size_t, const nest::TargetData&): Assertion `syn_id < secondary_send_buffer_pos_[ tid ][ lid ].size()' failed. [host:23979] *** Process received signal *** [host:23979] Signal: Aborted (6) [host:23979] Signal code: (-6) My guess is that we are transferring something incorrectly (maybe during the initialization/setup phase?), but I'm not sure what the assert expects to have in secondary_send_buffer_pos_[ tid ][ lid ].size() and how this field should be set. Best, Michele

2 months, 2 weeks

2
1
0 0

Open NEST Developer Video Conference Monday January 29, at 11:30 CET (UTC+0100)

by Dennis Terhorst

Dear NEST Users & Developers! I would like to invite you to our next fortnightly Open NEST Developer Video Conference today *Monday January 29, at 11:30 CET (UTC+0100).* As usual, in the Project team round, a contact person of each team will give a short statement summarizing ongoing and planned work in the team and highlight cross-cutting points that need discussion among the teams. The remainder of the meeting we would go into a bunch of topics that came up and that were suggested by the teams. Feel free to join the meeting also if it’s just to bring your own quick questions for direct discussion in the in-depth section. Agenda * Welcome * Review of NEST User Mailing List * Project team round * In-depth discussion o Changed Model registration for NEST Models (from last meeting) o Proposed new model with an approximation for nonlinear NMDA receptors (#2979 <https://github.com/nest/nest-simulator/pull/2979>) o Proposed multi-state models (#3069 <https://github.com/nest/nest-simulator/pull/3069>) o Report from LASCON IX The agenda for this meeting is also available online, see https://github.com/nest/nest-simulator/wiki/2024-01-29-Open-NEST-Developer-… <https://github.com/nest/nest-simulator/wiki/2024-01-29-Open-NEST-Developer-…> Looking forward to seeing you soon! Cheers, Dennis Terhorst Log-in information We use a virtual conference room provided by DFN <https://www.dfn.de/en/> (Deutsches Forschungsnetz). You can use the web client to connect. We however encourage everyone to use a headset for better audio quality or even a proper video conferencing system (see below) or software when available. Web client * Visit https://conf.dfn.de/webapp/conference/97938800 <https://conf.dfn.de/webapp/conference/97938800> * Enter your name and allow your browser to use camera and microphone * The conference does not need a PIN to join, just click join and you’re in. In case you see a dfnconf logo and the phrase “Auf den Meetingveranstalter warten”, just be patient, the meeting host needs to join first (a voice will tell you). VC system/software How to log in with a video conferencing system, depends on you VC system or software. * Using the H.323 protocol (eg Polycom): |vc.dfn.net##97938800| or |194.95.240.2##97938800| * Using the SIP protocol:97938800@vc.dfn.de <mailto:97938800@vc.dfn.de> * By telephone: |+49-30-200-97938800| For those who do not have a video conference system or suitable software, Polycom provides a pretty good free app for iOS and Android, so you can join from your tablet (Polycom RealPresence Mobile, available from AppStore/PlayStore). Note that firewalls may interfere with videoconferencing in various and sometimes confusing ways. For more technical information on logging in from various VC systems, please see http://vcc.zih.tu-dresden.de/index.php?linkid=1.1.3.4 <http://vcc.zih.tu-dresden.de/index.php?linkid=1.1.3.4> -- Dipl.-Phys. Dennis Terhorst Coordinator Software Development Institute for Advanced Simulation (IAS-6), Computational and Systems Neuroscience & JARA-Institute Brain Structure-Function Relationships (INM-10) Institute of Neuroscience and Medicine Jülich Research Center, Member of the Helmholz Association 52425 Jülich, Germany http://www.csn.fz-juelich.de/ Phone +49 2461 61-85062 ---------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschaeftsfuehrung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Karsten Beneke (stellv. Vorsitzender), Dr. Ir. Pieter Jansens ----------------------------------------------------------------------

2 months, 3 weeks

1
0
0 0

Google Gemini NESTML code

by Alvin Spivey

I'm using Google Bard's Gemini to learn NESTML. It's giving me some unexpected results: import nestml from nestml.network import Network from nestml.models import ConeNakaRushton # Define network with Naka-Rushton cone model for primate 'cone_type' cones network = Network() cone_population = network.add_population("cones", ConeNakaRushton(species= "human", cone_type=cone_type)) The NESTML network and models packages don't seem to exist. Also, how do I find more information about the network and models packages (or packages in general) for NESTML? -- Alvin J. Spivey, Ph.D. (c) 843.267.8055

2 months, 3 weeks

3
2
0 0

Open NEST Developer Video Conference Monday January 15, at 11:30 CET (UTC+0100)

by Dennis Terhorst

Dear NEST Users & Developers! 🎈🎉🎈 Happy New Year! 🎈🎊🎈 I would like to invite you to our first fortnightly Open NEST Developer Video Conference in 2024, today *Monday January 15, at 11:30 CET (UTC+0100).* As usual, in the Project team round, a contact person of each team will give a short statement summarizing ongoing and planned work in the team and highlight cross-cutting points that need discussion among the teams. The remainder of the meeting we would go into a more in-depth discussion of topics that came up on the mailing list or that are suggested by the teams. Feel free to join the meeting also if it’s just to bring your own quick questions for direct discussion in the in-depth section. Agenda * Welcome * Review of NEST User Mailing List * Project team round * In-depth discussion The agenda for this meeting is also available online, see https://github.com/nest/nest-simulator/wiki/2024-01-15-Open-NEST-Developer-… <https://github.com/nest/nest-simulator/wiki/2024-01-15-Open-NEST-Developer-…> Looking forward to seeing you soon! Cheers, Dennis Terhorst Log-in information We use a virtual conference room provided by DFN <https://www.dfn.de/en/> (Deutsches Forschungsnetz). You can use the web client to connect. We however encourage everyone to use a headset for better audio quality or even a proper video conferencing system (see below) or software when available. Web client * Visit https://conf.dfn.de/webapp/conference/97938800 <https://conf.dfn.de/webapp/conference/97938800> * Enter your name and allow your browser to use camera and microphone * The conference does not need a PIN to join, just click join and you’re in. In case you see a dfnconf logo and the phrase “Auf den Meetingveranstalter warten”, just be patient, the meeting host needs to join first (a voice will tell you). VC system/software How to log in with a video conferencing system, depends on you VC system or software. * Using the H.323 protocol (eg Polycom): |vc.dfn.net##97938800| or |194.95.240.2##97938800| * Using the SIP protocol:97938800@vc.dfn.de <mailto:97938800@vc.dfn.de> * By telephone: |+49-30-200-97938800| For those who do not have a video conference system or suitable software, Polycom provides a pretty good free app for iOS and Android, so you can join from your tablet (Polycom RealPresence Mobile, available from AppStore/PlayStore). Note that firewalls may interfere with videoconferencing in various and sometimes confusing ways. For more technical information on logging in from various VC systems, please see http://vcc.zih.tu-dresden.de/index.php?linkid=1.1.3.4 <http://vcc.zih.tu-dresden.de/index.php?linkid=1.1.3.4> -- Dipl.-Phys. Dennis Terhorst Coordinator Software Development Institute of Neuroscience and Medicine Computational and Systems Neuroscience (INM-6) & Institute for Advanced Simulation (IAS-6) Theoretical Neuroscience & JARA (INM-10) Jülich Research Center, Member of the Helmholz Association 52425 Jülich, Germany http://www.csn.fz-juelich.de/ Phone +49 2461 61-85062 ---------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Stefan Müller Geschaeftsfuehrung: Prof. Dr. Astrid Lambrecht (Vorsitzende), Karsten Beneke (stellv. Vorsitzender), Dr. Ir. Pieter Jansens ----------------------------------------------------------------------

3 months

1
0
0 0

2024

2023

2022

2021

2020

2019

NEST Users