29 Nov
2021
29 Nov
'21
6:32 p.m.
Dear NEST users,
I am trying to debug my network and I found that my spike_recorder with
*ascii* option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms) and then do
some structure changes and then re-run for another phase of a few ms (say
4000. ms)
In the compact version, I am trying to see how this sample code would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate' :
100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but for
memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and False.
In the previous version (v2.20) I also used the same approach and had no
issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of NESTv2.x and
NESTv3.x
Am I missing something? It's really important for my work, I hope I could
find a solution.
--
Thanks and Regards
*Maryada*
29 Nov
29 Nov
10:24 p.m.
Dear Maryada,
I have just played a bit with your example (see attachment for a slightly extended and
reformatted version) and as far as I can see, NEST behaves exactly as expected (and
documented).
Please let me explain:
* When using the recording backend
ascii<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute
overwrite_files<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to True, you have to move files away at the location in your script saying do
something, otherwise the data files get overwritten by each new call to Simulate() and you
consequently get to see only the results from the last run (which is what you report).
* If you set overwrite_files to False, and did not move the file away, you will get an
error upon repeated calls to Simulate(), as the file to be created by the backend already
exists.
* When using the recording backend
memory<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
overwrite_files is not even considered, as that does not actually write files.
* Data just accumulates in the recording backend memory until it is explicitly cleared
by running Input_SD.n_events = 0.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
Dear NEST users,
I am trying to debug my network and I found that my spike_recorder with ascii option
isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms) and then do some structure
changes and then re-run for another phase of a few ms (say 4000. ms)
In the compact version, I am trying to see how this sample code would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate' : 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
Issue: I only get the data recorded from 100th millisecond but for memory, I get all the
data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and False.
In the previous version (v2.20) I also used the same approach and had no issues. Is there
a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of NESTv2.x and NESTv3.x
Am I missing something? It's really important for my work, I hope I could find a
solution.
--
Thanks and Regards
Maryada
_______________________________________________
NEST Users mailing list --
users@nest-simulator.org<mailto:users@nest-simulator.org>
To unsubscribe send an email to
users-leave@nest-simulator.org<mailto:users-leave@nest-simulator.org>
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
11:19 p.m.
Hi Jochen,
I re-tested my code and realized that I didn't set overwrite_files to False
properly, so you are right, I received the error.
However, since I am importing my code from NEST 2.xx version to v3.1 and
all my previous implementation also have similar two-phase simulations,
this was not the case earlier. To my understanding, unless you call
*ResetKernel*, it is a single simulation and should not overwrite the
current file as is the case when recording in memory.
I ran your script (with syntax changes) on my laptop which only has nest
2.20 version and the attached data file contains recording for the whole
run and not just the last subpart of simulation. I would understand if the
functionality is changed in the latest version, however, I was wondering if
there is a possible work-around. I surely cannot manually store all the
data given that I also record from a multimeter and these are
memory-consuming simulations.
On Mon, Nov 29, 2021 at 10:24 PM Jochen Martin Eppler <
j.eppler(a)fz-juelich.de> wrote:
Dear Maryada,
I have just played a bit with your example (see attachment for a slightly
extended and reformatted version) and as far as I can see, NEST behaves
exactly as expected (and documented).
Please let me explain:
-
When using the recording backend ascii
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute overwrite_files
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to True, you have to move files away at the location in your
script saying do something, otherwise the data files get overwritten
by each new call to Simulate() and you consequently get to see only
the results from the last run (which is what you report).
-
If you set overwrite_files to False, and did not move the file away,
you will get an error upon repeated calls to Simulate(), as the file
to be created by the backend already exists.
-
When using the recording backend memory
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
overwrite_files is not even considered, as that does not actually
write files.
-
Data just accumulates in the recording backend memory until it is
explicitly cleared by running Input_SD.n_events = 0.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
Dear NEST users,
I am trying to debug my network and I found that my spike_recorder with
*ascii* option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms) and then
do some structure changes and then re-run for another phase of a few ms
(say 4000. ms)
In the compact version, I am trying to see how this sample code would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate'
: 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but for
memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and False.
In the previous version (v2.20) I also used the same approach and had no
issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of NESTv2.x and
NESTv3.x
Am I missing something? It's really important for my work, I hope I could
find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Thanks and Regards
*Maryada*
30 Nov
30 Nov
9:13 a.m.
Dear Maryada,
I re-tested my code and realized that I didn't set
overwrite_files to
False properly, so you are right, I received the error.
Good.
However, since I am importing my code from NEST 2.xx
version to v3.1
and all my previous implementation also have similar two-phase
simulations, this was not the case earlier. To my understanding,
unless you call /ResetKernel/, it is a single simulation and should
not overwrite the current file as is the case when recording in memory.
The behavior has indeed been as you describe with NEST 2.x. However, it
was changed with NEST 3.0.
In the 2.x series of NEST, all recording devices had a flag
|close_after_simulate|, which defaulted to |False|. As this flag
resulted in a high bookkeeping complexity internally and was confusing
to many users, it has been removed in favor of the clearer semantics
that each call to |Simulate()| produces it’s own file.
I ran your script (with syntax changes) on my laptop
which only has
nest 2.20 version and the attached data file contains recording for
the whole run and not just the last subpart of simulation. I would
understand if the functionality is changed in the latest version,
however, I was wondering if there is a possible work-around. I surely
cannot manually store all the data given that I also record from a
multimeter and these are memory-consuming simulations.
If you want to run repetitive simulations /and/ get all your data in a
single file with NEST 3.x, you have several possibilities:
1.
You change the kernel property |data_path| inbetween calls to
|Simulate()|. This would let NEST put the data for /all/ recorders
in different directories. You can use |os.mkdir| to create the
directories from within your simulation script:
|import nest, os # Create nodes # Connect nodes
os.mkdir("path_for_sim1") nest.data_path = "path_for_sim1"
nest.Simulate(1000) os.mkdir("path_for_sim2") nest.data_path =
"path_for_sim2" nest.Simulate(2000) |
2.
You can move your files to somewhere safe using |os.rename()|,
instead of setting |data_path|.
3.
You can set a different |data_prefix| between calls to |Simulate()|
(i.e., |nest.data_prefix = "sim1_"|, |nest.data_prefix =
"sim2_"|,
…). That way you wouldn’t have to create directories, but your data
files would instead be prefixed as to your choice.
4.
You can use |Prepare()|, |Run()|, |Cleanup()| instead of |Simulate()
as explained in the [Guide on running
simulations](https://nest-simulator.readthedocs.io/en/latest/guides/running_simulations.html#split-a-simulation-into-multiple-intervals).
For each call to|Run()|, the recorders would actually append to the
files, as those are only closed in|Cleanup()|(and re-opened
in|Prepare()|). In fact,|Simulate()` is actually just a wrapper
around the three aforementioned functions.
Cheers,
Jochen!
On Mon, Nov 29, 2021 at 10:24 PM Jochen Martin Eppler
<j.eppler(a)fz-juelich.de> wrote:
Dear Maryada,
I have just played a bit with your example (see attachment for a
slightly extended and reformatted version) and as far as I can
see, NEST behaves exactly as expected (and documented).
Please let me explain:
*
When using the recording backend |ascii|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute |overwrite_files|
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to |True|, you have to move files away at the location in
your script saying |do something|, otherwise the data files
get overwritten by each new call to |Simulate()| and you
consequently get to see only the results from the last run
(which is what you report).
*
If you set |overwrite_files| to |False|, and did not move the
file away, you will get an error upon repeated calls to
|Simulate()|, as the file to be created by the backend already
exists.
*
When using the recording backend |memory|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
|overwrite_files| is not even considered, as that does not
actually write files.
*
Data just accumulates in the recording backend |memory| until
it is explicitly cleared by running |Input_SD.n_events = 0|.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
Dear NEST users,
I am trying to debug my network and I found that my
spike_recorder with /ascii/ option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms)
and then do some structure changes and then re-run for another
phase of a few ms (say 4000. ms)
In the compact version, I am trying to see how this sample code
would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate' : 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but
for memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and
False.
In the previous version (v2.20) I also used the same approach and
had no issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of
NESTv2.x and NESTv3.x
Am I missing something? It's really important for my work, I hope
I could find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
9:36 a.m.
Dear Maryada,
[Re-sending with properly rendered Markdown. Sorry for the noise]
I re-tested my code and realized that I didn't set
overwrite_files to
False properly, so you are right, I received the error.
Good.
However, since I am importing my code from NEST 2.xx
version to v3.1
and all my previous implementation also have similar two-phase
simulations, this was not the case earlier. To my understanding,
unless you call /ResetKernel/, it is a single simulation and should
not overwrite the current file as is the case when recording in memory.
The behavior has indeed been as you describe with NEST 2.x. However, it
was changed with NEST 3.0.
In the 2.x series of NEST, all recording devices had a flag
|close_after_simulate|, which defaulted to |False|. As this flag
resulted in a high bookkeeping complexity internally and was confusing
to many users, it has been removed in favor of the clearer semantics
that each call to |Simulate()| produces its own file.
I ran your script (with syntax changes) on my laptop
which only has
nest 2.20 version and the attached data file contains recording for
the whole run and not just the last subpart of simulation. I would
understand if the functionality is changed in the latest version,
however, I was wondering if there is a possible work-around. I surely
cannot manually store all the data given that I also record from a
multimeter and these are memory-consuming simulations.
If you want to run repetitive simulations /and/ get all your data
(either in separate files or a single file) with NEST 3.x, you have
several possibilities:
1.
You change the kernel property |data_path| inbetween calls to
|Simulate()|. This would let NEST put the data for /all/ recorders
in different directories. You can use |os.mkdir| to create the
directories from within your simulation script:
|import nest, os # Create nodes # Connect nodes
os.mkdir("path_for_sim1") nest.data_path = "path_for_sim1"
nest.Simulate(1000) os.mkdir("path_for_sim2") nest.data_path =
"path_for_sim2" nest.Simulate(2000) |
2.
You can move your files to somewhere safe using |os.rename()|,
instead of setting |data_path|.
3.
You can set a different |data_prefix| between calls to |Simulate()|
(i.e., |nest.data_prefix = "sim1_"|, |nest.data_prefix =
"sim2_"|,
…). That way you wouldn’t have to create directories, but your data
files would instead be prefixed as to your choice.
4.
You can use |Prepare()|, |Run()|, |Cleanup()| instead of
|Simulate()| as explained in the Guide on running simulations
<https://nest-simulator.readthedocs.io/en/latest/guides/running_simulations.html#split-a-simulation-into-multiple-intervals>.
For each call to |Run()|, the recorders would actually append to the
files, as those are only closed in |Cleanup()| (and re-opened in
|Prepare()|). In fact, |Simulate()| is actually just a wrapper
around the three aforementioned functions.
Cheers,
Jochen!
On Mon, Nov 29, 2021 at 10:24 PM Jochen Martin Eppler
<j.eppler(a)fz-juelich.de> wrote:
Dear Maryada,
I have just played a bit with your example (see attachment for a
slightly extended and reformatted version) and as far as I can
see, NEST behaves exactly as expected (and documented).
Please let me explain:
*
When using the recording backend |ascii|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute |overwrite_files|
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to |True|, you have to move files away at the location in
your script saying |do something|, otherwise the data files
get overwritten by each new call to |Simulate()| and you
consequently get to see only the results from the last run
(which is what you report).
*
If you set |overwrite_files| to |False|, and did not move the
file away, you will get an error upon repeated calls to
|Simulate()|, as the file to be created by the backend already
exists.
*
When using the recording backend |memory|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
|overwrite_files| is not even considered, as that does not
actually write files.
*
Data just accumulates in the recording backend |memory| until
it is explicitly cleared by running |Input_SD.n_events = 0|.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
Dear NEST users,
I am trying to debug my network and I found that my
spike_recorder with /ascii/ option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms)
and then do some structure changes and then re-run for another
phase of a few ms (say 4000. ms)
In the compact version, I am trying to see how this sample code
would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate' : 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but
for memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and
False.
In the previous version (v2.20) I also used the same approach and
had no issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of
NESTv2.x and NESTv3.x
Am I missing something? It's really important for my work, I hope
I could find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
11:54 a.m.
Dear Jochen,
Thanks for taking the time for this. I tested 4th option from your
suggestions as that is the cleanest way to import my work without further
impacting my analysis script. Everything seems to work as expected.
I am not sure if this change is documented or is an intuitive change.
However, it would be nice to have it on the comparison page.
Thanks for your help!
On Tue, Nov 30, 2021 at 9:36 AM Jochen Martin Eppler <j.eppler(a)fz-juelich.de>
wrote:
Dear Maryada,
[Re-sending with properly rendered Markdown. Sorry for the noise]
I re-tested my code and realized that I didn't set overwrite_files to
False properly, so you are right, I received the error.
Good.
However, since I am importing my code from NEST 2.xx version to v3.1 and
all my previous implementation also have similar two-phase simulations,
this was not the case earlier. To my understanding, unless you call
*ResetKernel*, it is a single simulation and should not overwrite the
current file as is the case when recording in memory.
The behavior has indeed been as you describe with NEST 2.x. However, it
was changed with NEST 3.0.
In the 2.x series of NEST, all recording devices had a flag
close_after_simulate, which defaulted to False. As this flag resulted in
a high bookkeeping complexity internally and was confusing to many users,
it has been removed in favor of the clearer semantics that each call to
Simulate() produces its own file.
I ran your script (with syntax changes) on my laptop which only has nest
2.20 version and the attached data file contains recording for the whole
run and not just the last subpart of simulation. I would understand if the
functionality is changed in the latest version, however, I was wondering if
there is a possible work-around. I surely cannot manually store all the
data given that I also record from a multimeter and these are
memory-consuming simulations.
If you want to run repetitive simulations *and* get all your data (either
in separate files or a single file) with NEST 3.x, you have several
possibilities:
1.
You change the kernel property data_path inbetween calls to Simulate().
This would let NEST put the data for *all* recorders in different
directories. You can use os.mkdir to create the directories from
within your simulation script:
import nest, os# Create nodes# Connect nodes
os.mkdir("path_for_sim1")
nest.data_path = "path_for_sim1"
nest.Simulate(1000)
os.mkdir("path_for_sim2")
nest.data_path = "path_for_sim2"
nest.Simulate(2000)
2.
You can move your files to somewhere safe using os.rename(), instead
of setting data_path.
3.
You can set a different data_prefix between calls to Simulate() (i.e.,
nest.data_prefix
= "sim1_", nest.data_prefix = "sim2_", …). That way you wouldn’t
have
to create directories, but your data files would instead be prefixed as to
your choice.
4.
You can use Prepare(), Run(), Cleanup() instead of Simulate() as
explained in the Guide on running simulations
<https://nest-simulator.readthedocs.io/en/latest/guides/running_simulations.html#split-a-simulation-into-multiple-intervals>.
For each call to Run(), the recorders would actually append to the
files, as those are only closed in Cleanup() (and re-opened in
Prepare()). In fact, Simulate() is actually just a wrapper around the
three aforementioned functions.
Cheers,
Jochen!
On Mon, Nov 29, 2021 at 10:24 PM Jochen Martin Eppler <
j.eppler(a)fz-juelich.de> wrote:
--
Thanks and Regards
*Maryada*
Dear Maryada,
I have just played a bit with your example (see attachment for a slightly
extended and reformatted version) and as far as I can see, NEST behaves
exactly as expected (and documented).
Please let me explain:
-
When using the recording backend ascii
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute overwrite_files
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to True, you have to move files away at the location in your
script saying do something, otherwise the data files get overwritten
by each new call to Simulate() and you consequently get to see only
the results from the last run (which is what you report).
-
If you set overwrite_files to False, and did not move the file away,
you will get an error upon repeated calls to Simulate(), as the file
to be created by the backend already exists.
-
When using the recording backend memory
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
overwrite_files is not even considered, as that does not actually
write files.
-
Data just accumulates in the recording backend memory until it is
explicitly cleared by running Input_SD.n_events = 0.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
Dear NEST users,
I am trying to debug my network and I found that my spike_recorder with
*ascii* option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms) and then
do some structure changes and then re-run for another phase of a few ms
(say 4000. ms)
In the compact version, I am trying to see how this sample code would
work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate'
: 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but for
memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and False.
In the previous version (v2.20) I also used the same approach and had no
issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of NESTv2.x and
NESTv3.x
Am I missing something? It's really important for my work, I hope I could
find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
12:04 p.m.
As an aside, if `Input_SD.n_events = 0` is the canonical way of clearing a
spike recorder, maybe a `clear` function could be added. That's a bit
easier to document and find?
On Mon, 29 Nov 2021, 22:24 Jochen Martin Eppler, <j.eppler(a)fz-juelich.de>
wrote:
Dear Maryada,
I have just played a bit with your example (see attachment for a slightly
extended and reformatted version) and as far as I can see, NEST behaves
exactly as expected (and documented).
Please let me explain:
-
When using the recording backend ascii
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute overwrite_files
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to True, you have to move files away at the location in your
script saying do something, otherwise the data files get overwritten
by each new call to Simulate() and you consequently get to see only
the results from the last run (which is what you report).
-
If you set overwrite_files to False, and did not move the file away,
you will get an error upon repeated calls to Simulate(), as the file
to be created by the backend already exists.
-
When using the recording backend memory
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
overwrite_files is not even considered, as that does not actually
write files.
-
Data just accumulates in the recording backend memory until it is
explicitly cleared by running Input_SD.n_events = 0.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
Dear NEST users,
I am trying to debug my network and I found that my spike_recorder with
*ascii* option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms) and then
do some structure changes and then re-run for another phase of a few ms
(say 4000. ms)
In the compact version, I am trying to see how this sample code would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate'
: 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but for
memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and False.
In the previous version (v2.20) I also used the same approach and had no
issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of NESTv2.x and
NESTv3.x
Am I missing something? It's really important for my work, I hope I could
find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
12:06 p.m.
I am not sure if this change is documented or is an
intuitive change.
However, it would be nice to have it on the comparison page.
Already working on it :-)
Cheers,
Jochen!
On Tue, Nov 30, 2021 at 9:36 AM Jochen Martin Eppler
<j.eppler(a)fz-juelich.de> wrote:
Dear Maryada,
[Re-sending with properly rendered Markdown. Sorry for the noise]
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
I re-tested my code and realized that I
didn't set
overwrite_files to False properly, so you are right, I received
the error.
Good.
However, since I am importing my code from
NEST 2.xx version to
v3.1 and all my previous implementation also have similar
two-phase simulations, this was not the case earlier. To my
understanding, unless you call /ResetKernel/, it is a single
simulation and should not overwrite the current file as is the
case when recording in memory.
The behavior has indeed been as you describe with NEST 2.x.
However, it was changed with NEST 3.0.
In the 2.x series of NEST, all recording devices had a flag
|close_after_simulate|, which defaulted to |False|. As this flag
resulted in a high bookkeeping complexity internally and was
confusing to many users, it has been removed in favor of the
clearer semantics that each call to |Simulate()| produces its own
file.
I ran your script (with syntax changes) on my
laptop which only
has nest 2.20 version and the attached data file contains
recording for the whole run and not just the last subpart of
simulation. I would understand if the functionality is changed in
the latest version, however, I was wondering if there is a
possible work-around. I surely cannot manually store all the data
given that I also record from a multimeter and these are
memory-consuming simulations.
If you want to run repetitive simulations /and/ get all your data
(either in separate files or a single file) with NEST 3.x, you
have several possibilities:
1.
You change the kernel property |data_path| inbetween calls to
|Simulate()|. This would let NEST put the data for /all/
recorders in different directories. You can use |os.mkdir| to
create the directories from within your simulation script:
|import nest, os # Create nodes # Connect nodes
os.mkdir("path_for_sim1") nest.data_path = "path_for_sim1"
nest.Simulate(1000) os.mkdir("path_for_sim2") nest.data_path =
"path_for_sim2" nest.Simulate(2000) |
2.
You can move your files to somewhere safe using |os.rename()|,
instead of setting |data_path|.
3.
You can set a different |data_prefix| between calls to
|Simulate()| (i.e., |nest.data_prefix = "sim1_"|,
|nest.data_prefix = "sim2_"|, …). That way you wouldn’t have
to create directories, but your data files would instead be
prefixed as to your choice.
4.
You can use |Prepare()|, |Run()|, |Cleanup()| instead of
|Simulate()| as explained in the Guide on running simulations
<https://nest-simulator.readthedocs.io/en/latest/guides/running_simulations.html#split-a-simulation-into-multiple-intervals>.
For each call to |Run()|, the recorders would actually append
to the files, as those are only closed in |Cleanup()| (and
re-opened in |Prepare()|). In fact, |Simulate()| is actually
just a wrapper around the three aforementioned functions.
Cheers,
Jochen!
On Mon, Nov 29, 2021 at 10:24 PM Jochen Martin Eppler
<j.eppler(a)fz-juelich.de> wrote:
Dear Maryada,
I have just played a bit with your example (see attachment
for a slightly extended and reformatted version) and as far
as I can see, NEST behaves exactly as expected (and documented).
Please let me explain:
*
When using the recording backend |ascii|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute |overwrite_files|
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to |True|, you have to move files away at the
location in your script saying |do something|, otherwise
the data files get overwritten by each new call to
|Simulate()| and you consequently get to see only the
results from the last run (which is what you report).
*
If you set |overwrite_files| to |False|, and did not move
the file away, you will get an error upon repeated calls
to |Simulate()|, as the file to be created by the backend
already exists.
*
When using the recording backend |memory|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
|overwrite_files| is not even considered, as that does
not actually write files.
*
Data just accumulates in the recording backend |memory|
until it is explicitly cleared by running
|Input_SD.n_events = 0|.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org Dear NEST users,
I am trying to debug my network and I found that my
spike_recorder with /ascii/ option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000.
ms) and then do some structure changes and then re-run for
another phase of a few ms (say 4000. ms)
In the compact version, I am trying to see how this sample
code would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device =
nest.Create("poisson_generator",params={'start':
0.,'rate' :
100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond
but for memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True
and False.
In the previous version (v2.20) I also used the same
approach and had no issues. Is there a newer way to do such
a simulation paradigm?
I didn't find anything on this on the comparison page of
NESTv2.x and NESTv3.x
Am I missing something? It's really important for my work, I
hope I could find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR
B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt
(Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid
Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
12:15 p.m.
Hi Robin!
We consciously decided against that many many times to avoid a fat
interface and ending up in a situation where we have to maintain
hundreds of special functions in our interfaces (C++, Python, SLI). I
mean, where would you stop? Input_SD.events["times"] could be
get_spike_times(), Input_SD.events could be get_events(), and so on...
Performing "function calls" via SetStatus is an easy way of achieving a
minimal and maintainable interface and I have not yet seen an easier way
to achieve the same. I am, however, open to suggestions!
On this specific topic, the documentation should definitely improved,
though.
Cheers,
Jochen!
On 30.11.21 12:04, Robin Gilbert De Schepper wrote:
As an aside, if `Input_SD.n_events = 0` is the
canonical way of
clearing a spike recorder, maybe a `clear` function could be added.
That's a bit easier to document and find?
On Mon, 29 Nov 2021, 22:24 Jochen Martin Eppler,
<j.eppler(a)fz-juelich.de> wrote:
Dear Maryada,
I have just played a bit with your example (see attachment for a
slightly extended and reformatted version) and as far as I can
see, NEST behaves exactly as expected (and documented).
Please let me explain:
*
When using the recording backend |ascii|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_ascii.html>
with the kernel attribute |overwrite_files|
<https://nest-simulator.readthedocs.io/en/v3.1/ref_material/pynest_apis.html#nest.NestModule.overwrite_files>
set to |True|, you have to move files away at the location in
your script saying |do something|, otherwise the data files
get overwritten by each new call to |Simulate()| and you
consequently get to see only the results from the last run
(which is what you report).
*
If you set |overwrite_files| to |False|, and did not move the
file away, you will get an error upon repeated calls to
|Simulate()|, as the file to be created by the backend already
exists.
*
When using the recording backend |memory|
<https://nest-simulator.readthedocs.io/en/v3.1/models/recording_backend_memory.html>,
|overwrite_files| is not even considered, as that does not
actually write files.
*
Data just accumulates in the recording backend |memory| until
it is explicitly cleared by running |Input_SD.n_events = 0|.
See the linked backend documentation pages for more information.
I hope this helps to clear up things.
Cheers,
Jochen!
On 29.11.21 18:32, Maryada Maryada wrote:
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
Dear NEST users,
I am trying to debug my network and I found that my
spike_recorder with /ascii/ option isn't working as expected.
I am simulating a network for a few milliseconds (say 1000. ms)
and then do some structure changes and then re-run for another
phase of a few ms (say 4000. ms)
In the compact version, I am trying to see how this sample code
would work.
import nest
nest.ResetKernel()
nest.SetKernelStatus({
"overwrite_files": True,
"data_path": './Debug-Log',
"data_prefix": 'DEBUG-SPIKE-REC',
})
noise_device = nest.Create("poisson_generator",params={'start':
0.,'rate' : 100.})
sd_params = {
"record_to": 'ascii',
"label": "Input_spike_recorder"
}
Input_SD = nest.Create("spike_recorder",1, params=sd_params)
nest.Connect(noise_device,Input_SD)
nest.Simulate(100.)
# do something here
nest.Simulate(100.)
*Issue:* I only get the data recorded from 100th millisecond but
for memory, I get all the data from 0-200 ms.
This behavior is consistent for both overwrite_files as True and
False.
In the previous version (v2.20) I also used the same approach and
had no issues. Is there a newer way to do such a simulation paradigm?
I didn't find anything on this on the comparison page of
NESTv2.x and NESTv3.x
Am I missing something? It's really important for my work, I hope
I could find a solution.
--
Thanks and Regards
*Maryada*
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
--
Dr. Jochen Martin Eppler
Phone: +49(2461)61-96653
----------------------------------
Simulation Laboratory Neuroscience
Jülich Supercomputing Centre
Institute for Advanced Simulation
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
NEST Users mailing list -- users(a)nest-simulator.org
To unsubscribe send an email to users-leave(a)nest-simulator.org
_______________________________________________
NEST Users mailing list --users(a)nest-simulator.org
To unsubscribe send an email tousers-leave(a)nest-simulator.org
878
days inactive
879
days old
8 comments
3 participants
participants (3)
-
Jochen Martin Eppler -
Maryada Maryada -
Robin Gilbert De Schepper