Hello,
I am trying to access the list of approx. 13 000 000 synapses in my network. However, accessing any of the values in SynapseCollection runs longer than my whole simulation and can cause out of memory issues.
In the following snippet, just the `conns.get('weight')` line takes my computer 13 minutes to compute and uses over 5 GB of memory, even though the data I am interested in should not be larger than 50 MB.
importnest neurons= nest.Create('iaf_psc_exp', 20000) conn_dict= {'rule': 'fixed_indegree', 'indegree': 300} nest.Connect(neurons, neurons, conn_dict, syn_spec={'weight': 1.0}) conns= nest.GetConnections() conns.get('weight')
How should I efficiently access/export the `source`, `target`, `weight` and `distance` synaptic variables without unnecessary time or memory overhead?
Thank you.
Best regards,
Maurycy Miękus
Hi Maurycy,
Unfortunately, NEST does not have an efficient way of extracting single information values from large sets of connections at the moment.
What you could do at the Python level to reduce the memory overhead would be to loop over the source neurons.
The code below ran in about 3.5 minutes on my MacBook Pro and used slighly less than 500 MB (Jupyter notebook included):
The total number of connections is always available as nest.num_connections so you don’t need to calculate it to pre-size the numpy array if you want all connections.
If you frequently need to do this operation on large networks, we could consider a more efficient implementation, but that would require work a the C++ level. Would you be interested in contributing to that?
Best regards, Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser Research Committee Chair, Faculty of Science and Technology Head, Department of Data Science
Department of Data Science Faculty of Science and Technology Norwegian University of Life Sciences PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560 Email hans.ekkehard.plesser@nmbu.nomailto:hans.ekkehard.plesser@nmbu.no Home http://arken.nmbu.no/~plesser
From: M. Maurycy Miękus miekus@em.uni-frankfurt.de Date: Wednesday, 15 October 2025 at 09:30 To: users@nest-simulator.org users@nest-simulator.org Subject: [NEST Users] SynapseCollection.get() causes overhead Some people who received this message don't often get email from miekus@em.uni-frankfurt.de. Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
Hello,
I am trying to access the list of approx. 13 000 000 synapses in my network. However, accessing any of the values in SynapseCollection runs longer than my whole simulation and can cause out of memory issues.
In the following snippet, just the `conns.get('weight')` line takes my computer 13 minutes to compute and uses over 5 GB of memory, even though the data I am interested in should not be larger than 50 MB. import nest neurons = nest.Create('iaf_psc_exp', 20000) conn_dict = {'rule': 'fixed_indegree', 'indegree': 300} nest.Connect(neurons, neurons, conn_dict, syn_spec={'weight': 1.0}) conns = nest.GetConnections() conns.get('weight')
How should I efficiently access/export the `source`, `target`, `weight` and `distance` synaptic variables without unnecessary time or memory overhead?
Thank you.
Best regards,
Maurycy Miękus
-
Hans Ekkehard Plesser -
M. Maurycy Miękus