Dear Xavier,
I think the most efficient way would be to drop the entires “src_iter” loop and simply place the entire source node collection in conn_filter, just as the target collection. Then,
one would only need a single GetConnections() call and would avoid a lot of duplicate parsing of the connection tables.
The one downside with that approach is that one may need quite some memory to store the connectome returned by GetConnections(). If we assume a layer size of 10000 neurons and a
connectivity of 10% within the layer, we get 10 million connections. Each of those requires 40B in the representation returned by GetConnections(), so 400 MB in total. But this is manageable in most cases. And if the simulation is distributed on say 8 MPI
ranks, it would only be 50 MB per rank, and only until DumpLayerConnections completes.
It would be nice if you could try to implement this. Could you try to use modern C++-style loops ( for ( auto& conn : …) )?
The proper solution to avoid the memory overhead would be for GetConnections not to return the actual collection of connections, but actually a iterator over it so that we can avoid
the memory overhead entirely. But that would be a larger change to the code.
Best,
Hans Ekkehard
--
Prof. Dr. Hans Ekkehard Plesser
Department of Data Science
Faculty of Science and Technology
Norwegian University of Life Sciences
PO Box 5003, 1432 Aas, Norway
Phone +47 6723 1560
Email hans.ekkehard.plesser@nmbu.no
Home http://arken.nmbu.no/~plesser