Simulate experiments with Scikit-NeuroMSI

This tutorial covers the basic pipeline for simulating multisensory integration experiments using Scikit-NeuroMSI. We show how ParameterSweep class works and provide details about the NDResultCollection object and its main methods.

Parameter Sweeps

You can simulate multisensory integration experiments using the ParameterSweep class. This class allows to run multiple model executions while changing (‘sweeping’) a specific parameter while keeping the others constant.

Here we simulate the responses of the network model developed by Cuppini et al. (2017) on a spatial disparity paradigm. In this simulation, the visual position changes at each experimental condition, while the visual position remains constant:

from skneuromsi.sweep import ParameterSweep
from skneuromsi.neural import Cuppini2017
import numpy as np

## Model setup
model_cuppini2017 = Cuppini2017(neurons=90, position_range=(0, 90))

## Experiment setup
spatial_disparities = np.array([-24, -12, -6, -3, 3, 6, 12, 24])

sp_cuppini2017 = ParameterSweep(
    model=model_cuppini2017,
    target="visual_position",
    repeat=1,
    range=45 + spatial_disparities,
)

Note that the ParameterSweep class requires to specify a target run parameter (here visual_position) and the its range of values.

Now we call the run method of the ParameterSweep object. Here we can define the run parameters of the model for each iteration:

## Experiment run
res_sp_cuppini2017 = sp_cuppini2017.run(
    auditory_position=45, auditory_sigma=32, visual_sigma=4
)

res_sp_cuppini2017

<NDResultCollection 'ParameterSweep' len=8>

The output of the ParameterSweep run method is an NDResultCollection. Let’s explore this object:

vars(res_sp_cuppini2017)

{'_name': 'ParameterSweep',
 '_cndresults': array([<CompressedNDResult '21.4 MB' (97.60%)>,
        <CompressedNDResult '21.5 MB' (98.02%)>,
        <CompressedNDResult '21.4 MB' (97.73%)>,
        <CompressedNDResult '21.4 MB' (97.40%)>,
        <CompressedNDResult '21.4 MB' (97.41%)>,
        <CompressedNDResult '21.4 MB' (97.73%)>,
        <CompressedNDResult '21.5 MB' (98.02%)>,
        <CompressedNDResult '21.4 MB' (97.60%)>], dtype=object),
 '_tqdm_cls': tqdm.auto.tqdm,
 '_cache': <_cache {'dims', 'time_ranges', 'position_ranges', 'causes', 'mtypes', 'modes', 'run_parameters', 'nmaps', 'output_mode', 'time_resolutions', 'position_resolutions', 'run_parameters_values', 'modes_variances_sum', 'mnames'}>}

As it name suggests, this objects holds a collection of multiple NDResult objects, which can be accessed by indexing the NDResultCollection object:

res_sp_cuppini2017[0]

<NDResult 'Cuppini2017', modes=['auditory' 'visual' 'multi'], times=10000, positions=90, positions_coordinates=1, causes=2>

The NDResultCollection also holds the information of each model execution, which can be accessed by its internal method disparity_matrix. This method outputs the parameter values of each model run during this iterative process. Note how all parameters remain fixed but visual_position (our target parameter):

res_sp_cuppini2017.disparity_matrix()

Parameters	auditory_position	visual_position	auditory_sigma	visual_sigma	auditory_intensity	visual_intensity	auditory_duration	auditory_onset	auditory_stim_n	visual_duration	...	auditory_soa	visual_soa	noise	noise_level	feedforward_weight	cross_modal_weight	causes_kind	causes_dim	causes_peak_threshold	causes_peak_distance
0	45	21	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
1	45	33	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
2	45	39	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
3	45	42	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
4	45	48	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
5	45	51	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
6	45	57	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None
7	45	69	32	4	28	27	None	0	1	None	...	None	None	False	0.4	18	1.4	count	space	0.15	None

8 rows × 22 columns

Sensory bias computation

A common metric obtained from multisensory integration experiments is the cross-modal sensory bias. This metric captures the influence of one sensory modality over the responses observed in another sensory modality.

We can compute cross-modal sensory bias by calling the method bias from the NDResultCollection object:

res_sp_cuppini2017.bias(
    influence_parameter="auditory_position", mode="auditory"
)

Changing parameter	visual_position
Influence parameter	auditory_position
Iteration	0
Disparity
-24	0.000000
-12	0.833333
-6	0.833333
-3	1.000000
3	1.000000
6	0.833333
12	0.833333
24	0.000000

In this method, the influence_parameter argument refers to the parameter that is being influenced by the parameter that was manipulated (i.e. the target defined in the ParameterSweep object), and mode refers to the modality of the parameter that is being influenced.

The output of the bias method reveals that the auditory position detected by the model is biased when the stimuli are closer together (lower disparity), and this effect is reduced as the spatial disparity increases.

Note: In this experiment simulation, the sensory bias is measured as the difference between the position of the auditory stimulus and the position detected by the model, divided by the distance between the auditory and visual stimuli.

Causal inference computation

Another frequently recorded metric in multisensory integration experiments is the causal inference responses provided by participants. This metric assesses whether participants attribute the presented stimuli to a common source or to distinct origins. We can compute causal inference responses by calling the method causes from the NDResultCollection object:

res_sp_cuppini2017.causes()

	Causes
visual_position
21	0.0
33	1.0
39	1.0
42	1.0
48	1.0
51	1.0
57	1.0
69	0.0

The output of the causes method shows that the model reports a single cause only for visual positions that are close to the position of the auditory stimuli (fixed at 45).

Note: In this experiment simulation, causal inference is defined as the number of unique causes detected by the model out of the multiple sensory inputs.

Processing Strategy

You can personalize ParameterSweep outputs by creating processing strategies with the ProcessingStrategyABC class. This is useful when we aim to avoid outputting the default NDResultCollection (for memory efficiency) or when we need to define a specific model readout that mirrors the participants’ responses in the simulated experiment.

Here we define a processing strategy to extract the auditory position detected by the model:

Note: Here the position detected by the model is defined as the spatial point where the model registered the maximal neural activity.

from skneuromsi.sweep import ProcessingStrategyABC


class AuditoryPositionProcessingStrategy(ProcessingStrategyABC):
    def map(self, result):
        auditory_position = result.stats.dimmax(modes="auditory")["positions"]
        return auditory_position

    def reduce(self, results, **kwargs):
        return np.array(results)

Next we can input our AuditoryPositionProcessingStrategy to a new ParameterSweep object:

causes_sp_cuppini2017 = ParameterSweep(
    model=model_cuppini2017,
    target="visual_position",
    repeat=1,
    range=45 + spatial_disparities,
    processing_strategy=AuditoryPositionProcessingStrategy(),
)

Now we can run our ParameterSweep object with the customized processing_strategy, with the same parameterization as before:

res = causes_sp_cuppini2017.run(
    auditory_position=45, auditory_sigma=32, visual_sigma=4
)
res

array([45, 35, 40, 42, 48, 50, 55, 45])

The output of our experiment simluation now is a numpy.array as we defined in our processing strategy. We observe that the auditory position detected by the model changes as the visual stimuli is manipulated (although in the simulations the auditory stimuli was always fixed at 45).

Refer to the API documentation for more details about the ParameterSweep module.