{ "cells": [ { "cell_type": "markdown", "id": "7fc6ea05", "metadata": {}, "source": [ "# Simulate experiments with Scikit-NeuroMSI" ] }, { "cell_type": "markdown", "id": "ff1cf4dc", "metadata": {}, "source": [ "This tutorial covers the basic pipeline for simulating multisensory integration experiments using `Scikit-NeuroMSI`. We show how `ParameterSweep` class works and provide details about the `NDResultCollection` object and its main methods." ] }, { "cell_type": "markdown", "id": "0f4c978b", "metadata": {}, "source": [ "## Parameter Sweeps" ] }, { "cell_type": "markdown", "id": "a4df1239", "metadata": {}, "source": [ "You can simulate multisensory integration experiments using the `ParameterSweep` class. This class allows to run multiple model executions while changing ('sweeping') a specific parameter while keeping the others constant." ] }, { "cell_type": "markdown", "id": "50c61609", "metadata": {}, "source": [ "Here we simulate the responses of the network model developed by Cuppini et al. (2017) on a spatial disparity paradigm. In this simulation, the visual position changes at each experimental condition, while the visual position remains constant: " ] }, { "cell_type": "code", "execution_count": 1, "id": "73577eba", "metadata": {}, "outputs": [], "source": [ "from skneuromsi.sweep import ParameterSweep\n", "from skneuromsi.neural import Cuppini2017\n", "import numpy as np\n", "\n", "## Model setup\n", "model_cuppini2017 = Cuppini2017(neurons=90, position_range=(0, 90))\n", "\n", "## Experiment setup\n", "spatial_disparities = np.array([-24, -12, -6, -3, 3, 6, 12, 24])\n", "\n", "sp_cuppini2017 = ParameterSweep(\n", " model=model_cuppini2017,\n", " target=\"visual_position\",\n", " repeat=1,\n", " range=45 + spatial_disparities,\n", ")" ] }, { "cell_type": "markdown", "id": "ce1b82e3", "metadata": {}, "source": [ "Note that the `ParameterSweep` class requires to specify a `target` run parameter (here `visual_position`) and the its `range` of values." ] }, { "cell_type": "markdown", "id": "cf626b6a", "metadata": {}, "source": [ "Now we call the `run` method of the `ParameterSweep` object. Here we can define the run parameters of the model for each iteration:" ] }, { "cell_type": "code", "execution_count": 2, "id": "f2ea8688", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "ebec91c95de44bd6955e4c24bf2e292b", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sweeping 'visual_position': 0%| | 0/8 [00:00" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Experiment run\n", "res_sp_cuppini2017 = sp_cuppini2017.run(\n", " auditory_position=45, auditory_sigma=32, visual_sigma=4\n", ")\n", "\n", "res_sp_cuppini2017" ] }, { "cell_type": "markdown", "id": "29166ec5", "metadata": {}, "source": [ "The output of the `ParameterSweep` `run` method is an `NDResultCollection`. Let's explore this object:" ] }, { "cell_type": "code", "execution_count": 3, "id": "8bd43c67", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_name': 'ParameterSweep',\n", " '_cndresults': array([,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ], dtype=object),\n", " '_tqdm_cls': tqdm.auto.tqdm,\n", " '_cache': <_cache {'dims', 'time_ranges', 'position_ranges', 'causes', 'mtypes', 'modes', 'run_parameters', 'nmaps', 'output_mode', 'time_resolutions', 'position_resolutions', 'run_parameters_values', 'modes_variances_sum', 'mnames'}>}" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vars(res_sp_cuppini2017)" ] }, { "cell_type": "markdown", "id": "ab4cc1f7", "metadata": {}, "source": [ "As it name suggests, this objects holds a collection of multiple `NDResult` objects, which can be accessed by indexing the `NDResultCollection` object:" ] }, { "cell_type": "code", "execution_count": 4, "id": "741af30c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res_sp_cuppini2017[0]" ] }, { "cell_type": "markdown", "id": "e6c79a8e", "metadata": {}, "source": [ "The `NDResultCollection` also holds the information of each model execution, which can be accessed by its internal method `disparity_matrix`. This method outputs the parameter values of each model run during this iterative process. Note how all parameters remain fixed but `visual_position` (our `target` parameter):" ] }, { "cell_type": "code", "execution_count": 5, "id": "27b8bf59", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Parametersauditory_positionvisual_positionauditory_sigmavisual_sigmaauditory_intensityvisual_intensityauditory_durationauditory_onsetauditory_stim_nvisual_duration...auditory_soavisual_soanoisenoise_levelfeedforward_weightcross_modal_weightcauses_kindcauses_dimcauses_peak_thresholdcauses_peak_distance
045213242827None01None...NoneNoneFalse0.4181.4countspace0.15None
145333242827None01None...NoneNoneFalse0.4181.4countspace0.15None
245393242827None01None...NoneNoneFalse0.4181.4countspace0.15None
345423242827None01None...NoneNoneFalse0.4181.4countspace0.15None
445483242827None01None...NoneNoneFalse0.4181.4countspace0.15None
545513242827None01None...NoneNoneFalse0.4181.4countspace0.15None
645573242827None01None...NoneNoneFalse0.4181.4countspace0.15None
745693242827None01None...NoneNoneFalse0.4181.4countspace0.15None
\n", "

8 rows × 22 columns

\n", "
" ], "text/plain": [ "Parameters auditory_position visual_position auditory_sigma visual_sigma \\\n", "0 45 21 32 4 \n", "1 45 33 32 4 \n", "2 45 39 32 4 \n", "3 45 42 32 4 \n", "4 45 48 32 4 \n", "5 45 51 32 4 \n", "6 45 57 32 4 \n", "7 45 69 32 4 \n", "\n", "Parameters auditory_intensity visual_intensity auditory_duration \\\n", "0 28 27 None \n", "1 28 27 None \n", "2 28 27 None \n", "3 28 27 None \n", "4 28 27 None \n", "5 28 27 None \n", "6 28 27 None \n", "7 28 27 None \n", "\n", "Parameters auditory_onset auditory_stim_n visual_duration ... \\\n", "0 0 1 None ... \n", "1 0 1 None ... \n", "2 0 1 None ... \n", "3 0 1 None ... \n", "4 0 1 None ... \n", "5 0 1 None ... \n", "6 0 1 None ... \n", "7 0 1 None ... \n", "\n", "Parameters auditory_soa visual_soa noise noise_level feedforward_weight \\\n", "0 None None False 0.4 18 \n", "1 None None False 0.4 18 \n", "2 None None False 0.4 18 \n", "3 None None False 0.4 18 \n", "4 None None False 0.4 18 \n", "5 None None False 0.4 18 \n", "6 None None False 0.4 18 \n", "7 None None False 0.4 18 \n", "\n", "Parameters cross_modal_weight causes_kind causes_dim causes_peak_threshold \\\n", "0 1.4 count space 0.15 \n", "1 1.4 count space 0.15 \n", "2 1.4 count space 0.15 \n", "3 1.4 count space 0.15 \n", "4 1.4 count space 0.15 \n", "5 1.4 count space 0.15 \n", "6 1.4 count space 0.15 \n", "7 1.4 count space 0.15 \n", "\n", "Parameters causes_peak_distance \n", "0 None \n", "1 None \n", "2 None \n", "3 None \n", "4 None \n", "5 None \n", "6 None \n", "7 None \n", "\n", "[8 rows x 22 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res_sp_cuppini2017.disparity_matrix()" ] }, { "cell_type": "markdown", "id": "8ba1e63b", "metadata": {}, "source": [ "## Sensory bias computation" ] }, { "cell_type": "markdown", "id": "34155661", "metadata": {}, "source": [ "A common metric obtained from multisensory integration experiments is the cross-modal sensory bias. This metric captures the influence of one sensory modality over the responses observed in another sensory modality." ] }, { "cell_type": "markdown", "id": "423ee49c", "metadata": {}, "source": [ "We can compute cross-modal sensory bias by calling the method `bias` from the `NDResultCollection` object:" ] }, { "cell_type": "code", "execution_count": 6, "id": "de26d56f", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e1518c7fcded4a369a7e971bdbc96e88", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Calculating biases: 0%| | 0/8 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Changing parametervisual_position
Influence parameterauditory_position
Iteration0
Disparity
-240.000000
-120.833333
-60.833333
-31.000000
31.000000
60.833333
120.833333
240.000000
\n", "" ], "text/plain": [ "Changing parameter visual_position\n", "Influence parameter auditory_position\n", "Iteration 0\n", "Disparity \n", "-24 0.000000\n", "-12 0.833333\n", "-6 0.833333\n", "-3 1.000000\n", " 3 1.000000\n", " 6 0.833333\n", " 12 0.833333\n", " 24 0.000000" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res_sp_cuppini2017.bias(\n", " influence_parameter=\"auditory_position\", mode=\"auditory\"\n", ")" ] }, { "cell_type": "markdown", "id": "1c966113", "metadata": {}, "source": [ "In this method, the `influence_parameter` argument refers to the parameter that is being influenced by the parameter that was manipulated (i.e. the `target` defined in the ParameterSweep object), and `mode` refers to the modality of the parameter that is being influenced. " ] }, { "cell_type": "markdown", "id": "fdb8f9d0", "metadata": {}, "source": [ "The output of the `bias` method reveals that the auditory position detected by the model is biased when the stimuli are closer together (lower disparity), and this effect is reduced as the spatial disparity increases." ] }, { "cell_type": "markdown", "id": "d49cd43b", "metadata": {}, "source": [ "> **Note**: In this experiment simulation, the sensory bias is measured as the difference between the position of the auditory stimulus and the position detected by the model, divided by the distance between the auditory and visual stimuli." ] }, { "cell_type": "markdown", "id": "c77a3304", "metadata": {}, "source": [ "## Causal inference computation" ] }, { "cell_type": "markdown", "id": "9d6abd91", "metadata": {}, "source": [ "Another frequently recorded metric in multisensory integration experiments is the causal inference responses provided by participants. This metric assesses whether participants attribute the presented stimuli to a common source or to distinct origins. We can compute causal inference responses by calling the method `causes` from the `NDResultCollection` object:" ] }, { "cell_type": "code", "execution_count": 7, "id": "db18fa27", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Causes
visual_position
210.0
331.0
391.0
421.0
481.0
511.0
571.0
690.0
\n", "
" ], "text/plain": [ " Causes\n", "visual_position \n", "21 0.0\n", "33 1.0\n", "39 1.0\n", "42 1.0\n", "48 1.0\n", "51 1.0\n", "57 1.0\n", "69 0.0" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res_sp_cuppini2017.causes()" ] }, { "cell_type": "markdown", "id": "f4e07935", "metadata": {}, "source": [ "The output of the `causes` method shows that the model reports a single cause only for visual positions that are close to the position of the auditory stimuli (fixed at 45). " ] }, { "cell_type": "markdown", "id": "9833252d", "metadata": {}, "source": [ "> **Note**: In this experiment simulation, causal inference is defined as the number of unique causes detected by the model out of the multiple sensory inputs." ] }, { "cell_type": "markdown", "id": "0bd38e69", "metadata": {}, "source": [ "## Processing Strategy" ] }, { "cell_type": "markdown", "id": "553d5f7d", "metadata": {}, "source": [ "You can personalize `ParameterSweep` outputs by creating processing strategies with the `ProcessingStrategyABC` class. This is useful when we aim to avoid outputting the default NDResultCollection (for memory efficiency) or when we need to define a specific model readout that mirrors the participants' responses in the simulated experiment.\n", "\n", "Here we define a processing strategy to extract the auditory position detected by the model:\n", "\n", "> **Note**: Here the position detected by the model is defined as the spatial point where the model registered the maximal neural activity." ] }, { "cell_type": "code", "execution_count": 8, "id": "45be33ff", "metadata": {}, "outputs": [], "source": [ "from skneuromsi.sweep import ProcessingStrategyABC\n", "\n", "\n", "class AuditoryPositionProcessingStrategy(ProcessingStrategyABC):\n", " def map(self, result):\n", " auditory_position = result.stats.dimmax(modes=\"auditory\")[\"positions\"]\n", " return auditory_position\n", "\n", " def reduce(self, results, **kwargs):\n", " return np.array(results)" ] }, { "cell_type": "markdown", "id": "32cd7a1b", "metadata": {}, "source": [ "Next we can input our `AuditoryPositionProcessingStrategy` to a new `ParameterSweep` object:" ] }, { "cell_type": "code", "execution_count": 9, "id": "d973b1b7", "metadata": {}, "outputs": [], "source": [ "causes_sp_cuppini2017 = ParameterSweep(\n", " model=model_cuppini2017,\n", " target=\"visual_position\",\n", " repeat=1,\n", " range=45 + spatial_disparities,\n", " processing_strategy=AuditoryPositionProcessingStrategy(),\n", ")" ] }, { "cell_type": "markdown", "id": "b4829d7f", "metadata": {}, "source": [ "Now we can run our `ParameterSweep` object with the customized `processing_strategy`, with the same parameterization as before: " ] }, { "cell_type": "code", "execution_count": 10, "id": "560b10b1", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9d9764b9230f4e9eab5423d684cbc534", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sweeping 'visual_position': 0%| | 0/8 [00:00