{ "cells": [ { "cell_type": "markdown", "id": "18478834", "metadata": {}, "source": [ "# Sound-Induced Flash Illusion" ] }, { "cell_type": "markdown", "id": "59ca16a7", "metadata": {}, "source": [ "In this example we show how we can compare models in an audio-visual temporal disparity task to reproduce the Sound-Induced Flash Illusion as shown in:\n", "\n", ">Cuppini, C., Magosso, E., Bolognini, N., Vallar, G., & Ursino, M. (2014). A neurocomputational analysis of the sound-induced flash illusion. Neuroimage, 92, 248-266. doi: https://doi.org/10.1016/j.neuroimage.2014.02.001\n", "\n", ">Zhu, H., Beierholm, U., & Shams, L. (2024). The overlooked role of unisensory precision in multisensory research. Current Biology, 34(6), R229-R231. doi: https://doi.org/10.1016/j.cub.2024.01.057 \n", "\n", ">Paredes, R., Ferri, F., Romei, V., & Seriès, P. (2025). Increased excitation enhances the sound-induced flash illusion by impairing multisensory causal inference in the schizophrenia spectrum. Schizophrenia Research, 283, 1-10. doi: https://doi.org/10.1016/j.schres.2025.06.007" ] }, { "cell_type": "markdown", "id": "4b66ebb4", "metadata": {}, "source": [ "## Implementation of models" ] }, { "cell_type": "markdown", "id": "a648951f", "metadata": {}, "source": [ "For this paradigm we call the relevant models that account for temporal multisensory integration:" ] }, { "cell_type": "code", "execution_count": null, "id": "ca237509", "metadata": {}, "outputs": [], "source": [ "from skneuromsi.bayesian import Zhu2024\n", "from skneuromsi.neural import Cuppini2014\n", "from skneuromsi.neural import Paredes2025\n", "\n", "model_cuppini = Cuppini2014(\n", " time_range=(0, 550), neurons=30, position_range=(0, 30)\n", ")\n", "\n", "model_zhu = Zhu2024(\n", " n=100000,\n", " time_range=(0, 1000),\n", " time_res=1,\n", " numerosity_range=(0, 3),\n", " numerosity_res=1,\n", ")\n", "\n", "model_paredes = Paredes2025(\n", " time_range=(0, 550),\n", " neurons=30,\n", " position_range=(0, 30),\n", " tau=(6.560e00, 9.191e00, 1.200e02),\n", ")" ] }, { "cell_type": "markdown", "id": "0448eb75", "metadata": {}, "source": [ "## Experiment setup" ] }, { "cell_type": "markdown", "id": "f4e05a1e", "metadata": {}, "source": [ "We are interested to explore the visual illusion rate and causal inference responses when two auditory stimuli (beeps) and one visual stimulus (flash) are presented at different stimuli onset asynchronies (SOA)." ] }, { "cell_type": "markdown", "id": "002dd481", "metadata": {}, "source": [ "To simulate the visual illusion rate we define customized `ProcessingStrategy` routines. These routines depend on specific helper functions that are necessary to handle inputs and outputs of each model." ] }, { "cell_type": "code", "execution_count": 80, "id": "e5758ce0", "metadata": {}, "outputs": [], "source": [ "import itertools\n", "import numpy as np\n", "from skneuromsi.sweep import ProcessingStrategyABC\n", "from scipy.signal import find_peaks\n", "\n", "\n", "def calculate_auditory_times(visual_time, soas, auditory_stim_duration=17):\n", " a_times = []\n", " for soa in soas:\n", " if soa > 0:\n", " a_start = visual_time - auditory_stim_duration / 2\n", " a_end = (\n", " visual_time\n", " + auditory_stim_duration / 2\n", " + soa\n", " + auditory_stim_duration\n", " )\n", " if soa < 0:\n", " a_start = visual_time - auditory_stim_duration / 2 + soa - 17\n", " a_end = visual_time + auditory_stim_duration / 2\n", " a_stim_train_duration = a_end - a_start\n", " a_time = a_start + a_stim_train_duration / 2\n", " a_times.append(a_time)\n", " return np.array(a_times)\n", "\n", "\n", "def calculate_two_peaks_probability(visual_peaks_values):\n", " combinations = list(\n", " itertools.chain.from_iterable(\n", " itertools.combinations(visual_peaks_values, i + 2)\n", " for i in range(len(visual_peaks_values))\n", " )\n", " )\n", "\n", " probs_array = np.array([], dtype=np.float16)\n", "\n", " for i in combinations:\n", " probs_array = np.append(probs_array, np.array(i).prod())\n", "\n", " return probs_array.sum() / probs_array.size\n", "\n", "\n", "class BayesianIllusionRateProcessingStrategy(ProcessingStrategyABC):\n", " def map(self, result):\n", " e_visual_numerosity = result.e_[\"visual_numerosity\"]\n", " two_flashes_prop = e_visual_numerosity[2]\n", " del result._nddata\n", " return two_flashes_prop\n", "\n", " def reduce(self, results, **kwargs):\n", " return np.array(results)\n", "\n", "\n", "class NeuralIllusionRateProcessingStrategy(ProcessingStrategyABC):\n", " def map(self, result):\n", " max_pos = result.stats.dimmax().positions\n", " visual_activity = (\n", " result.get_modes(include=\"visual\")\n", " .query(f\"positions=={max_pos}\")\n", " .visual.values\n", " )\n", " peaks, peaks_props = find_peaks(\n", " visual_activity,\n", " height=0.15,\n", " prominence=0.15,\n", " distance=36 / 0.01,\n", " )\n", " if len(peaks) < 2:\n", " p_two_flashes = 0\n", " else:\n", " p_two_flashes = calculate_two_peaks_probability(\n", " peaks_props[\"peak_heights\"]\n", " )\n", " del visual_activity, peaks, peaks_props, max_pos\n", " del result._nddata\n", " return p_two_flashes\n", "\n", " def reduce(self, results, **kwargs):\n", " return np.array(results, dtype=np.float16)" ] }, { "cell_type": "markdown", "id": "3a50e5e4", "metadata": {}, "source": [ "Now we setup the experiment simulation for each model using the `ParameterSweep` class:" ] }, { "cell_type": "code", "execution_count": 84, "id": "69c04b60", "metadata": {}, "outputs": [], "source": [ "from skneuromsi.sweep import ParameterSweep\n", "\n", "soas = np.array(\n", " [36, 48, 60, 72, 84, 96, 108, 120, 132, 144, 156, 168, 180, 192, 204]\n", ")\n", "zhu_auditory_times = calculate_auditory_times(\n", " soas=soas, auditory_stim_duration=7, visual_time=16\n", ")\n", "\n", "cuppini_sp = ParameterSweep(\n", " model=model_cuppini,\n", " target=\"soa\",\n", " repeat=1,\n", " n_jobs=-2,\n", " range=soas,\n", " processing_strategy=NeuralIllusionRateProcessingStrategy(),\n", ")\n", "\n", "zhu_sp = ParameterSweep(\n", " model=model_zhu,\n", " target=\"auditory_time\",\n", " range=zhu_auditory_times,\n", " repeat=1,\n", " n_jobs=-2,\n", " processing_strategy=BayesianIllusionRateProcessingStrategy(),\n", ")\n", "\n", "paredes_sp = ParameterSweep(\n", " model=model_paredes,\n", " target=\"auditory_soa\",\n", " repeat=1,\n", " n_jobs=-2,\n", " range=soas,\n", " processing_strategy=NeuralIllusionRateProcessingStrategy(),\n", ")" ] }, { "cell_type": "markdown", "id": "5170a813", "metadata": {}, "source": [ "## Visual illusion responses" ] }, { "cell_type": "markdown", "id": "51b2c9c1", "metadata": {}, "source": [ "We are now ready to run the experiment and extract visual illusion responses for each model:" ] }, { "cell_type": "code", "execution_count": 85, "id": "79a7071e", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0d799b7eaedb487d92a0c5e829aa0f62", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sweeping 'soa': 0%| | 0/15 [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "# initializes figure and plots\n", "fig, axs = plt.subplots(1, 1, figsize=(5, 5), dpi=150)\n", "\n", "colors = plt.rcParams[\"axes.prop_cycle\"].by_key()[\"color\"]\n", "\n", "# Panel A\n", "ax1 = plt.subplot(111)\n", "ax1.plot(soas, cuppini_res)\n", "ax1.plot(soas, zhu_res)\n", "ax1.plot(soas, paredes_res)\n", "ax1.set_xlabel(\"SOA (ms)\", size=12, weight=\"bold\")\n", "ax1.set_ylabel(\"Visual Illusion (prop)\", size=12, weight=\"bold\")\n", "ax1.tick_params(axis=\"both\", labelsize=10)\n", "\n", "legend = ax1.legend(\n", " [\n", " \"Cuppini 2014\",\n", " \"Zhu 2024\",\n", " \"Paredes 2025\",\n", " ],\n", " fontsize=12,\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "neuromsi", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.18" } }, "nbformat": 4, "nbformat_minor": 5 }