{ "cells": [ { "cell_type": "markdown", "id": "b6acc10d", "metadata": {}, "source": [ "# Model implementation and execution with Scikit-NeuroMSI" ] }, { "cell_type": "markdown", "id": "3fe10105", "metadata": {}, "source": [ "This tutorial covers the basic pipeline for implementing and running multisensory integration models using `Scikit-NeuroMSI`. We show how to implement the different models classes currently supported by the package, organized in the `mle`, `bayesian` and `neural` modules. It covers the execution objects and methods that are shared by all the models of the package: `run` and `NDResult`. \n", "\n", "> **Note**: In this tutorial we assume that you already have a basic knowledge of `numpy` for scientific computing." ] }, { "cell_type": "markdown", "id": "4db3c3d5", "metadata": {}, "source": [ "## Implementation of Maximum Likelihood Estimation (MLE) models" ] }, { "cell_type": "markdown", "id": "4859b389", "metadata": {}, "source": [ "To easily implement the model developed by Alais and Burr (2004) you can import the corresponding module and instatiate the `AlaisBurr2004` class: " ] }, { "cell_type": "code", "execution_count": 2, "id": "a0da0aa5", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from skneuromsi.mle import AlaisBurr2004\n", "\n", "model = AlaisBurr2004()\n", "model" ] }, { "cell_type": "markdown", "id": "29a43eb1", "metadata": {}, "source": [ "### Exploration of the model object" ] }, { "cell_type": "markdown", "id": "22a3b1d4", "metadata": {}, "source": [ "By calling the model using the `vars` function you can explore its main built-in parameters and methods:" ] }, { "cell_type": "code", "execution_count": 3, "id": "5617849d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_mode0': 'auditory',\n", " '_mode1': 'visual',\n", " '_position_range': (-20, 20),\n", " '_position_res': 0.01,\n", " '_time_range': (1, 1),\n", " '_time_res': 1.0,\n", " '_random': Generator(PCG64) at 0x708DC82435A0,\n", " 'run': }" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vars(model)" ] }, { "cell_type": "markdown", "id": "9f313786", "metadata": {}, "source": [ "- `mode0`: Name of the first sensory modality defined in the model.\n", "- `mode1`: Name of the second sensory modality defined in the model.\n", "- `position_range`: The range of possible positions where the stimulus could be delivered (in degrees).\n", "- `position_res`: The resolution of the range of possible positions where the stimulus could be delivered (in degrees).\n", "- `time_range`: The range of possible times when the stimulus could be delivered. Here is set to 1 because the model has no temporal dimension.\n", "- `time_res`: The resolution of the range of possible times when the stimulus could be delivered.\n", "- `random`: Embedded random number generator (useful to include noise in the model responses).\n", "- `run`: Executes the model and saves the result. " ] }, { "cell_type": "markdown", "id": "1be5ac03", "metadata": {}, "source": [ "We can re-implement Alais and Burr (2004) model with different sensory modalities and position ranges by modifying the initial parameters of the model:" ] }, { "cell_type": "code", "execution_count": 8, "id": "d5b97cad", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_mode0': 'visual',\n", " '_mode1': 'proprioceptive',\n", " '_position_range': (-50, 50),\n", " '_position_res': 0.01,\n", " '_time_range': (1, 1),\n", " '_time_res': 1.0,\n", " '_random': Generator(PCG64) at 0x708DC7FA1540,\n", " 'run': }" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model = AlaisBurr2004(\n", " mode0=\"visual\", mode1=\"proprioceptive\", position_range=(-50, 50)\n", ")\n", "vars(model)" ] }, { "cell_type": "markdown", "id": "f96b2f60", "metadata": {}, "source": [ "The model object has a built-in `run` method:" ] }, { "cell_type": "code", "execution_count": 11, "id": "5efccf50", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.run" ] }, { "cell_type": "markdown", "id": "39433e2b", "metadata": {}, "source": [ "By calling this method we can observe its arguments:\n", "\n", "- `visual_position`: The position where the visual stimulus is delivered (in degrees).\n", "- `proprioceptive_position`: The position where the propioceptive stimulus is delivered (in degrees).\n", "- `visual_sigma`: Standard deviation of the auditory estimate.\n", "- `proprioceptive_sigma`: Standard deviation of the propioceptive estimate." ] }, { "cell_type": "markdown", "id": "596f7224", "metadata": {}, "source": [ "Now let's run the model for equidistant auditory and visual locations:" ] }, { "cell_type": "code", "execution_count": 13, "id": "f8c852d1", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = model.run(visual_position=-5, proprioceptive_position=5)\n", "res" ] }, { "cell_type": "markdown", "id": "bb1d3bc3", "metadata": {}, "source": [ "The model outputs one `NDResult` object containing the results of both unisensory estimators and the multisensory estimator. " ] }, { "cell_type": "markdown", "id": "c339ea78", "metadata": {}, "source": [ "Refer to the [API documentation](https://scikit-neuromsi.readthedocs.io/en/latest/api/mle/_alais_burr2004.html) for further information about the parameters to manipulate in this model." ] }, { "cell_type": "markdown", "id": "4215398b", "metadata": {}, "source": [ "### Exploration of the results object" ] }, { "cell_type": "markdown", "id": "24e10d6c", "metadata": {}, "source": [ "By calling the `NDResult` object using the `vars` function you can explore its main built-in parameters and methods:" ] }, { "cell_type": "code", "execution_count": 14, "id": "596349c4", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_mname': 'AlaisBurr2004',\n", " '_mtype': 'MLE',\n", " '_output_mode': 'multi',\n", " '_nmap': {'auditory': 'visual',\n", " 'visual': 'proprioceptive',\n", " 'auditory_weight': 'visual_weight',\n", " 'visual_weight': 'proprioceptive_weight'},\n", " '_time_range': array([1, 1]),\n", " '_position_range': array([-50, 50]),\n", " '_time_res': 1.0,\n", " '_position_res': 0.01,\n", " '_run_parameters': {'visual_position': -5,\n", " 'proprioceptive_position': 5,\n", " 'visual_sigma': 3.0,\n", " 'proprioceptive_sigma': 3.0,\n", " 'noise': None},\n", " '_extra': {'visual_weight': 0.5, 'proprioceptive_weight': 0.5},\n", " '_causes': None,\n", " '_nddata': Size: 240kB\n", " array([[[[1.84356985e-050],\n", " [1.93808093e-050],\n", " [2.03741451e-050],\n", " ...,\n", " [1.65115301e-074],\n", " [1.55331368e-074],\n", " [1.46125559e-074]]],\n", " \n", " \n", " [[[1.37463811e-074],\n", " [1.46125559e-074],\n", " [1.55331368e-074],\n", " ...,\n", " [2.14181549e-050],\n", " [2.03741451e-050],\n", " [1.93808093e-050]]],\n", " \n", " \n", " [[[4.33458701e-122],\n", " [4.84392981e-122],\n", " [5.41300350e-122],\n", " ...,\n", " [6.04879857e-122],\n", " [5.41300350e-122],\n", " [4.84392981e-122]]]])\n", " Coordinates:\n", " * modes (modes) \n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modesvisualproprioceptivemulti
timespositionspositions_coordinates
00x01.843570e-501.374638e-744.334587e-122
1x01.938081e-501.461256e-744.843930e-122
2x02.037415e-501.553314e-745.413003e-122
3x02.141815e-501.651153e-746.048799e-122
4x02.251541e-501.755135e-746.759122e-122
...............
9995x01.865646e-742.366862e-507.552692e-122
9996x01.755135e-742.251541e-506.759122e-122
9997x01.651153e-742.141815e-506.048799e-122
9998x01.553314e-742.037415e-505.413003e-122
9999x01.461256e-741.938081e-504.843930e-122
\n", "

10000 rows × 3 columns

\n", "" ], "text/plain": [ "modes visual proprioceptive \\\n", "times positions positions_coordinates \n", "0 0 x0 1.843570e-50 1.374638e-74 \n", " 1 x0 1.938081e-50 1.461256e-74 \n", " 2 x0 2.037415e-50 1.553314e-74 \n", " 3 x0 2.141815e-50 1.651153e-74 \n", " 4 x0 2.251541e-50 1.755135e-74 \n", "... ... ... \n", " 9995 x0 1.865646e-74 2.366862e-50 \n", " 9996 x0 1.755135e-74 2.251541e-50 \n", " 9997 x0 1.651153e-74 2.141815e-50 \n", " 9998 x0 1.553314e-74 2.037415e-50 \n", " 9999 x0 1.461256e-74 1.938081e-50 \n", "\n", "modes multi \n", "times positions positions_coordinates \n", "0 0 x0 4.334587e-122 \n", " 1 x0 4.843930e-122 \n", " 2 x0 5.413003e-122 \n", " 3 x0 6.048799e-122 \n", " 4 x0 6.759122e-122 \n", "... ... \n", " 9995 x0 7.552692e-122 \n", " 9996 x0 6.759122e-122 \n", " 9997 x0 6.048799e-122 \n", " 9998 x0 5.413003e-122 \n", " 9999 x0 4.843930e-122 \n", "\n", "[10000 rows x 3 columns]" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.get_modes()" ] }, { "cell_type": "markdown", "id": "794befd3", "metadata": {}, "source": [ "This method returns a pandas DataFrame with the output values for each modality at each time point and spatial coordinates. In the package we called such output values simply as `values`, given that the outputs are different across models. In this MLE model, the output values are probability density values.\n", "\n", ">**Note**: This model does not include the temporal dimension, hence here is represented as a fixed time point. This model does include spatial dimensions (generically named `positions` in the package). The package allows to have more than one spatial dimension (in the package called `position_coordinates`). In this model we have a single spatial dimension, defined as a `position_coordinate` named `x0` by default." ] }, { "cell_type": "markdown", "id": "1211b5a8", "metadata": {}, "source": [ "Now let's explore the `stats` method:" ] }, { "cell_type": "code", "execution_count": 47, "id": "f7a6644b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
describe
count3.000000e+04
mean1.000000e-02
std3.114718e-02
min4.334587e-122
25%3.091076e-43
50%4.385153e-20
75%3.429870e-06
max1.880632e-01
\n", "
" ], "text/plain": [ " describe\n", "count 3.000000e+04\n", "mean 1.000000e-02\n", "std 3.114718e-02\n", "min 4.334587e-122\n", "25% 3.091076e-43\n", "50% 4.385153e-20\n", "75% 3.429870e-06\n", "max 1.880632e-01" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.stats()" ] }, { "cell_type": "markdown", "id": "1000d994", "metadata": {}, "source": [ "By default it provides descriptive statistics of the model output. We can have more specific statistics by specifying the `kind` and `modes` argument, for example: " ] }, { "cell_type": "code", "execution_count": 49, "id": "d7ee3517", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.1329807601338109" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.stats(kind=\"max\", modes=\"visual\")" ] }, { "cell_type": "markdown", "id": "a170c131", "metadata": {}, "source": [ "A useful method is the `dimmax`, which provides the maximum value observed in a specific modality and its temporal and spatial coordinates:" ] }, { "cell_type": "code", "execution_count": 35, "id": "8906c01d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "times 0\n", "positions 4500\n", "positions_coordinates x0\n", "values 0.1329807601338109\n", "Name: max, dtype: object" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.stats.dimmax(modes=\"visual\")" ] }, { "cell_type": "markdown", "id": "7209b9a8", "metadata": {}, "source": [ "Refer to the [API documentation](https://scikit-neuromsi.readthedocs.io/en/latest/api/core/ndresult/index.html) for further information about the `NDResult` object." ] }, { "cell_type": "markdown", "id": "47d66f89", "metadata": {}, "source": [ "> So far we have covered the fundamentals of model classes and results objects in the Scikit-NeuroMSI package. These are applicable to all model classes. Now we are going to introduce the `Bayesian` and `neural` modules, and present some distinctive characteristics of the objects generated by this models." ] }, { "cell_type": "markdown", "id": "daa9244f", "metadata": {}, "source": [ "## Implementation of Bayesian models" ] }, { "cell_type": "markdown", "id": "6988de27", "metadata": {}, "source": [ "You can implement the Causal Inference model developed by Kording et al. (2007) by importing the corresponding module and instantiating the `Kording2007` class:" ] }, { "cell_type": "code", "execution_count": 65, "id": "4e4838e0", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from skneuromsi.bayesian import Kording2007\n", "\n", "model = Kording2007(position_range=(-20, 20), position_res=1, n=100000)\n", "model" ] }, { "cell_type": "code", "execution_count": 66, "id": "abd2dc6a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_n': 100000,\n", " '_mode0': 'auditory',\n", " '_mode1': 'visual',\n", " '_position_range': (-20, 20),\n", " '_position_res': 1.0,\n", " '_time_range': (0, 1),\n", " '_time_res': 1.0,\n", " '_random': Generator(PCG64) at 0x708DC5052C00,\n", " 'run': }" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vars(model)" ] }, { "cell_type": "markdown", "id": "b4bc01f4", "metadata": {}, "source": [ "Refer to the [API documentation](https://scikit-neuromsi.readthedocs.io/en/latest/api/bayesian/_kording2007.html) for more details about the available parameters for this model." ] }, { "cell_type": "markdown", "id": "b6af81ac", "metadata": {}, "source": [ "Let's run the model for two conflicting stimulus locations:" ] }, { "cell_type": "code", "execution_count": 67, "id": "1e2d0ebb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = model.run(auditory_position=-15, visual_position=15, causes_kind=\"count\")\n", "res" ] }, { "cell_type": "markdown", "id": "e16fa11c", "metadata": {}, "source": [ "In this model we have an output named `causes`, which provides information about the estimated number of causes (sources) of the stimuli presented to the model. This is a fundamental attribute of causal inference models of multisensory integration." ] }, { "cell_type": "markdown", "id": "29f2c23d", "metadata": {}, "source": [ "Let's now explore the model output:" ] }, { "cell_type": "code", "execution_count": 68, "id": "305a4972", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modesauditorymultivisual
timespositionspositions_coordinates
00x00.0067840.0014000.000000
1x00.0222070.0054520.000012
2x00.0562690.0180360.000061
3x00.1070570.0513570.000230
4x00.1650700.1025850.000460
5x00.1956750.1639960.001102
6x00.1850720.2004080.002312
7x00.1344550.1899350.004043
8x00.0768930.1394670.005690
9x00.0350840.0782070.008062
10x00.0114340.0348020.010011
11x00.0032870.0110540.012614
12x00.0006410.0028210.013558
13x00.0000700.0004400.015180
14x00.0000000.0000400.016972
15x00.0000000.0000000.018618
16x00.0000000.0000000.019018
17x00.0000000.0000000.020591
18x00.0000000.0000000.022480
19x00.0000000.0000000.024017
20x00.0000000.0000000.025083
21x00.0000000.0000000.026632
22x00.0000000.0000000.028375
23x00.0000000.0000000.030615
24x00.0000000.0000000.032612
25x00.0000000.0000000.034634
26x00.0000000.0000000.035905
27x00.0000000.0000000.039827
28x00.0000000.0000000.042272
29x00.0000000.0000000.044076
30x00.0000000.0000000.045953
31x00.0000000.0000000.046945
32x00.0000000.0000000.049003
33x00.0000000.0000000.048688
34x00.0000000.0000000.050613
35x00.0000000.0000000.048918
36x00.0000000.0000000.048156
37x00.0000000.0000000.046945
38x00.0000000.0000000.042006
39x00.0000000.0000000.037709
\n", "
" ], "text/plain": [ "modes auditory multi visual\n", "times positions positions_coordinates \n", "0 0 x0 0.006784 0.001400 0.000000\n", " 1 x0 0.022207 0.005452 0.000012\n", " 2 x0 0.056269 0.018036 0.000061\n", " 3 x0 0.107057 0.051357 0.000230\n", " 4 x0 0.165070 0.102585 0.000460\n", " 5 x0 0.195675 0.163996 0.001102\n", " 6 x0 0.185072 0.200408 0.002312\n", " 7 x0 0.134455 0.189935 0.004043\n", " 8 x0 0.076893 0.139467 0.005690\n", " 9 x0 0.035084 0.078207 0.008062\n", " 10 x0 0.011434 0.034802 0.010011\n", " 11 x0 0.003287 0.011054 0.012614\n", " 12 x0 0.000641 0.002821 0.013558\n", " 13 x0 0.000070 0.000440 0.015180\n", " 14 x0 0.000000 0.000040 0.016972\n", " 15 x0 0.000000 0.000000 0.018618\n", " 16 x0 0.000000 0.000000 0.019018\n", " 17 x0 0.000000 0.000000 0.020591\n", " 18 x0 0.000000 0.000000 0.022480\n", " 19 x0 0.000000 0.000000 0.024017\n", " 20 x0 0.000000 0.000000 0.025083\n", " 21 x0 0.000000 0.000000 0.026632\n", " 22 x0 0.000000 0.000000 0.028375\n", " 23 x0 0.000000 0.000000 0.030615\n", " 24 x0 0.000000 0.000000 0.032612\n", " 25 x0 0.000000 0.000000 0.034634\n", " 26 x0 0.000000 0.000000 0.035905\n", " 27 x0 0.000000 0.000000 0.039827\n", " 28 x0 0.000000 0.000000 0.042272\n", " 29 x0 0.000000 0.000000 0.044076\n", " 30 x0 0.000000 0.000000 0.045953\n", " 31 x0 0.000000 0.000000 0.046945\n", " 32 x0 0.000000 0.000000 0.049003\n", " 33 x0 0.000000 0.000000 0.048688\n", " 34 x0 0.000000 0.000000 0.050613\n", " 35 x0 0.000000 0.000000 0.048918\n", " 36 x0 0.000000 0.000000 0.048156\n", " 37 x0 0.000000 0.000000 0.046945\n", " 38 x0 0.000000 0.000000 0.042006\n", " 39 x0 0.000000 0.000000 0.037709" ] }, "execution_count": 68, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.get_modes()" ] }, { "cell_type": "markdown", "id": "fc478dcb", "metadata": {}, "source": [ "As in MLE models, the output values of the Bayesian models represent probability density values. Note that we also have a fixed time point." ] }, { "cell_type": "markdown", "id": "4154921e", "metadata": {}, "source": [ "Now let's see what happens if we reduce the distance of the stimuli:" ] }, { "cell_type": "code", "execution_count": 58, "id": "3c8a80f9", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = model.run(auditory_position=-3, visual_position=3, causes_kind=\"count\")\n", "res" ] }, { "cell_type": "markdown", "id": "5cea1cb5", "metadata": {}, "source": [ "The model now reports that the stimuli as originating from a common source. We can directly observe the probability of the stimuli originating from a common cause:" ] }, { "cell_type": "code", "execution_count": 44, "id": "96e344fb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "p(C=1): 0.561605322439197\n", "C: 1\n" ] } ], "source": [ "print(\"p(C=1):\", res.e_[\"mean_p_common_cause\"])\n", "print(\"C:\", res.causes_)" ] }, { "cell_type": "markdown", "id": "ea89f334", "metadata": {}, "source": [ "Refer to the [API documentation](https://scikit-neuromsi.readthedocs.io/en/latest/api/bayesian/index.html) for more details about the models available in the `bayesian` module." ] }, { "cell_type": "markdown", "id": "42d93cb8", "metadata": {}, "source": [ "> This demonstration of the Bayesian Causal Inference model mechanics is inspired in the tutorial of the [BCIT Toolbox](https://github.com/multisensoryperceptionlab/BCIT/blob/master/Documentation/BCIT_Documentation_5.1.2017.pdf)." ] }, { "cell_type": "markdown", "id": "c4b8a352", "metadata": {}, "source": [ "## Implementation of neural models" ] }, { "cell_type": "markdown", "id": "c138ce4b", "metadata": {}, "source": [ "You can implement the network model developed by Cuppini et al. (2017) on by importing the corresponding module and instantiating the `Cuppini2017` class:" ] }, { "cell_type": "code", "execution_count": 71, "id": "88e347ab", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 71, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from skneuromsi.neural import Cuppini2017\n", "\n", "model = Cuppini2017(neurons=90, position_range=(0, 90))\n", "model" ] }, { "cell_type": "code", "execution_count": 72, "id": "6a443370", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_neurons': 90,\n", " '_position_range': (0, 90),\n", " '_position_res': 1.0,\n", " '_time_range': (0, 100),\n", " '_time_res': 0.01,\n", " '_integrator': ,\n", " '_random': Generator(PCG64) at 0x708DC5052880,\n", " '_mode0': 'auditory',\n", " '_mode1': 'visual',\n", " 'run': }" ] }, "execution_count": 72, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vars(model)" ] }, { "cell_type": "markdown", "id": "967894cc", "metadata": {}, "source": [ "You can refer to the [API documentation](https://scikit-neuromsi.readthedocs.io/en/latest/api/neural/_cuppini2017.html) for more details about the available parameters." ] }, { "cell_type": "markdown", "id": "83e597dd", "metadata": {}, "source": [ "As before, let's run the model for two conflicting stimulus locations:" ] }, { "cell_type": "code", "execution_count": 73, "id": "09cc460f", "metadata": {}, "outputs": [], "source": [ "res = model.run(auditory_position=30, visual_position=60)" ] }, { "cell_type": "code", "execution_count": 74, "id": "4024e1a0", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'_mname': 'Cuppini2017',\n", " '_mtype': 'Neural',\n", " '_output_mode': 'multi',\n", " '_nmap': {'auditory': 'auditory', 'visual': 'visual'},\n", " '_time_range': array([ 0, 100]),\n", " '_position_range': array([ 0, 90]),\n", " '_time_res': 0.01,\n", " '_position_res': 1.0,\n", " '_run_parameters': {'auditory_position': 30,\n", " 'visual_position': 60,\n", " 'auditory_sigma': 32,\n", " 'visual_sigma': 4,\n", " 'auditory_intensity': 28,\n", " 'visual_intensity': 27,\n", " 'auditory_duration': None,\n", " 'auditory_onset': 0,\n", " 'auditory_stim_n': 1,\n", " 'visual_duration': None,\n", " 'visual_onset': 0,\n", " 'visual_stim_n': 1,\n", " 'auditory_soa': None,\n", " 'visual_soa': None,\n", " 'noise': False,\n", " 'noise_level': 0.4,\n", " 'feedforward_weight': 18,\n", " 'cross_modal_weight': 1.4,\n", " 'causes_kind': 'count',\n", " 'causes_dim': 'space',\n", " 'causes_peak_threshold': 0.15,\n", " 'causes_peak_distance': None},\n", " '_extra': {'causes_kind': 'count',\n", " 'causes_dim': 'space',\n", " 'causes_peak_threshold': 0.15,\n", " 'causes_peak_distance': None,\n", " 'stim_position': [30, 60]},\n", " '_causes': 2,\n", " '_nddata': Size: 22MB\n", " array([[[[1.19097530e-03],\n", " [1.31460645e-03],\n", " [1.44163460e-03],\n", " ...,\n", " [8.54958672e-04],\n", " [9.59982932e-04],\n", " [1.07232604e-03]],\n", " \n", " [[2.24271407e-03],\n", " [2.48409320e-03],\n", " [2.73379601e-03],\n", " ...,\n", " [1.59484536e-03],\n", " [1.79605986e-03],\n", " [2.01258518e-03]],\n", " \n", " [[3.17001717e-03],\n", " [3.52222241e-03],\n", " [3.88888969e-03],\n", " ...,\n", " ...\n", " ...,\n", " [2.31613388e-03],\n", " [2.33042543e-03],\n", " [2.33972890e-03]],\n", " \n", " [[2.34435053e-03],\n", " [2.34445919e-03],\n", " [2.34007850e-03],\n", " ...,\n", " [2.31613358e-03],\n", " [2.33042520e-03],\n", " [2.33972873e-03]],\n", " \n", " [[2.34435040e-03],\n", " [2.34445910e-03],\n", " [2.34007844e-03],\n", " ...,\n", " [2.31613328e-03],\n", " [2.33042497e-03],\n", " [2.33972856e-03]]]])\n", " Coordinates:\n", " * modes (modes) \n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
modesauditorymultivisual
timespositionspositions_coordinates
00x00.0011910.0000250.000002
1x00.0013150.0000250.000002
2x00.0014420.0000250.000002
3x00.0015700.0000250.000002
4x00.0016990.0000250.000002
..................
999985x00.0002670.0022710.000015
86x00.0003090.0022960.000015
87x00.0003560.0023160.000015
88x00.0004120.0023300.000015
89x00.0004760.0023400.000016
\n", "

900000 rows × 3 columns

\n", "" ], "text/plain": [ "modes auditory multi visual\n", "times positions positions_coordinates \n", "0 0 x0 0.001191 0.000025 0.000002\n", " 1 x0 0.001315 0.000025 0.000002\n", " 2 x0 0.001442 0.000025 0.000002\n", " 3 x0 0.001570 0.000025 0.000002\n", " 4 x0 0.001699 0.000025 0.000002\n", "... ... ... ...\n", "9999 85 x0 0.000267 0.002271 0.000015\n", " 86 x0 0.000309 0.002296 0.000015\n", " 87 x0 0.000356 0.002316 0.000015\n", " 88 x0 0.000412 0.002330 0.000015\n", " 89 x0 0.000476 0.002340 0.000016\n", "\n", "[900000 rows x 3 columns]" ] }, "execution_count": 75, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.get_modes()" ] }, { "cell_type": "markdown", "id": "fa3df3ae", "metadata": {}, "source": [ "The output values of this models represent neural activity values. Note that the model output now haves multiple time points, so we have the neural activity values at each time point of the simluation for each spatial coordinate encoded in the network." ] }, { "cell_type": "markdown", "id": "22df9f97", "metadata": {}, "source": [ "We observe that the model detects two distinct causes from the stimuli by looking at the causes output of the model:" ] }, { "cell_type": "code", "execution_count": 76, "id": "0649906f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C: 2\n" ] } ], "source": [ "print(\"C:\", res.causes_)" ] }, { "cell_type": "markdown", "id": "d12fd480", "metadata": {}, "source": [ "Now let's see what happens if we reduce the distance of the stimuli:" ] }, { "cell_type": "code", "execution_count": 78, "id": "76651266", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = model.run(auditory_position=40, visual_position=50)\n", "res" ] }, { "cell_type": "code", "execution_count": 79, "id": "b23b0e63", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C: 1\n" ] } ], "source": [ "print(\"C:\", res.causes_)" ] }, { "cell_type": "markdown", "id": "eaed42b1", "metadata": {}, "source": [ "The model outputs a common cause for stimuli close in space." ] }, { "cell_type": "markdown", "id": "ad1dac31", "metadata": {}, "source": [ "Refer to the [API documentation](https://scikit-neuromsi.readthedocs.io/en/latest/api/neural/index.html) for more details about the models available in the `neural` module." ] } ], "metadata": { "kernelspec": { "display_name": "neuromsi", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.18" } }, "nbformat": 4, "nbformat_minor": 5 }