Data Analysis Competitions

Please see the details below from the sites hosting our three data-analysis competitions:

  1. Dementia Screening challenge
  2. Epilepsy challenge
  3. Ketamine in depression

There will be prizes for all competitions and a dedicated workshop at biomag 2020 to discuss the entries and the analysis methods.

Dementia screening challenge

Organisers: Hokuto Hospital, Kumagaya General Hospital, and RICOH Company, Ltd., JAPAN.

Data providers: Hokuto Hospital, Kumagaya General Hospital, and Mihara Memorial Hospital, JAPAN.

Contact: Yoshihito Shigihara MD PhD, meaw.system@gmail.com

Competition Description: Dementia is a chronic and progressive syndrome caused by brain diseases with a few effective pharmacological treatments. It is important to diagnose it at the early stage of the syndrome, to provide effective interventions and slow the progression of the disease. Number of published studies have showed that magnetoencephalography (MEG) is sensitive to the subtle changes of brain activity related to dementia. MEG is a patient-friendly clinical examination tool, because it is totally non-invasive and the preparation/scanning time is short.

In this competition, your goal is to classify each of the MEG data sets (“test data”, N = 42) into one of three classes: healthy volunteer, mild cognitive impartment (MCI), and dementia. For coding your classification algorithms, another set of MEG data (“training data”) recorded from healthy volunteers (N = 100), patients with MCI (N = 15) and dementia (N = 29) are provided. All MEG data were recorded with the same paradigm (5-min resting-state recording with eyes-closed), but at two different sites (A or B, both sites have identical MEG systems). The sampling rate differed between sites: 1000Hz in site A, and 2000Hz in site B. All MEG data were recorded using 160-channel gradiometer system (Yokogawa at the site A and RICOH at the site B) and provided in SPM-12 format.

Beside the MEG data, age and gender information is available for all samples. If available, MMSE (Mini-Mental State Exam) score of the data-provider (healthy volunteer or patient) are also provided for the “training-data”. All participants and patients gave written informed consent to re-use their data for advances in medicine. Submitted answers will be evaluated from the viewpoint of clinical medicine, and submitter(s) who provided the most reasonable classifications will be selected as the winner(s) of the competition.

How to Join: Please send an email request to meaw.system@gmail.com, which is entitled ”BIOMAG DATA COMPETITION”. We will then send you back an URL for downloading the dataset and the password. After the completion of the task, please submit an answer sheet (Microsoft Excel format) and a set of materials (materials that are necessary for replicating your answers, e.g., analysis codes, algorithms) by email, which should be entitled ”SUBMISSION: BIOMAG DATA COMPETITION”. In the answer sheet, you are requested to provide “Estimated Classes” for each sample of the “test data”. Optionally, you may also provide a confidence score (e.g., 100% very confident) for each of the diagnoses.

Prize and Duty of the Winner(s): The winner(s) will share the monetary prize (5,000 pounds in total) (small prize-giving ceremony is planned during the Biomag2020 conference). The distribution of the prizes will be determined according to the number of the winner(s). The winner(s) obtains the copyright of the winning algorithms/codes, but the materials must be made freely available to the community (the organisers can help with this).

Deadline: 15th July 2020

Limitation: You are not allowed to use/distribute the data for any purposes irrelevant to the competition, without organiser’s permission.

Epilepsy Challenge

Organisers: Jean-Michel Badier, Christian Bénar, Institut de Neurosciences des Systèmes, Marseille, France. 

Data providers: In order to obtain the data, please register at https://meg.univ-amu.fr/biomag2020/download.html

Contact: jean-michel.badier@univ-amu.fr, christian.benar@univ-amu.fr

Participants are encouraged to also subscribe to this forum where questions related to the datasets will be discussed:
https://groups.google.com/forum/#!forum/biomag2020-epilepsy-competition

Objectives: Two datasets of resting state MEG are provided for analysis. The objective of the challenge is to identify, localize and provide network dynamics of the interictal activity for these two patients. These datasets are obtained during simultaneous MEG and intracerebral EEG. Participants need to provide the localization of the sources of abnormal activity as well as (if any) the network dynamics these events. Results will be compared to the depth recording.

MEG data: Data have been recorded on two epileptic patients in the Marseille MEG laboratory at Institut de Neuroscience des Systèmes (INS) and Timone Hospital Marseille (Assistance Publique – Hôpitaux de Marseille). The recording system is a 4D Neuroimaging 3600WH system. Anonymized raw data sets are supplied. Bad channels are listed in a separate file (bad_channels.txt). For technical reasons these channels may have locations that do not correspond to the real location of the coils. They must be discarded.

MRI data: The MRI of the patients are also supplied. In order to preserve the identification of the patients they have been defaced. Coordinates of the fiducial markers are given on the MRI volume in order to match the markers used during the MEG recording.

Results to supply: Localisation of the sources of epileptic discharges:

  • Summary of the localizations with a sublobar resolution.
  • Localization of selected discharges. Corresponding time should be provided and given in seconds.
  • Identification of the discharges (detection) may also be provided but is optional. Time of detection in seconds.

Dynamic of the sources of epileptic discharges:

  • If applicable, which is the leading region within the interictal network
  • Again, this will be presented as a summary and for a selected set of events.

Results format is free; however, they should be easy to read and interpret. For instance, graphic figures will be preferred to tables.

Evaluation criteria:

  • Accuracy of the reconstructed sources at the sublobar level
  • Identification of the leading node, if any, in the interictal network.

Deadline: June 8th, 2020 noon (GMT)

Limitations: You are not allowed to use/distribute the data for any purposes irrelevant to the competition, without organiser’s permission.

Ketamine in Depression

Organisers: Section on the Neurobiology and Treatment of Mood Disorders, National Institute of Mental Health, National Institutes of Health, Bethesda MD USA

Contact: Jessica Gilbert, PhD, jessica.gilbert@nih.gov

Data: Resting state MEG data from a clinical trial of ketamine’s mechanisms of action in major depression

The data include 250 second eyes-closed resting-state data from a randomized, double-blind, placebo-controlled study of ketamine’s mechanisms of action. Data for the competition come from a total of 36 male and female participants aged 18–65 years old, 22 of which had major depression (MDD) and 14 of which were healthy control subjects. Subjects with MDD had been diagnosed with recurrent MDD without psychotic features using the Structured Clinical Interview for Axis I DSM-IV Disorders (SCID)-Patient Version. Subjects with MDD were required to have a score greater than 20 on the Montgomery-Åsberg Depression Rating Scale (MADRS) at screening. MDD participants were considered treatment-resistant and had to have not responded to at least one adequate antidepressant trial during their current episode, as assessed using the Antidepressant Treatment History Form and the current episode had to have lasted at least four weeks. Subjects were free from psychotropic medications in the two weeks before randomization (five weeks for fluoxetine, three weeks for aripiprazole). Healthy control subjects had no Axis I disorder as determined by SCID-NP, and no family history of Axis I disorders in first degree relatives. Healthy control subjects were also free of medications affecting neuronal function or cerebral blood flow or metabolism. Subjects in both groups were in good physical health as determined by medical history, physical exam, blood labs, electrocardiogram, chest x-ray, urinalysis, and toxicology. 

The competition data include two resting-state scans per participant, one occurring 6-to-9 hours following ketamine administration and one occurring 6-to-9 hours following placebo saline administration. These data were collected using a CTF Omega 275-channel system. The data have been de-identified, so each participant has been given a randomized 8-letter code at the beginning of the filename. You will also find the date of the scan in the filename (i.e., YYYYMMDD). Each scan occurred approximately 14-days apart, and participants were randomized to receive either ketamine or placebo-saline during their first infusion. You can approach the data analysis competition in one of two ways. First, you can attempt to classify participants with MDD from healthy control subjects (i.e., a between-subjects factor). Second, you can attempt to classify the scan session (i.e., a within-subjects factor: ketamine versus placebo). In either case, you should submit a short, written report detailing how you approached the data analysis (i.e., describing whether you classified based on the between-subjects or within-subjects factor), the methods used, and defining which scan(s) correspond to your chosen grouping of interest. You will receive one point for each scan that is correctly classified at either the within or between-subjects level. 

Deadline: 15th July 2020

Limitations: You are not allowed to use/distribute the data for any purposes irrelevant to the competition, without the organizer’s permission.

The data are available to download via Globus (https://www.globus.org/), which is a free service. The steps to access the data are noted below:

  1. Set up a Globus account, which is free. Many universities offer access to Globus, so check to see whether you already have a free account through your institution. To check whether you have access to Globus via your home institution or need to set up a free account, click on ‘Log In’ in the upper right-hand corner of the Globus home page.
  2. Once you have an account, you will need to download the Globus Connect Client (GCC) to the computer where you would like to download the data (https://www.globus.org/globus-connect-personal). Using the GCC, you will set up an Endpoint on your local computer, which is the location where you want to download the data.
  3. Email jessica.gilbert@nih.gov with the email address you used to set up your Globus account/Endpoint. Jessica will add you to the list of shared users of the “BIOMAG data share” Endpoint, where the data is stored.
  4. Once you’ve been added to the list, you can access/download the data at the following URL: https://app.globus.org/file-manager?origin_id=dcd7f0de-599a-11ea-9682-0e56c063f437&origin_path=%2F
  5. This link will open the Globus File Manager. You should see the BIOMAG data share collection/path in the left-hand side of the File Manager window. Click “select all” in the left-hand window to select all the data files from the BIOMAG share. In the right-hand collection window, click “Search” and locate your Endpoint on your local computer from step 2 above.  Then, click “Tranfer and Sync to…”, making sure you’ve selected the directory on your computer where you want the data to download. Click the left-hand side “Start” button in order to transfer the data from the BIOMAG data share to your computer (see below example).