Difference between revisions of "RG: Machine Learning"

From gem
Jump to navigation Jump to search
 
(17 intermediate revisions by the same user not shown)
Line 15: Line 15:
 
| 4. Bashi Ferdousi    || AFRL || [mailto:banafsheh.ferdousi@spaceforce.edu ]
 
| 4. Bashi Ferdousi    || AFRL || [mailto:banafsheh.ferdousi@spaceforce.edu ]
 
|-
 
|-
| 5. Valluri Sai Gowtam    || UAF || [mailto:svalluri@alaska.edu ]
+
| 5. Valluri Sai Gowtam    || NASA GSFC/CUA || [mailto:saigowtam.valluri@nasa.gov ]
 
|}
 
|}
 +
 +
==mini-GEM 2024==
 +
 +
At the 2024 GEM summer workshop, ML-GEM participants selected three challenge storms to facilitate community-wide, AI-driven research activities over the coming years:
 +
 +
'''1. January 4, 2023 (minimum Sym-H :-74nT at 09:04UT)'''
 +
 +
'''2. May 6, 2023 (minimum Sym-H : -108 nT at 05:11 UT)'''
 +
 +
'''3. May 11, 2024 (minimum Sym-H : -518nT at 02:14 UT)'''
 +
 +
The goals of this initiative are to
 +
1) Understand the current status of AI-driven heliophysics models,
 +
2) Share new AI techniques and lessons learned,
 +
3) Compile a catalog of ML-ready dataset, and
 +
3) Unify community-wide AI efforts under a single modeling framework, AIMFAHR.
 +
 +
The ML-GEM resource group invites AI-driven studies on the challenge storms to a mini-GEM session, which will be scheduled in the afternoon of Sunday, December 8, prior to AGU, as well as to the regular ML-GEM session during the 2025 GEM summer workshop. 
 +
 +
We look forward to your participation.
  
 
==About the ML-GEM Resource Group==
 
==About the ML-GEM Resource Group==
Line 43: Line 63:
 
'''1. ML-GEM stand-alone session (10:30am - 12:00pm on 06/27 Thursday):'''  All ML efforts across the GEM research areas are invited.
 
'''1. ML-GEM stand-alone session (10:30am - 12:00pm on 06/27 Thursday):'''  All ML efforts across the GEM research areas are invited.
  
'''Zoom Link:''' https://unh.zoom.us/j/92641406400?pwd=ghwL2kuHUr4ZUCcZZVd8rVFk1Q5XGn.1, '''Password:''' 452277.
+
Zoom Link: https://unh.zoom.us/j/92641406400?pwd=ghwL2kuHUr4ZUCcZZVd8rVFk1Q5XGn.1, Password: 452277.
  
 
'''8min/presentation.''' We recommend 6min presentation, 1min question, and 1min transition.
 
'''8min/presentation.''' We recommend 6min presentation, 1min question, and 1min transition.
Line 72: Line 92:
 
|-
 
|-
 
| 11 || Xiangning Chu || LASP || Imbalanced regressive neural network model for whistler-mode hiss waves: spatial and temporal evolution || 11:50 || Inner Mag
 
| 11 || Xiangning Chu || LASP || Imbalanced regressive neural network model for whistler-mode hiss waves: spatial and temporal evolution || 11:50 || Inner Mag
 +
|-
 +
| 12 || Drew Turner || APL/JHU || Unsupervised clustering and a generative model for Earth's bow shock using MMS data || 11:58 || Dayside
 
|}
 
|}
  
 
'''2. ML-GEM joint session with the Inner Magnetosphere Focus Groups (3:30 - 05:00 PM on 06/27 Thursday):''' ML efforts particularly in the inner magnetosphere research area are invited. We will discuss how to align ML efforts with current science associated with the SCIMM/RB/CP Focus Groups. An outcome will be to identify an open question, pair ML and data analysts, and create a plan to pursue answers for the next GEM Workshop.
 
'''2. ML-GEM joint session with the Inner Magnetosphere Focus Groups (3:30 - 05:00 PM on 06/27 Thursday):''' ML efforts particularly in the inner magnetosphere research area are invited. We will discuss how to align ML efforts with current science associated with the SCIMM/RB/CP Focus Groups. An outcome will be to identify an open question, pair ML and data analysts, and create a plan to pursue answers for the next GEM Workshop.
  
'''Zoom Link:''' https://unh.zoom.us/j/93264767597?pwd=tjaJrNendpMdMGP8aaYbRKrIGnCVbO.1. '''Password:''' 435443
+
Zoom Link: https://unh.zoom.us/j/93264767597?pwd=tjaJrNendpMdMGP8aaYbRKrIGnCVbO.1. Password: 435443
  
 
'''The discussion-only session with inner MAG focus groups.'''
 
'''The discussion-only session with inner MAG focus groups.'''
Line 87: Line 109:
 
| 2 || Everyone || || Discussion ||  ||  
 
| 2 || Everyone || || Discussion ||  ||  
 
|-
 
|-
|  ||  ||  || - Identify an open question, pair ML and data analysts, and create a plan to pursue answers for the next GEM Workshop. ||  ||  
+
|  ||  ||  || - Identify an open question, pair ML and data analysis, and create a plan to pursue answers for the next GEM Workshop. ||  ||  
 
|}
 
|}
  
  
'''3. ML-GEM discussion session (10:30am - 12:00pm on 06/28 Friday):''' We will showcase currently available AI models in the heliophysics community and discuss how to unify them into a single AI modeling framework for understanding and predicting the geospace environment. Please submit a 2-page summary of your AI model at https://docs.google.com/presentation/d/1qzgn880hCaDHLkEhL6GUEcEI3m5SJjQK/edit?usp=drive_link&ouid=115923735998882159942&rtpof=true&sd=true
+
'''3. ML-GEM discussion session (10:30am - 12:00pm on 06/28 Friday):''' We will showcase currently available AI models in the heliophysics community and discuss how to unify them into a single AI modeling framework for understanding and predicting the geospace environment.
 +
 
 +
Zoom Link: https://unh.zoom.us/j/99110890924?pwd=LsvPTw5LtrTwQVeprH8uQZCbLp4DEF.1. Password: 034895
 +
 
 +
The session will host three talks from the standalone session (10:30 to 10:54am) and then move to the discussion sessions (10:54am -12:00pm).
  
 
{| class="wikitable"
 
{| class="wikitable"
 
|+ style="caption-side:top; text-align:middle | ML-GEM Standalone/Discussion Session: 10:30 - 12:00pm on Friday, Jun 28
 
|+ style="caption-side:top; text-align:middle | ML-GEM Standalone/Discussion Session: 10:30 - 12:00pm on Friday, Jun 28
 
!  !! Name !! Affiliation !! Title !! Time !! Topic  
 
!  !! Name !! Affiliation !! Title !! Time !! Topic  
|-
 
Standalone Session (Overflow) - 8min/talk
 
 
|-
 
|-
 
| 1 || Jacob Bortnik || UCLA || Using interpretable AI to discover the drivers of acceleration vs depletion radiation belt events || 10:30 || Inner Mag
 
| 1 || Jacob Bortnik || UCLA || Using interpretable AI to discover the drivers of acceleration vs depletion radiation belt events || 10:30 || Inner Mag
Line 104: Line 128:
 
|-
 
|-
 
| 3 || Man Hua || UCLA || Machine‐learning based identification of the critical driving factors controlling storm‐time outer radiation belt electron flux dropouts || 10:46 || Inner Mag
 
| 3 || Man Hua || UCLA || Machine‐learning based identification of the critical driving factors controlling storm‐time outer radiation belt electron flux dropouts || 10:46 || Inner Mag
|-
 
Discussion Session
 
 
|-
 
|-
 
| 4 || Hyunju Connor || NASA/GSFC || Overview of ML-GEL goals, plan, and discussion topics || 10:54 ||
 
| 4 || Hyunju Connor || NASA/GSFC || Overview of ML-GEL goals, plan, and discussion topics || 10:54 ||
 
|-
 
|-
| 5 || Everyone || || Presentation of current ML models (2min/model) : Submit slide format || 11:00 ||
+
| 5 || '''Everyone''' || || '''Presentation of current ML models (2min/model) : Submit slide format''' [https://docs.google.com/presentation/d/1qzgn880hCaDHLkEhL6GUEcEI3m5SJjQK/edit#slide=id.g2e728881d58_0_266] || 11:00 ||
 
|-
 
|-
|  ||  ||  || Discussion ||  ||
+
|  ||  ||  || '''Discussion''' ||  ||
 
|-
 
|-
 
|  ||  || || 1. Integrate the ML models into a unified AI framework of Heliophysics Modeling ||
 
|  ||  || || 1. Integrate the ML models into a unified AI framework of Heliophysics Modeling ||
Line 117: Line 139:
 
| || || || 2. Select a common testing dataset (year, storms, substorms) for the framework validation ||
 
| || || || 2. Select a common testing dataset (year, storms, substorms) for the framework validation ||
 
|-
 
|-
| || || || 3. Determine actim items for next ML-GEM meeting ||
+
| || || || 3. Determine action items for the next ML-GEM meeting ||
 
|}
 
|}
  
 
'''4. ML-GEM tutorial session (1:30 - 3:00pm on 06/28 Friday):'''  A hands-on tutorial on the Long Short-Term Memory (LSTM) technique that models time-series data. This tutorial will use the LSTM model and the SuperMAG geomagnetic field data, published in Blandin et al. (2022; https://doi.org/10.3389/fspas.2022.846291).
 
'''4. ML-GEM tutorial session (1:30 - 3:00pm on 06/28 Friday):'''  A hands-on tutorial on the Long Short-Term Memory (LSTM) technique that models time-series data. This tutorial will use the LSTM model and the SuperMAG geomagnetic field data, published in Blandin et al. (2022; https://doi.org/10.3389/fspas.2022.846291).
  
 +
Zoom Link: https://unh.zoom.us/j/97969090948?pwd=KRF3SEW7E5GELUn8DXozlvTYbKsvMa.1. Password: 074062.
 +
 +
{| class="wikitable"
 +
|+ style="caption-side:top; text-align:middle | ML-GEM Tutorial Session: 1:30 - 3:00pm on Friday, Jun 28
 +
!  !! Name !! Affiliation !! Title !! Time !! Topic
 +
|-
 +
| 1 || Hyunju Connor || NASA/GSFC || Long Short-TERM Memory Model of Alaska geomagnetic field variation || 1:30 ||
 +
|-
 +
| 2 || Matt Blandin || UAF || Introduction of the LSTM code || 1:40 ||
 +
|-
 +
| 3 || Everyone || - || Hands-on Activities: Follow the '''instructions below''' to get started. || 1:50 ||
 +
|}
  
 
'''Instructions for the ML-GEM Tutorial Session:'''
 
'''Instructions for the ML-GEM Tutorial Session:'''
Line 127: Line 161:
 
Welcome to the ML-GEM tutorial session! We will be using the Python programming language and the Google Colab platform for our exercises. There’s no need to download or install anything. Just follow these simple steps:
 
Welcome to the ML-GEM tutorial session! We will be using the Python programming language and the Google Colab platform for our exercises. There’s no need to download or install anything. Just follow these simple steps:
  
'''Step 1:''' Download the tutorial Jupyter notebook and the data files via the provided link. [Link to be determined soon]
+
'''Step 1:''' Download the tutorial Jupyter notebook and the data files via the provided link:https://drive.google.com/drive/u/1/folders/1CflXlSi5KZ4Iy0f4MkWRrR_j8t6mmLO1
  
 
'''Step 2:''' Sign in to your Google Drive account.
 
'''Step 2:''' Sign in to your Google Drive account.

Latest revision as of 06:23, 23 October 2024

Chairs

Table 1: RG Chairs.
Name Affiliation Email
1. Hyunju Connor NASA GSFC [1]
2. Matthew Argall UNH [2]
3. Xiangning Chu LASP, CU Boulder [3]
4. Bashi Ferdousi AFRL [4]
5. Valluri Sai Gowtam NASA GSFC/CUA [5]

mini-GEM 2024

At the 2024 GEM summer workshop, ML-GEM participants selected three challenge storms to facilitate community-wide, AI-driven research activities over the coming years:

1. January 4, 2023 (minimum Sym-H :-74nT at 09:04UT)

2. May 6, 2023 (minimum Sym-H : -108 nT at 05:11 UT)

3. May 11, 2024 (minimum Sym-H : -518nT at 02:14 UT)

The goals of this initiative are to 1) Understand the current status of AI-driven heliophysics models, 2) Share new AI techniques and lessons learned, 3) Compile a catalog of ML-ready dataset, and 3) Unify community-wide AI efforts under a single modeling framework, AIMFAHR.

The ML-GEM resource group invites AI-driven studies on the challenge storms to a mini-GEM session, which will be scheduled in the afternoon of Sunday, December 8, prior to AGU, as well as to the regular ML-GEM session during the 2025 GEM summer workshop.

We look forward to your participation.

About the ML-GEM Resource Group

Machine Learning based Geospace Environment Modeling (ML-GEM) is a new resource group selected by the GEM Steering Committee, with two primary goals: advancing system-of-systems science in Sun-Earth interaction from a data-driven perspective and developing an ML-based Geospace Environment Modeling by integrating community-wide ML efforts.

Goals and Objectives

1. Invite community-wide ML efforts: Extend invitations to a diverse community, including GEM, CEDAR, SHINE, and ML communities, fostering broader participation and collaboration.

2. Share ML advancements: Disseminate the latest ML techniques, including their advantages and disadvantages, and lessons learned. This knowledge-sharing will support community-wide geospace environment modeling efforts.

3. Facilitate integration discussions: Stimulate discussions on integrating diverse ML models across geospace systems, such as solar wind, magnetosheath, cusp, ring current, radiation belt, plasmasphere, and ionosphere.

4. Develop ML-based models: Initiate the development of an ML-based geospace environment model and encourage the strategic creation and inclusion of ML components in ML-GEM for enhanced performance.

5. Explore systematic responses: Investigate system-of-systems science in the solar wind – Earth interaction by leveraging ML-GEM and other ML models.

6. Compile a catalog of ML-ready dataset: Cleaning and preparing datasets constitute a substantial portion of machine learning model development. We will curate a list of existing ML-ready datasets in heliophysics for various scientific purposes.

GEM-2024 Activities

ML-GEM chairs have scheduled 4 sessions for the upcoming 2024 GEM Summer workshop held in Fort Collins, Colorado during June 23-28.


1. ML-GEM stand-alone session (10:30am - 12:00pm on 06/27 Thursday): All ML efforts across the GEM research areas are invited.

Zoom Link: https://unh.zoom.us/j/92641406400?pwd=ghwL2kuHUr4ZUCcZZVd8rVFk1Q5XGn.1, Password: 452277.

8min/presentation. We recommend 6min presentation, 1min question, and 1min transition.

ML-GEM Standalone Session Schedule: 10:30am - 12:00pm on Thursday, Jun 27
Name Affiliation Title Time Topic
1 Mikhail Sitnov APL/JHU Data mining of the cislunar magnetotail 10:30 Tail/nightside
2 Savvas Raptis APL/JHU Modeling Earth's Plasma Sheet with Machine learning 10:38 Tail/nightside
3 Xiaofei Shi UCLA Data mining-based reconstruction of the transition region in the nightside magnetosphere validated through ELFIN isotropic boundaries 10:46 Tail/nightside
4 James Edmond/Jimmy Raeder UNH Clustering of Plasma Region Observations using Self-Organizing Maps 10:54 Dayside
5 Connor O'Brien BU PRIME-SH: A Data-Driven Probabilistic Model of Earth’s Magnetosheath 11:02 Dayside
6 Sai Gowtam Valluri UAF Machine Learning-based Particle Precipitation Model (ML-PPM) 11:10 Ionosphere
7 Matthew Blandin UAF Residual Convolutional Neural Networks for Global Geomagnetic Field Predictions 11:18 GIC
8 Jonathan Mellina LASP Tidal Effects in Earth's Plasmasphere Using a Machine Learning Model 11:26 Plasmasphere
9 Wen Li BU Integrate Deep Learning into Physics-Based Simulation: Modeling Global Electron Precipitation Driven by Whistler Mode Waves 11:34 Inner Mag
10 Evan McPherson LASP Imbalanced Regressive Model of Electron Fluxes in the Earth's Outer Radiation Belt 11:42 Inner Mag
11 Xiangning Chu LASP Imbalanced regressive neural network model for whistler-mode hiss waves: spatial and temporal evolution 11:50 Inner Mag
12 Drew Turner APL/JHU Unsupervised clustering and a generative model for Earth's bow shock using MMS data 11:58 Dayside

2. ML-GEM joint session with the Inner Magnetosphere Focus Groups (3:30 - 05:00 PM on 06/27 Thursday): ML efforts particularly in the inner magnetosphere research area are invited. We will discuss how to align ML efforts with current science associated with the SCIMM/RB/CP Focus Groups. An outcome will be to identify an open question, pair ML and data analysts, and create a plan to pursue answers for the next GEM Workshop.

Zoom Link: https://unh.zoom.us/j/93264767597?pwd=tjaJrNendpMdMGP8aaYbRKrIGnCVbO.1. Password: 435443

The discussion-only session with inner MAG focus groups.

MLGEM-RB-SCIMM-CP Joint Session: 3:30 - 5:00pm on Thursday, Jun 27
Name Affiliation Title Time Topic
1 FG/RG Chairs Science questions for MLGEM, RB, SCIMM, and CP fields
2 Everyone Discussion
- Identify an open question, pair ML and data analysis, and create a plan to pursue answers for the next GEM Workshop.


3. ML-GEM discussion session (10:30am - 12:00pm on 06/28 Friday): We will showcase currently available AI models in the heliophysics community and discuss how to unify them into a single AI modeling framework for understanding and predicting the geospace environment.

Zoom Link: https://unh.zoom.us/j/99110890924?pwd=LsvPTw5LtrTwQVeprH8uQZCbLp4DEF.1. Password: 034895

The session will host three talks from the standalone session (10:30 to 10:54am) and then move to the discussion sessions (10:54am -12:00pm).

ML-GEM Standalone/Discussion Session: 10:30 - 12:00pm on Friday, Jun 28
Name Affiliation Title Time Topic
1 Jacob Bortnik UCLA Using interpretable AI to discover the drivers of acceleration vs depletion radiation belt events 10:30 Inner Mag
2 Qusai Al Shidi WVU Reduced Order Probabilistic Emulator of RAM-SCB: Towards Non-linearity with Deep Learning 10:38 Inner Mag
3 Man Hua UCLA Machine‐learning based identification of the critical driving factors controlling storm‐time outer radiation belt electron flux dropouts 10:46 Inner Mag
4 Hyunju Connor NASA/GSFC Overview of ML-GEL goals, plan, and discussion topics 10:54
5 Everyone Presentation of current ML models (2min/model) : Submit slide format [6] 11:00
Discussion
1. Integrate the ML models into a unified AI framework of Heliophysics Modeling
2. Select a common testing dataset (year, storms, substorms) for the framework validation
3. Determine action items for the next ML-GEM meeting

4. ML-GEM tutorial session (1:30 - 3:00pm on 06/28 Friday): A hands-on tutorial on the Long Short-Term Memory (LSTM) technique that models time-series data. This tutorial will use the LSTM model and the SuperMAG geomagnetic field data, published in Blandin et al. (2022; https://doi.org/10.3389/fspas.2022.846291).

Zoom Link: https://unh.zoom.us/j/97969090948?pwd=KRF3SEW7E5GELUn8DXozlvTYbKsvMa.1. Password: 074062.

ML-GEM Tutorial Session: 1:30 - 3:00pm on Friday, Jun 28
Name Affiliation Title Time Topic
1 Hyunju Connor NASA/GSFC Long Short-TERM Memory Model of Alaska geomagnetic field variation 1:30
2 Matt Blandin UAF Introduction of the LSTM code 1:40
3 Everyone - Hands-on Activities: Follow the instructions below to get started. 1:50

Instructions for the ML-GEM Tutorial Session:

Welcome to the ML-GEM tutorial session! We will be using the Python programming language and the Google Colab platform for our exercises. There’s no need to download or install anything. Just follow these simple steps:

Step 1: Download the tutorial Jupyter notebook and the data files via the provided link:https://drive.google.com/drive/u/1/folders/1CflXlSi5KZ4Iy0f4MkWRrR_j8t6mmLO1

Step 2: Sign in to your Google Drive account.

Step 3: Create a new folder named “LSTM_tutorial” in your Google Drive.

Step 4: Upload the Jupyter notebook and the data files into the “LSTM_tutorial” folder.

Step 5: Open the tutorial file by double-clicking on it. It will launch in the Google Colab platform.

Step 6: By default, the code will execute using the “CPU” option. To change this, locate the dropdown menu next to the “Connect” button in the top-right corner. Select “Change runtime type,” then choose the “T4 GPU” option.

You’re now fully prepared for the tutorial. If you run into any issues during the setup process, don’t hesitate to reach out. One of the co-chairs will be available to assist you. We’re excited to have you join us for the tutorial!