RG: Modeling Methods and Validation

Chairs

Mike Liemohn
Lutz Rastaetter
Alexa Halford
Josh Rigler

Contact:

Resource Group Leads

Community Discussion

Resource Group Formation

Since 2005, the Metrics and Validation and Modeling Methods and Validation Focus Groups have supported modeling efforts at GEM. There was recently a grassroots movement to change the approach to have a standing committee – the Methods & Validation Resource Group. A proposal was approved by the GEM SC in 2019 to play that role instead. The M&V Resource Group has four members on staggered terms of four years, with two new members selected every two years after an announcement in the GEM Messenger, usually prior to mini-GEM.

2020 Activities

Virtual GEM Summer Workshop (July 2020):

The Methods and Validation (M&V) Resource Group arranged for a plenary session tutorial from Dr. Tara Jenson, “METplus - A Unified Verification and Validation Tool for a Coupled Modeling System.” She presented the NOAA forecast verification process and statistical techniques for data-model comparisons. She showed examples of how some of these tools could be used with forecasts of total electron content, solar wind parameters, and auroral zone locations. The only breakout session led by the M&V group was in conjunction with the 3D Ionospheric Electrodynamics and Its Impact on MIT Coupling (IEMIT) Focus Group (FG), in relation to the ionospheric conductance challenge. We also prepared a single slide about the resource group that was shown in nearly every breakout session throughout the week, raising awareness of our existence, plans, and availability to assist with community-wide challenges and metrics assessments. Lutz Rastätter was selected among the M&V leadership to serve as the Resource Group’s representative on the GEM Steering Committee.

Virtual Mini-GEM workshop (pushed back to January 2021):

At the GEM mini-workshop, the M&V leadership led one breakout session to have a discussion about what the community wants and expects from this Resource Group. With 17 participants, it had no formal presentations but a lively conversation generating many ideas about possible directions.

Near-term goals:

From these awareness-building activities, discussions, and feedback, the main near-term goals for the M&V Resource Group are as follows:

1. Conduct a thorough and systematic survey of past GEM FG chairs and challenge leaders about their experiences running these community-wide projects
2. Development of a GEM Challenge Best Practices document
3. Disseminate the key guidelines from this document to all FG leaders and potential leaders writing new FG proposals
4. Participate in the organization of challenges with any FG that desires assistance, including posing the correct question set to be addressed, event selection, conducting simulations, managing output, metrics selection, metrics calculation, result dissemination, and archival report development.
5. Strategize about possible event challenges and modeling parametric/sensitivity studies to be conducted and then actively encourage the community to undertake these activities

2021 Activities

Virtual GEM Summer Workshop (July 2021):

The GEM Methods and Validation (M&V) Resource Group will host a VGEM session on Friday, July 30th, 2021 (after lunch break, 1-2:30pm EDT). In brief, the M&V Resource Group exists to facilitate geospace model validation efforts, to document and promote common and best practices for M&V amongst the GEM community, and when possible, to archive validation M&V results for future analysis and comparisons. This will be a relatively open session to share recent tools, methodologies, and results of validation efforts related to Geospace modeling. Topics may cover any/all current and recent GEM focus group efforts. Discussion of community modeling "challenges" is certainly welcome, however we would like to keep this session focused on specific tools and results. Please reach out to the M&V team (gem-methods-validation-leads@googlegroups.com) if you have anything substantial to share, and we'll try to carve out a short block of time for you to present. Or just bring a slide or two to add to interactive discussion. This will be the last set of concurrent sessions for the week, and we really hope to encourage open and productive cross-focus-group discussion and interactions that will continue after the close of VGEM 2021.

2021 mini-GEM

Hybrid session at the 2021 mini-GEM, Dec. 12, 2021

Topic: GEM M&V Mini-GEM Session: M&V White Papers Time: Dec 12, 2021, 02:00 PM Central time (New Orleans) / 03:00 PM Eastern time

The GEM Methods and Validation Resource Group will host a session, “Methods and Validation White Paper Discussion,” at the upcoming 2021 Hybrid Mini-GEM Workshop on Sunday, December 12, at 2:00 – 3:30 pm Central US time. We welcome all Mini-GEM attendees to join in this conversation, either in person at the hotel conference room (at the Hilton Garden Inn New Orleans Convention Center) or remotely via Zoom (link available to Mini-GEM registrants). Please bring your ideas for potential white papers to the upcoming Solar and Space Physics Decadal Survey regarding modeling methodologies and data-model comparison validation techniques. If you would like to present a slide or two on your idea, then please come prepared to do so.

Ongoing activities:

We are gathering information on past modeling and validation studies that have been performed in recently completed Focus Groups and in groups that are nearing their end. Soon after the GEM workshop we plan to contact leaders for information on their experiences on modeling and validation (M&V) and create a document that we will post here.

We intend this document to be a continuously growing resource for existing Focus Groups and new Focus Groups ahead of proposals in the Fall.