RG: Modeling Methods and Validation
Chairs
Mike Liemohn
Lutz Rastaetter
Alexa Halford
Josh Rigler
Contact:
Resource Group Formation
Since 2005, the Metrics and Validation and Modeling Methods and Validation Focus Groups have supported modeling efforts at GEM. There was recently a grassroots movement to change the approach to have a standing committee – the Methods & Validation Resource Group. A proposal was approved by the GEM SC in 2019 to play that role instead. The M&V Resource Group has four members on staggered terms of four years, with two new members selected every two years after an announcement in the GEM Messenger, usually prior to mini-GEM.
2020 Activities
Virtual GEM Summer Workshop (July 2020):
The Methods and Validation (M&V) Resource Group arranged for a plenary session tutorial from Dr. Tara Jenson, “METplus - A Unified Verification and Validation Tool for a Coupled Modeling System.” She presented the NOAA forecast verification process and statistical techniques for data-model comparisons. She showed examples of how some of these tools could be used with forecasts of total electron content, solar wind parameters, and auroral zone locations. The only breakout session led by the M&V group was in conjunction with the 3D Ionospheric Electrodynamics and Its Impact on MIT Coupling (IEMIT) Focus Group (FG), in relation to the ionospheric conductance challenge. We also prepared a single slide about the resource group that was shown in nearly every breakout session throughout the week, raising awareness of our existence, plans, and availability to assist with community-wide challenges and metrics assessments. Lutz Rastätter was selected among the M&V leadership to serve as the Resource Group’s representative on the GEM Steering Committee.
Virtual Mini-GEM workshop (pushed back to January 2021):
At the GEM mini-workshop, the M&V leadership led one breakout session to have a discussion about what the community wants and expects from this Resource Group. With 17 participants, it had no formal presentations but a lively conversation generating many ideas about possible directions.
Near-term goals:
From these awareness-building activities, discussions, and feedback, the main near-term goals for the M&V Resource Group are as follows:
- Conduct a thorough and systematic survey of past GEM FG chairs and challenge leaders about their experiences running these community-wide projects
- Development of a GEM Challenge Best Practices document
- Disseminate the key guidelines from this document to all FG leaders and potential leaders writing new FG proposals
- Participate in the organization of challenges with any FG that desires assistance, including posing the correct question set to be addressed, event selection, conducting simulations, managing output, metrics selection, metrics calculation, result dissemination, and archival report development.
- Strategize about possible event challenges and modeling parametric/sensitivity studies to be conducted and then actively encourage the community to undertake these activities
2021 Activities
Virtual GEM Summer Workshop (July 2020):
Our session (marked MMV) is on Friday, July 30, from 1 PM to 2:30 PM Eastern Time.
Ongoing activities:
We are gathering information on past modeling and validation studies that have been performed in recently completed Focus Groups and in groups that are nearing their end. Soon after the GEM workshop we plan to contact leaders for information on their experiences on modeling and validation (M&V) and create a document that we will post here.
We intend this document to be a continuously growing resource for existing Focus Groups and new Focus Groups ahead of proposals in the Fall.
