Team ID: O1-03
Team Title:
Toward a community consensus for metrics in solar physics and space weather
Team Lead:
Evangelia Samara NASA/GSFC, USA evangelia.sam@gmail.com
Team Co-Lead:
C. Nick Arge NASA/GSFC, USA charles.n.arge@nasa.gov
Keywords (Other):
- space weather
- metrics standardization
Keywords (Activity Type):
Understanding , Requirements , Assessment , Roadmap
Introduction:
During the last decades, the solar and space weather community has done significant progress in developing models to characterise and forecast the solar activity and its effects in the interplanetary space. A number of novel spacecraft were also recently launched (PSP, SolO, Bepi Colombo) which, in combination with the older ones (STEREO, ACE, Wind, SOHO, SDO), help us validate the solar and interplanetary conditions with in situ and remote observations. At some point it became clear to the community that validation and evaluation of those modeling efforts was crucial to understand the quality of our models and their progress throughout the years. That's why a continuous interest toward employing appropriate metrics arose, and different authors and modeling groups started employing "their own" metrics and techniques to quantify their modeling output (solar wind/CMEs/SEPs prediction, coronal hole reconstructions etc) in the literature. With the years, it became clear that there is no single metric that can quantify holistically the performance of a model in reference to a specific space weather phenomenon (solar wind, CMEs, SEPs, coronal holes etc). On the contrary, a combination of metrics seemed necessary for the complete and comprehensive evaluation of the modeling output. A number of efforts in the community (see also action teams within the COSPAR-ISWAT initiative e.g., in H1, H2, S2 and other clusters) started to build databases of real and historic predictions by including as many metrics as possible, started creating scoreboards etc, to achieve this goal.
Objectives:
The current action team opens a call of collaboration to the community in order to reach a consensus on the metrics that should be used for the assessment and evaluation of different space weather phenomena. We aim towards the standardisation of metrics for (a) solar wind/CMEs/SEP predictions, (b) the modelling and reconstruction of coronal holes/polar fields in magnetograms, and (c) any other phenomenon/event the community is interested in addressing. With this effort we want to identify which is the minimum combination of metrics that, for each solar/space weather phenomenon of interest, will provide the most complete evaluation so that the community can adopt it for scientific research and publications. The goal is to motivate the community to start using the same metrics for the evaluation of modeling results (suitable metrics will be decided in the frame of this team) in order to achieve transparency, accuracy and consistency for all modeling efforts and scoreboards. We also aim to brainstorm and identify which is the best metric(s) to employ for phenomena that we are still uncertain how to quantify (no metrics available yet) when there is no ground truth because of lack of observations (e.g., polar fields in magnetograms).
Action Topics:
Cluster with overlapping topics:
S2: Ambient solar magnetic field, heating and spectral irradiance, H1: Heliospheric magnetic field and solar wind, H2: CME structure, evolution and propagation through heliosphere
Link to external website: