Evaluation of automatic ionogram scaling for use in real-time ionospheric density profile specification: Dourbes DGS-256/ARTIST-4 performance

Statistical evaluation of the Dourbes (4.6 ̊E, 50.1 ̊N) digisonde automatic scaling of the more frequently used ionospheric parameters (foF2, foF1, foE, h’F2, h’F, h’E, and M3000F2) was performed using automatically and manually scaled data from the time period of 2002 to 2008. Automatic scaling was provided in 92% to 94% of cases for most characteristics, except for foF1 (81%). In terms of the automatic scaling accuracy, the magnitude of the residual error for foF2 and M3000F2 (automatically minus manually scaled values) varied according to local time, season, and solar activity. Although geomagnetic storms appear to affect the automatic scaling, the overall results for the influence of geomagnetic activity were inconclusive. Based on this analysis, error bounds were determined (95% probability) for each characteristic: foF2 (–0.75,+0.85), foF1(–0.25,+0.35), foE(–0.35,+0.40), h’F2(–68,+67), h’F(–38,+32), h’E(–26,+2), and M3000F2(–0.55,+0.45).


Introduction
Real-time ionospheric measurements and updates of empirical models are increasingly sought after due to the requirements for timely corrections to precise satellite ranging systems and to various other space weatherforecasting services [e.g.Stamper et al. 2004, Zolesi et al. 2004, Belehaki et al. 2006, Warnant et al. 2007, Buresova et al. 2009].Considering the still unsurpassed quality of groundbased vertical incidence sounding and the increased number of digital ionosondes that are being installed around the world, many of these services and researchers will rely on this network of ionosondes for valuable input.
The automatic scaling (autoscaling) of ionograms has long been recognized as a priority development considering the ever improving digital ionosonde (digisonde) technology, which increases sounding rates and the cost of manual scaling [Mazzetti and Perona 1978, Reinisch and Huang 1983, Fox and Blundell 1989, Galkin et al. 1996, Reinisch et al. 2005, Pezzopane and Scotto 2005, McNamara 2006, Ding et al. 2007].All of these have led to steady improvements in autoscaling techniques over the years, and modern digital ionosondes now offer real-time autoscaling estimates for the important ionospheric characteristics (Figure 1).
We developed a technique for the reconstruction of vertical electron-density profiles from digisonde and global positioning system (GPS) measurements, which was subsequently converted into a prototype real-time procedure [Stankov et al. 2003].This technique has evolved over the years, and a fully operational system for the monitoring of the vertical electron-density distribution in the local ionosphere has been implemented [Stankov et al. 2011] at the site of the Dourbes digital ionosonde.As this monitoring service relies heavily on autoscaled digital ionograms to provide input to the profile reconstruction procedure, it is necessary to evaluate the reliability of this input.Earlier comparisons of manual versus automatic computer processing have been reported; however, these were for older versions of the autoscaling software [Jodogne 1998], for other digisondes/ locations and time periods [Bamford et al. 2008], and/or for only some of the characteristics that are of interest to us [Pezzopane andScotto 2005, 2007].
The purpose of the present study is to assess the performance of the Royal Meteorological Institute digital ionosonde (Lowell DGS-256) and its autoscaling software (ARTIST -Automatic Real-Time Ionogram Scaler with True height v.4) in terms of both their ability to provide autoscaled values (i.e.independent of the accuracy) and the quality of their autoscaling (i.e.accuracy and precision), for the period of 2002 to 2008.This includes the deduction of error bounds for the autoscaled characteristics that are of particular importance to our service.

Real-time electron-density profile specification based on concurrent measurements from the collocated digital ionosonde and GPS receiver
Digital ionosonde measurements and concurrently obtained GPS total electron content (TEC) data, together with the empirically modeled upper O + /H + ion transition level, are used for determination of the key unknown characteristics: the topside O + and H + ion density scale heights.These scale heights are calculated by the solving of a transcendental equation that arises from the incorporation of a suitable ionospheric profiler (Chapman, Epstein, or Exponential) into formulae that describe the basic features of the plasma distribution/processes in the upper ionosphere [Stankov et al. 2011].The system, which has been dubbed as LIEDR (Local Ionospheric Electron Density profile Reconstruction), acquires and promptly processes the incoming measurements, computes the fullheight ionospheric electron-density profile using the above technique, and displays the resulting profilograms.The current update rate is 15 min, and the latency is about 10 min.LIEDR is primarily designed to operate in continuous real-time mode for service applications, and to simultaneously provide historical data/plots for research applications and for further developments of the system.

The data
All of the measurements were made at the Geophysical Centre in Dourbes (50.1˚N, 4.6˚E) [Jodogne and Stankov 2002).The Dourbes digital ionosonde (URSI code: DB049) carries out regular vertical ionospheric soundings with a For the purpose of the evaluation, the hourly data from the period covering the year 2002 through to 2008 was analyzed.This period comprises both solar maximum and minimum conditions, and geomagnetically quiet and disturbed/storm conditions.The analysis will focus on the scaling of seven of the most important ionospheric characteristics: the critical frequencies (foF2, foF1, foE), the minimum virtual heights (h'F2, h'F, h'E), and the propagation factor M(D) (a conversion factor applied to the critical frequency at vertical incidence to obtain the maximum frequency usable for a given oblique propagation over a distance D; in this case, D = 3000 km, i.e.M(D) = M(3000)F2).The residual error, which is defined as the difference between the automatically scaled EVALUATION OF THE AUTOMATIC IONOGRAM SCALING: DGS-256/ARTIST-4 (autoscaled) and the corresponding manually scaled values from the same sounding, were investigated for each of the above-mentioned characteristics.All of the manual corrections of the autoscaled values were performed by experienced scalers who followed the ionogram scaling rules [Piggott andRawer 1972, Wakai et al. 1987] and who worked in close cooperation.Notwithstanding the possibility of occasional human error, the approach implicitly assumes that the manually scaled values are correct.The data used for this study are based on more than 60000 ionograms.

Assessment of the digisonde ability to provide autoscaled values
In this section, we investigate the performance of the digisonde autoscaling algorithm, in terms of the software ability to provide autoscaled values as output, independent of the accuracy.The autoscaling quality (i.e.accuracy) will be the subject of the following section.
For this purpose, we extracted all of the cases when the digisonde failed to deliver autoscaled values for any of the seven characteristics in question (e.g. Figure 2a, b).
Possible reasons for these failures are represented by Dlayer absorption, blanketing by the sporadic E-layer, bands of strong interference, among others [e.g.Bamford et al. 2008, Buresova et al. 2009].The number of totally unscalable ionograms was insignificant.The remaining cases of failed autoscaling for which it was possible to obtain some/all of the characteristics after manual scaling were analyzed, and the data are summarized in Figure 3.
The scatter plots in Figure 3a show the occurrence of these autoscaling failures with respect to the Dst and F10.7 indices.The majority of the cases are clustered around the low Dst and F10.7 values, with no obvious dependence on magnetic activity/ storm conditions.More detailed statistics (Figure 3b) confirm that there is no significant difference in the distribution of failed and autoscaled values, with respect to the geomagnetic index Dst.However, failures appear to occur more frequently during low solar activity conditions for foF1, during mid-level solar activity for foF2, h'F2, h'F, h'E, and M(3000)F2, and during higher solar activity for foE.Also, the autoscaling was more susceptible to failure during the daytime.Overall, the autoscaled values are provided in 92% to 94% of cases, except for foF1 (81%).

Comparisons between the automatically and manually scaled characteristics
A more demanding task is to investigate the quality of the autoscaling.For this purpose, the residual error for each characteristic was calculated and analyzed statistically, in search of possible dependency on solar and geomagnetic activities, local time, and season, and of the probable reasons behind these dependencies.
First, crude time series of the autoscaling errors were produced for the 2002 to 2008 period.This period comprised all levels of solar and geomagnetic activity conditions.Figure 4 initially reveals some obvious patterns in the error variations.For example, the foF2 characteristic appear to be frequently (and on several occasions quite substantially) underestimated (which causes higher residual errors) by the autoscaling procedure during years of high solar activity, compared to the period of minimum solar activity from year 2007 onwards.Given the higher number of geomagnetic storms during high solar activity, this indicated the need to also look at a probable influence from the geomagnetic activity.A somewhat opposite effect is seen in the foE errors.There is an obvious seasonal effect in the error for h'F2, with that error showing a substantial increasing in the summer.It appears also that the h'E is systematically underestimated by the autoscaling software.
Further on, the scatter plots of the autoscaled against the manually scaled hourly characteristics were produced (Figure 5A).Each set of data was fitted to a line.For the frequency characteristics, Figure 5A demonstrates that the fit is generally good, with a slope close to 1 and only small intercepts.Nonetheless, rather large scattering is observed for the virtual heights and the propagation factor, meaning that, in general, the virtual heights are determined less reliably than the critical frequencies.Also, prominent layers ('ledges'), extending to the right of the 'ideal' line (with a slope of 1), are observed in foF2 (Figure 5A).It is obvious that the autoscaling algorithm underestimates the true values of the frequencies at around 4 MHz, 6 MHz, 7 MHz, 9.5 MHz, and 11.5 MHz.This is caused by gaps in the echo trace that lead to the premature truncation of the automatic layer traces.Investigations of the raw ionogram data revealed that this phenomenon is most probably due to interference from external radio signals, the presence of which can significantly raise the noise level [Bamford et al. 2008].As the digisonde monitors the noise for each sounding frequency and only retains signals that are larger than a pre-defined noise threshold, an increased noise level would cause a gap in the echo trace (Figure 2c, d).While the smaller gaps can sometimes be successfully managed, the larger gaps can cause either false extrapolations (Figure 2c) or truncation (Figure 2d).This latter can be quite substantial, and can occur more often around certain frequencies, which thus form ledges in the scatter plot.Virtual heights can also be difficult to scale properly; sometimes the echo trace is weak or the algorithm is misled by false inflexion point/s in the echo trace (Fig. 2E, F).In such cases, the errors can be substantial, especially during storms (Figure 2f ).For h'F2, autoscaling underestimations are more often observed (cf. Figure 5a, with scatter at around 200 km).
In addition to the scatter plots, error histograms were produced (Figure 5b).'Perfect' matches (error within ±0.05 MHz) are achieved in 58% of cases for foF2, 78% for foF1, and 54% for foE.About 56% of the M(3000)F2 errors are within the ±0.05 MHz limit, and more than 68% of the h'F2 and h'F errors are within the ±5 km range.The h'E error histogram confirms that the autoscaling is systematically lower (exceeding 10 km, on average).
To assess the quality of the automatic scaling under varying solar and geomagnetic activity conditions, the residual errors in the autoscaled hourly values were compared (Figure 6) with the F10.7 and Dst indices.With respect to the F10.7 index, the plots show that the foF2 errors    are more often negative than positive, with larger errors observed with high solar activity.The majority of the h'E errors are negative at all levels of solar activity (cf. Figure 4).For the Dst index, the great majority of the errors are clustered around the crossing point of the zero axes, which suggests that there is no obvious relationship between the Dst index and the errors in any of the autoscaled values (Figure 6b).Examination of the correlation coefficients between Dst and each of the characteristics for the entire period reveals that the magnitudes of the coefficient do not exceed 0.2, which indicates that the variables are essentially uncorrelated.This is in agreement with the evaluation of the Chilton digisonde autoscaling [Bamford et al. 2008].
To analyse the diurnal and seasonal variations in the autoscaling accuracy, contour plots were generated for the residual-error magnitude (autoscaled minus manually scaled values), with respect to the day of the year and the local time (Figure 7).The color scale indicates the error magnitude, where blue indicates negative residuals and red indicates positive residuals.Visual inspection of these data in Figure 7 reveals that there are several patterns.The foF2 data show pronounced underestimation of the soundings in the dusk hours during high and medium solar activity.As an opposite effect, overestimation is observed during the winter nights at high and medium solar activity.These two patterns gradually fade away at lower levels of solar activity, when they are replaced by slight underestimation during the daytime.The autoscaling quality improves noticeably during the solar minimum years of 2007 and 2008.The h'F2 results are characterized by slight underestimation, most of the time.Some structures appear in the dusk hours at higher solar activity, although these disappear during the low activity years.Substantial negative residuals are seen during the summer daytime in 2003, 2005, and 2008, some of which coincide with periods of geomagnetic storms.The foE error is predominantly positive (with a tendency to increase at lower solar activity), and as already shown, the h'E error is almost exclusively negative.The error variations in M(3000)F2 are almost opposite to those in foF2: overestimation after dusk, and underestimation at night and during minimum solar activity.

Conclusions
A comparison has been made here between autoscaled and manually scaled ionospheric characteristics based on data from 2002 to 2008 from the Dourbes digisonde.These data show that that the DGS-256/ARTIST-4 system can provide automatic scaling in 94% to 98% of cases for all of the characteristics, except for foF1 (89%).In terms of the autoscaling accuracy, for some characteristics (most notably for foF2 and M3000F2), the magnitude of the residual error (autoscaled minus manually scaled values) varies according to local time, season and solar activity.Although geomagnetic storms appear to affect the autoscaling, the overall results for the influence of geomagnetic activity remain inconclusive.
Overall, the automated ionogram processing/scaling has demonstrated sufficiently good performance to allow the use of the instantaneous ionospheric sounding data for the operation of a monitoring system (LIEDR) for deduction and imaging of the vertical distribution of the electron density in the local ionosphere (http://ionosphere.meteo.be/).The growing availability of Global Navigation Satellite System TEC and ionosonde measurements, combined with the demonstrated ability to run LIEDR in real-time, gives both this approach and this system great potential.

Figure 2 .
Figure 2. Autoscaled ionograms from Dourbes.(a, b) Autoscaling failures, as (a) partial, with only the E-layer parameters scaled, and (b) completely unscaled, as an example of the severely depleted ionosphere during a geomagnetic storm.(c, d) Gap occurrence (due to interference), where smaller gaps are successfully ignored/ interpolated, and larger gaps are falsely interpolated/extrapolated, resulting in the automatic layer trace being truncated prematurely (d).(e, f ) Incorrect autoscaling of the h'F2 virtual height.Auto, automatic scaling; manu, manual scaling.

Figure 3 .
Figure 3.The digisonde autoscaling ability for different ionospheric characteristics during the period of 2002 to 2008.(A) Failed autoscaling, Dst [nT] versus F10.7 [10-22 Wm-2Hz-1].(B) Distribution of the failed autoscaling (black) and the number of autoscaled values (red) with respect to the Dst (left) and the F10.7 (right).The percentages of the autoscaled values are given below the label for each characteristic.

Figure 5 .
Figure 5. Comparisons between the automatically and manually scaled ionospheric characteristics during the period of 2002 to 2008.(a) Scatter plots of the residual errors (automatically versus manually scaled values).(b) Histograms of the residual errors.