Comparison of ionospheric scintillation models with experimental data for satellite navigation applications

A comparison between two of the most used scintillation models and experimental data is presented. The experimental data have been derived from a GPS scintillation monitor developed at Cornell University and placed in Tucuman (Argentina), under the peak of the anomaly. The models used (GISM and WBMOD) have been run for the geophysical conditions corresponding to the measurements. The comparison is done by subdividing the information on the basis of an ionospheric grid of 5°×5° surface square boxes. The comparison has been performed for several local times, from 18 LT until 04 LT. Here, only a few cases of particular interest are shown. The goal is to understand if the models are able to forecast actual scintillation morphology (from the satellite navigation systems point of view) and if they could be used to yield an estimate of scintillation effects on satellite navigation systems. Mailing address: Dr. Biagio Forte, The Abdus Salam International Centre for Theoretical Physics (ICTP), Strada Costiera 11, 34100 Trieste, Italy; e-mail: bforte@ictp.trieste.it


Introduction
When traversing drifting ionospheric irregularities, a radio wave experiences fluctuations in its amplitude and phase: the fluctuation characteristics depend on the radio frequency, magnetic and solar activity, time of day, season of the year and magnetic latitude of the observation point in the case of satellite to ground links.
Nowadays, one of the most important fields where as complete as possible knowledge of ionospheric scintillation effects is needed is that of satellite based navigation systems. Ionospheric scintillation is responsible for transionospheric signal degradation that can affect the performance of navigation systems. For this reason several type of analysis of ionospheric scintillation effects on GPS signals have been carried out. In principle, scintillation data can be measured by means of GPS scintillation monitors, in order to directly assess the effect of ionospheric scintillation on GPS signals and thus on satellite navigation operations.
The actual effect of ionospheric scintillation on satellite navigation systems cannot be assessed by analysing experimental data (even if measured by GPS scintillation monitors) in a «classical» way. The classical method used to analyze scintillation data deals with average scintillation activity varying with geophysical conditions such as season and solar activity. Such an approach does not take into account any particular navigation system characteristics and is only able to generate from experimental data global morphology of scintillation activity, as observed at a given site (or in a given area) by means of a particular GPS receiver. Using such an approach the actual effect of ionospheric plasma irregularities on satellite navigation operations can be assessed only to a limited extent.
For satellite navigation systems two levels should be considered at least: user level and

Comparison of ionospheric scintillation models with experimental data for satellite navigation applications Biagio Forte and Sandro M. Radicella
The Abdus Salam International Centre for Theoretical Physics (ICTP), Trieste, Italy Abstract A comparison between two of the most used scintillation models and experimental data is presented. The experimental data have been derived from a GPS scintillation monitor developed at Cornell University and placed in Tucuman (Argentina), under the peak of the anomaly. The models used (GISM and WBMOD) have been run for the geophysical conditions corresponding to the measurements. The comparison is done by subdividing the information on the basis of an ionospheric grid of 5°×5°surface square boxes. The comparison has been performed for several local times, from 18 LT until 04 LT. Here, only a few cases of particular interest are shown. The goal is to understand if the models are able to forecast actual scintillation morphology (from the satellite navigation systems point of view) and if they could be used to yield an estimate of scintillation effects on satellite navigation systems.
Mailing address: Dr. Biagio Forte, The Abdus Salam International Centre for Theoretical Physics (ICTP), Strada Costiera 11, 34100 Trieste, Italy; e-mail: bforte@ictp.trieste.it monitoring station level (Forte et al., 2002). The impact of ionospheric scintillation on each level is, in principle, different and needs to be properly described.
In this work a different approach to the analysis of scintillation data will be presented. This method of analysis is conceived as system oriented since it considers the main operations made by satellite navigation systems at the two aforementioned levels. The next step is to consider data coming from different sites and measured simultaneously, in order to assess scintilation effects on a regional scale.
One tool used for planning satellite systems robust against ionospheric scintillation is modeling. Several modeling efforts have been attempted in order to forecast scintillation activity under given geophysical conditions as well as its effect on satellite navigation systems.
This work compares two of the most used scintillation models (GISM and WBMOD) with experimental data. The goal is to understand if the models are able to forecast actual scintillation morphology (from the satellite navigation systems point of view) and if they could be used to have an give an estimate of scintillation effects on satellite navigation systems.
A validation of WBMOD has been attempted in the Southeast Asian sector (Cervera et al., 2001).

The models used
Several modeling efforts have previously been made to describe the physical mechanism giving rise to scintillation as well as the statistical distribution of ionospheric electron density irregularities. The most important goal is the forecast of scintillation activity. This is required in the evaluation of ionospheric scintillation impact on important applications of radio wave propagation, as satellite based navigation and positioning.

WBMOD
The WBMOD model describes a worldwide climatology of the ionospheric plasma density irregularities which can produce scintillation of radio waves traversing the ionosphere. WB-MOD is a climatological model for ionospheric electron density irregularities coupled with a propagation model able to describe the effects of ionospheric plasma irregularities on transionospheric radio signals (Secan et al., 1995). The parameters of ionospheric irregularities are modeled on the basis of experimental data (Secan et al., 1995), while the propagation model is essentially a phase screen model (Rino, 1979). The model provides the intensity scintillation index S 4 and the phase scintillation index σφ, computed by means of the propagation model under the pre-specified geophysical conditions. The model can compute the percentage of time that those indices would exceed a specified threshold level for given geophysical conditions. It is also able to compute the SI scintillation index, which has engineering applications (Secan et al., 1995).

GISM
The Global Ionospheric Scintillation Model (GISM) provides the statistical characteristics of the transmitted signals, in particular scintillation indices, fade duration, and cumulative probability of the signal, permitting the determination of the margins to be included in a link budget. Maps of scintillation indices can easily be obtained. The model is composed of two submodels. The first model provides the mean errors and is based on a solution of the ray equation for a background propagation plasma. The refractive index is calculated from an accurate value of the electron density inside the medium, which is provided by the NeQuick model (Radicella and Leitinger, 2001). The second model is based on the multiple phase screen technique and provides second order propagation effects, namely radio wave scintillation. Inputs to the model consist of geophysical paramaters, inhomogeneity data, and the operating data (e.g., carrier frequency, link geometry, local time, etc.). The inhomogeneity data are essentially deduced from measurements. Intensity and phase scintillation indices are provided as output (Beniguel, 2002).

The dataset used
The data set used for this study was collected at the Institute of Physics of the National University of Tucuman (Argentina), by means of a scintillation monitor developed and provided by Cornell University (Beach and Kintner, 1999). S 4 scintillation indices averaged over one minute periods are computed. Data were collected over 11 months; the measurements time interval is 12 h during nighttime.
The ionosphere over the Tucuman sky is assumed to be approximated by a thin shell at 350 km altitude. This modeled ionosphere is then interfaced with a grid divided into 5°×5°surface square boxes.
The use of an ionospheric grid at 350 km is suitable to spatially and temporally divide both experimental and modeled information. Thus, an ionospheric grid of 5°×5°surface square boxes is adopted. In this way a clear geographical distribution of the information can be displayed. This method, repeated for several hours, gives at the end a full spatial-temporal view of the observed and modeled phenomena, enabling not only a careful comparison between models and experimental data, but also determination of the impact of ionospheric scintillation on GPS positioning and navigation operations.
The reason for such an approach is to test the use of scintillation models for their application to operational systems like Space Based Augmentation Systems (SBAS). SBAS are GPS augmentation systems which will be used for precision air navigation procedures. Three systems are planned: a) the US Wide Area Augmentation System (WAAS); b) the European Geostationary Navigation Overaly System (EGNOS), and c) the Japanese MTSAT (Multifunctional Transport Satellite) satellite based augmentation system (MSAS).
Basic GPS service cannot meet the accuracy, availability and integrity required by the air navigation precision approach. SBAS are safety critical navigation systems that should provide a quality of positioning information never available to the aviation community before. It should improve accuracy, integrity and availability of the basic GPS service.
SBAS are based on a network of Ranging and Integrity Monitoring Stations (RIMS). Signals from GPS satellites are received by RIMS which determine the actual ionospheric error for a particular area. The corresponding correction to be applied to a single frequency positioning determination is computed and broadcast to a user inside a given area via a geostationary satellite. The analysis presented hereafter has been carried out in the framework of the EGNOS project and its goal is to estimate actual effects of ionospheric scintillation on GPS positioning and on the RIMS calculation of the ionospheric error in a given area. Thus, the excercise of comparing such experimental data with models' predictions has been done to assess whether theoretical models (at least those available) can be used in defining the impact of scintillation on these systems, instead of using actual and not widely available experimental data.
Penetration points are considered at ionospheric grid height, namely at 350 km of altitude. Experimental and modeled scintillation indices are analyzed in each ionospheric grid box. For each ionospheric grid box, percentages of occurrence of S4 greater than given thresholds (i.e. 0.5, 0.8, and 0.9) are computed, in order to assess whether scintillation may have a critical impact on RIMS operations and, thus, on SBAS performance (Forte et al., 2002).

Discussion
The comparison between data and WB-MOD is made on the basis of percentages of occurrence of an S 4 greater than a given threshold (i.e. 0.5). The comparison between data and GISM simulations has to be taken into account with caution, because while GISM is directly computing the most probable S4 within a given ionospheric box and during particular geophysical conditions, the experimental data are in the form of percentages of occurrence of an S4 greater than a given thresholds. This kind of comparison, even if not complete, is in any case significant: high percentages of occurrence of an S 4 >0.5, for instance, indicate a high probability that the average S4 in that particular box is greater than 0.5 and from here the comparison with GISM predictions is feasible.
The experimental data are used in two ways: 1) in a climatological way, where all the measurements taken during a given month and at a particular hour are put together to compute percentages of occurrences; 2) in a «single spot» way, where the measurements taken during just one night and at particular local times are compared with the corresponding model output.
The comparison is considered for several hours, ranging from 18 LT to 04 LT, for both climatological and spot comparisons, and for the months of September 1998, October 1998, February 1999, March 1999, September 1999, and October 1999 Here, the representative cases of September 1998, October 1998, September 1999, and October 1999 Several limitations arise from such a comparison. First of all, the models predict scintillation activity only in a fixed time interval at low latitudes. WBMOD predicts scintillation activity from 18 LT to 02 LT, while GISM predicts scintillation activity from 20 LT to 02 LT. As can be deduced from experimental data (going from 18 LT to 04 LT) ionospheric scintillations can occur on GPS signals even outside these local time intervals. This happens because usually the models are calibrated using particular data sets and do not take into account the proper physical mechanism giving rise to the onset of ionospheric small scale irregularities producing scintillation of radiowaves. Most of the times, this calibration is made on the basis of the percentage of occurrence of scintillation activity in a given hour. Of course, low percentages of occurrence do not mean total absence of scintillation activity. A given model, even if based on experimental data, should produce consistent predictions in other conditions, different from those of the data set used.
Figures 1a-f to 4a-f show the comparison between experimental data and models output for different months at 20 LT and 22 LT, in  terms of percentages of occurrence of an S4 greater than 0.5. Another limitation arising from the comparison of experimental data and theoretical models, is that the models fail to describe the patchy irregularity structure. GPS signals with different link geometries can encounter several regions containing small scale irregularities able to produce radiowaves scintillation. This is a typical feature of ionospheric irregularities: the patchy structure depends on several factors. At low latitudes, different mechanisms can produce radiowaves scintillation: irregularities forming in the bottomside (which produce bottomside spread F) and those associated with fully developed bubbles. The dynamics of the small scale irregularities embedded in the bubbles is extremely difficult to model and the presence or absence of an irregularity able to produce radiowaves scintillation depends on the electron density deviation from the surrounding background electron density and on the Fresnel scale.
These factors strongly depend on the particular geophysical conditions during which such irregularities develop. Both WBMOD and GISM depict a smoothed situation. WBMOD predicts onset of small scale irregularities giving rise to Lband scintillation only in a belt located southward of the observing site. GISM predicts scintillation activity with strongly geographic latitudinal dependence: the peak appears to be located in a wide region northward of the observing site.
This limitation clearly arises from figs. 5a-f to 7a-f, where model predictions are compared to single day experimental data (i.e. the day 15 of each month) at 20 LT. Particular cases are shown, where single day measurements are compared to model predictions. In fig. 7a-f, a case of no experimental S 4 indices greater than 0.5 occurred at 23 LT on 15 September 1999, while the models predict some scintillation events in different ways.
These difficulties lead to modeling results that have to be interpreted in probabilistic  terms. GISM predicts scintillation activity in a wide area over the observing site, with scintillation intensity decreasing with latitude: the patchy structure of ionospheric irregularities giving rise to L-band scintillation (mostly related to bubbles) is completely absent. GISM predicts the same behaviour for scintillation activity at different local times, changing just the scintillation intensity, but not its morphology. WBMOD predictions appear to be more sensitive to changes in local times. WBMOD predicts scintillation activity in a belt southward of the observing site, with peaks in scintillation intensity moving from east to west, for local times going from 18 LT to 02 LT. This description in general seems to be much more realistic than GISM because it takes into account the drift of the ionized plasma (with electron density irregularities embedded in it), as well as the formation and decay of large scale structures (like bubbles in this case) which depend on the local times. This mechanism could be responsible for an increase in scintillation activity in some sectors of the ionosphere above the observing site, namely eastward of the observing site where the onset of irregularities caused by background ionization decay is most likely to occur. Then peaks in scintillation activity move along the east-west direction, corresponding to the east-west drift of ionization decay and of electron density irregularities which form in the evening hours (just after sunset), last for some hours and then decay allowing re-distribution of electron density. In WBMOD, as well as in GISM, the patchy structure of irregularities producing radiowaves scintillation is absent. As already mentioned this patchy structure seems really very hard to model. The scintillation models here analyzed appear to be limited in their ability to estimate the impact of ionospheric small scale irregularities on satellite positioning and navigation systems, like GPS and Galileo, for a series of reasons.
As previoulsy mentioned, the patchiness feature of those irregularities giving rise to Lband scintillation (as clearly visible from ex- perimental data here presented) is not properly modelled. This feature is completely absent from model predictions, even if the models are compared with «climatological» data because experimental data show that the occurrence of patchy irregularities is much more structured (figs. 1a-f to 4a-f). Radiowaves scintillation is just an effect of radiowaves propagation through electron density irregularities. This effect strongly depends on the receiver used and, in principle, different receivers could react in different ways to the same scintillating conditions. Scintillation models just try to predict the occurrence of electron density irregularities and, in a probabilistic way, try to give an estimate of the scintillation indices for a given transionospheric link (by specifying satellite and receiver positions as well as the transmission frequency). The models do not take into account that the effect of scintillation is strongly dependent on the receiver used: for instance, while an S4=0.8 could mean loss of lock for a particular receiver, the same S 4 value could not affect signal tracking in another receiver. This concept is a very important one and indicates that the estimate of the actual impact of scintillation on the performance of a given satellite system has to be considered. In the latter case, the engineeristic approach appears to be extremely relevant.
Scintillation models are climatological models. They of course fail when compared to «single spot» situations (for instance, when compared to single measurements nights), where patchy structures are present and where the receiver response is critically dominant. As climatological models both WBMOD and GISM work quite well since they are able to identify, more or less, the area and the local times affected by scintillation problems. Of course, they fail to resolve the details, because they are just theoretical models calibrated on the global morphology of scintillation activity (figs. 5a-f and 6a-f).
The most important limitation in the use of scintillation models for scintillation effects assessment is the following. Those models are based on the weak scattering theory and, even modeling an irregularity on a given link in the best way, they are limited to low scintillation indices values. This is an intrinsic limitation of the models, since the weak scattering approximation works well only when the electron density fluctuations are not very high (i.e. of the order of few tens of percentage). Low scintillation indices at L-band and in the case of GPS operations do not create significant problems for them. The most severe limitation in the comparison of scintillation models with GPS derived experimental data is focused on very high scintillation activity (i.e. scintillation indices) which is responsible for loss of signal lock and consequently degrading of GPS positioning and navigation operations.
This limitation also justifies the interpretation of such a comparison in probabilistic terms. Because of the weak scattering approximation and validity of models, the comparison of time series of experimental and modeled scintillation indices has absolutely no meaning. What is still consistent, even in the presence of weak scattering approximation in the propagation models, is the modeling of small scale irregularities (and their spatial-temporal features) able to produce radiowaves scintillation at L-band, always in a probabilistic and climatological way.

Conclusions
WBMOD and GISM models have been analyzed against GPS derived scintillation data. These models (as probably any other model in general) are able to reproduce the global morphology of small scale irregularities, including their onset and decay.
Most of the time, the models fail to forecast scintillation indices on a given GPS link. As visible from climatological comparison (figs. 1a-f to 4a-f), the experimental S 4 indices are distributed in areas not predicted by the models, which describe a sort of smoothed area (southward of the receiver for WBMOD and along North-South direction for GISM) where the onset of small scale irregularities is likely to occur.
When compared to single day data, the models clearly show a lack of precision in the detailed resolution. The single day data can be distributed in localized spots, clearly indicating patchy structures. The models, on the contrary, are not able to reproduce the patchy feature of small scale irregularities, since they predict probabilistic belts (North-South elongated for GISM and East-West elongated for WBMOD) of occurrence of small scale irregularities rather than localised or patchy irregularities.
These models reproduce well enough the global morphology of ionospheric scintillations, being calibrated with data sets that do not include GPS derived data. In the case of GPS links the geometry is always changing and different from geostationary or polar orbiting satellite radio links. This leads to another limitation of scintillation models when applied to GPS cases.
From the analysis of the comparison made the use of scintillation models to estimate ionospheric scintillation effects on single GPS links and thus on GPS operations does not appear to be feasible. In addition it should be noticed that ionospheric scintillation effects on GPS operations are strongly and critically dependent on the particular receiver used and on the satellite navigation system requirements. Different receivers can have different responses in the same scintillating environment. The use of scintillation models, then, should be limited to the description of global scintillation morphology and to particular cases as for beacon satellites, taking into account all the aforementioned limitations for satellite navigation and positioning systems.