Quality analysis of dual-frequency smartphone-based ionospheric TEC measurements: what can be achieved?

The growing quality of smartphone-based Global Navigation Satellite Systems (GNSS) chipsets opens a new frontier for scientific research in positioning, navigation and timing applications. The portability and a↵ordability of these instruments could enhance the current GNSS receiver global network for atmospheric monitoring purposes. However, the quality of the measurements gathered from smartphones have not yet been fully assessed. In this paper, an analysis of the quality of smartphone-based Total Electron Content (TEC) measurements is performed. The primary focus of this work is to provide a general analysis on the potential of using smartphone observations for ionospheric sciences. Dual-frequency phase observations are used to measure the relative TEC. For this experiment, GPS L1/L5 and Galileo E1/E5a observations acquired with the Xiaomi Mi8 and Huawei Mate20 X smartphones were considered. Both devices are equipped with the Broadcom BCM47755 chipset, which enables GNSS dual-frequency measurements. More than 100 hours of phase observations at mid-latitude during a low solar activity period were gathered. Three di↵erent setup configurations were defined to assess the e↵ects multipath or signal strength may have in the quality of the phase observations. In addition, to detect and discard unrealistic fluctuating phase observations, a quality-check was performed. In the results, good agreement between the slant TEC (sTEC) measurements from the smartphone and the sTEC obtained from a co-located geodetic receiver is presented. Furthermore, the amount and quality of observations discarded by the quality-check are reported, which emphasises the use of the signal strength to indicate the quality of phase observations. The results indicate that the C/N0 and multipath are important —when gathering the data from a geodetic antenna, around 80% of the collected data passed a quality threshold. However, collecting data with the addition of an attenuator, or directly from the smartphone antenna, reduced the valid data to below 50%. However, given the ease of use of a smartphone for data collection, even at 50% of data being usable, this shows potential as a useful course of TEC for ionospheric observations.


Introduction
The access to raw measurements from dual-frequency Global Navigation Satellite System (GNSS) chipsets embedded within smartphones opens a new frontier for positioning, navigation and timing solutions. Combined with the increasing number of smartphones around the globe, the opportunity for future low-cost real-time global atmospheric monitoring services to support precise positioning is real.
Several studies have been recently done exploiting the dual-frequency capabilities for precise positioning based on smartphone observations. Robustelli et al. (2019) performed an analysis on single point positioning (SPP) performance and multipath errors of the Xiaomi Mi8 (Mi8) smartphone, using dual-frequency and multi constellations observations. Psychas et al. (2019) presented SPP and precise point positioning (PPP) results using GPS and Galileo observations, showing the improvements achieved when using dual-frequency over single-frequency positioning. In a similar approach, B. Chen et al. (2019) presented PPP results from di↵erent single frequency smartphones and compared them to the results obtained from the Mi8 dual-frequency smartphone. Fortunato, Critchley-Marrows, et al. (2019) evaluated the performance of PPP and real-time kinematic (RTK) in kinematic mode using the Mi8 measurements, and compared them against the NMEA location provided by the smartphone. They achieved an accuracy of a few meters within two minutes of collecting data, showing improvements over the GNSS-only location provided by the smartphone. Furthermore, several previous authors analysed the quality of smartphonebased measurements. Their studies highlighted issues like the duty cycle mechanism and low carrier-to-noise density ratios (C/N 0 ) Gogoi et al., 2019). These investigations showed that low C/N0 values even appear for high elevation satellites. Therefore, to better account for the code noise (Banville et al., 2019), they suggested using C/N0 based weighting schemes instead of the often used elevation weighting for positioning when using smartphone measurements (X. . A key factor for smartphone-based positioning, and for smartphone-based ambiguity resolution in particular, is the quality of the antenna. The low quality of the antenna and its non-homogeneous gain-pattern yield high values of local multipath and a large and hard to predict phase biases (Humphreys et al., 2016). In Geng and Li (2019), the authors investigate the feasibility of resolving ambiguities when using smartphone observations. They evaluate the impact of initial phase biases on ambiguity resolution considering the Mi8 and Huawei Mate20 X devices, showing the use of an external patch antenna as the solution for semi-professional works. A single-frequency PPP solution, using the Huawei Mate20 X (Mate20 X), was investigated in Banville et al. (2019), who found instability in the positionsolution caused by the multipath. The problem was tackled using constraints from global ionospheric maps, resulting in reduced accuracy of the positioning and showing the need for network-based precise slant Total Electron Content (sTEC) corrections.  analysed the RTK and PPP-RTK accuracy using di↵erent smartphones in short and long baselines, and achieved mm-level accuracy when fixing ambiguities and removing multipath e↵ects. These results support those presented by Humphreys et al. (2016), who showed the potential for ambiguity resolution using smartphone measurements, while also underlining the challenges in a real environment with high levels of multipath.
The use of low-cost receivers for ionospheric sensing has not been extensively evaluated. Hernández-Pajares et al. (2018) compared the Total Electron Content (TEC) obtained from single-frequency low-cost receivers against dual-frequency geodetic receivers. B.  estimated ionospheric vertical TEC and satellite biases with a combination of low-cost and geodetic receivers.
In general, the authors of this paper found one publication only regarding the estimation of ionospheric TEC using dual-frequency smartphone observations. Fortunato, Ravanelli, et al. (2019) presented a comparison of the change of sTEC over time ( sT EC/ t) between measurements from a Mi8 and a co-located geodetic receiver for 15 minutes. They showed that di↵erences below 0.05 TEC units (TECu) in sT EC/ t could be achieved if a moving median on windows of 5 to 30 seconds is applied.
In this paper, a comparison of sTEC calculated from two di↵erent smartphones and a co-located geodetic receiver is reported, to provide a quality analysis of smartphone data for ionospheric sensing. Results from three di↵erent experiments with data collected for around 120 hours are compared, evaluating the impact of the strength of the signal and multipath. To the authors knowledge, this is the second such study conducted, with the main improvements being the comparison of performances across three di↵erent signal environments and longer time-series.

Setup and data collection
Dual-frequency code and phase observations from two di↵erent smartphones, the Mi8 and the Mate20X, were collected. Although the Mate20X employs the HiSilicon Kirin 980 chipset for its platform, both devices are equipped with the Broadcom BCM47755 dual-frequency GNSS chip (Broadcom, 2018), which can track GPS L1/L5, Galileo E1/E5a, QZSS L1/L5, GLONASS L1 and BeiDou B1 signals. In this paper, only the GPS and Galileo constellations are analysed, as dual-frequency signals are generally needed to retrieve the TEC from phase observations.
The data was gathered at Geo++ GmbH in Garbsen, Lower Saxony, Germany. Three di↵erent scenarios were defined. The first set of data was collected by placing the smartphone on a geodetic pillar on the rooftop of the company building. This scenario provides observations close to reality, as they are a↵ected by multipath with surrounding surfaces that can reflect the signals. However, the smartphone and geodetic antennas are separated by roughly 10 m and can be a↵ected by di↵erent surface reflections. This setup will be referred to as smartphone-antenna hereafter. The second set of observations was collected inside an RF-enclosure 1 connected to a geodetic antenna on the rooftop, as explained in  and depicted in Figure 1. The smartphone was located inside the RF enclose, and the signals received by the geodetic antenna on the rooftop were re-transmitted inside the enclose. From the double-di↵erence and positioning analysis carried out in Darugna et al., 2019, we can assume that the reflections within the RF enclosure are all the same for a given signal (e.g. L1). In fact, no additional bias is observed in the double-di↵erence. This set provides an ideal scenario, where far and near-field e↵ects that can influence the smartphone measurements, such as multipath, are minimised. This setup will be referred to as geodeticantenna hereafter. The third experiment was also performed within the RF enclosure but adding a 13 dB attenuator after the antenna. This last scenario provides carrier-to-noise values closer to a real user-case, but still limits the e↵ects of multipath. This setup will be referred to as geodetic-antenna + attenuator hereafter. The di↵erent multipath e↵ects on each scenario are of important consideration. Therefore, no elevation mask is used.
The Geo++ RINEX Logger (Geo++ GmbH, 2017) was used to retrieve the phase and code observations in RINEX format with a sampling interval of 1 s. All the tests were performed with the "Force full GNSS measurements" feature from the Android smartphone activated on the Mi8. This option allows shutting down the duty cycling, which increases the data availability and continuity of the phase measurements. Although this feature is not available on the Mate20 X, phase measurements were continuously collected, and the duty cycle was therefore assumed to be o↵ during the GNSS observation period (Banville et al., 2019). Table 1 summarises all the data used within this paper. The measurements were performed during several days with di↵erent measurement times.

TEC measurements from smartphone observations
The geometry-free linear combination was used to obtain sTEC measurements from both geodetic and smartphone phase observations. With phase observations 1 and 5 as L1/E1 and L5/E5a respectively, the geometry-free combination, following Goad (1985), yields: Figure 1: Setup of the experiments carried out in this work. A choke-ring antenna on the roof of the Geo++ GmbH building is connected to a splitter feeding the RF enclosure where the signal is re-transmitted to the smartphone. A 0-13 dB attenuator was applied to the signal re-transmitted to the smartphone. The pillars on the roof were used to collect open-sky data. ( where f 1 and f 5 correspond to GPS and Galileo L1/E1 and L5/E5a bands central frequencies, respectively. GF is given in TECu. Eq. 1 removes all but the frequency-dependent parameters. GF only depends on the ionospheric sTEC, station dependent errors (e.g. multipath) and phase integer ambiguities and biases.
In order to remove the e↵ect of the unknown integer ambiguity value, in this paper smartphone and geodetic sTEC di↵erences will be used as validation. These are calculated by normalising the sTEC from the entire satellite pass after applying data-continuity filtering techniques, which are introduced later in this section, to the mean sTEC value: where N epoch is the number of epochs throughout the satellite pass. The mean value was selected to minimise the e↵ect of the smartphone's phase noise in the normalisation. For the smartphone measurements, two di↵erent data quality techniques to assess the phase observations are compared. The first one is a standard C/N0 threshold filtering, following Liu et al. (2019), where the authors recommend discarding measurements below 30 dB Hz C/N0 when doing positioning using smartphone observations. For this experiment, the C/N0 threshold value was set at 30 dB Hz.
The second technique is a two-step quality check. This alternative method was developed as not all low C/N0 observations are unusable, and because high C/N0 can occur even at low elevations, where the observations may not be suitable for obtaining reliable TEC measurements.
First, all observation gaps caused by the invalid phase observation flag from the Android API, mostly caused by low C/N0 values and multipath, are identified and catalogued as invalid epoch. Secondly, for the remaining observations, a change in TEC between two epochs higher than a given threshold is identified as cycle slip. Nominal values of TEC change in mid-latitudes for geodetic receivers are 0.01 TECU/s (Zhizhao and W. Chen, 2009). Larger changes can occur naturally as manifestations of scintillation, which is unlikely within the ionosphere at mid-latitudes during geomagnetically calm conditions. Considering the phase noise is around 1 second (0.2 TECU for L1) for the data gathered with the smartphone antenna, and that the length of a phase cycle is around 1.1 TECu in L1, in this paper an in-between threshold value of 0.5 TECu/s is selected. To minimise the e↵ect of the cycle-slips, the comparison between geodetic and smartphone sTEC di↵erence is reinitialised after every epoch identified as cycle-slip. Section 2.1 presents the results obtained using this technique.
After the quality check in the smartphone data, all valid sTEC measurements were compared against the sTEC retrieved from the co-located geodetic receiver. In the next section, the results from this comparison are introduced, alongside an analysis of the quality check process.

Results
In this section, the data quality evaluation and the sTEC comparison results are shown and discussed. First, the results from the quality analysis of the phase observations retrieved from the smartphones are introduced.

Quality assessment of smartphone phase observations
The quality check of the smartphone observations was performed as described in Section 1.2. In this section, valid and discarded sTEC measurements from the new quality check method are presented for the data gathered with both the Mate20X and the Mi8. Results for each frequency from GPS and Galileo, compared to C/N0 and elevation, are shown. First, in Figures 2 and 3, valid and discarded sTEC measurements from the smartphoneantenna scenario are shown. Secondly, in Figures 4 and 5 the data validation is again compared to C/N0 and elevation, but for the geodetic-antenna scenario. All the TEC measurements shown in the figures are from satellites transmitting both L1 and L5. The tests were conducted for di↵erent periods, ranging from 1 hour to 27 hours. Therefore, Figures 2-5 show results starting from the first epoch of each independent experiment, and not sequentially in time.
In these figures, the relation between the phase quality, the C/N0 and the elevation can be observed. Most of the discarded observations correspond to low values of C/N0 at low elevations. However, in Figures 4 and 5, observations with high C/N0 and elevation are discarded, caused by the unavailability of GPS L5 measurements. The reason behind this problem with L5 and not with L1 might be the use of di↵erent antennas for each frequency. Furthermore, the Geo++ RINEX logger does not accept measurements if the phase-validity flag of the Android Application Programming Interface (API) is not set. Another reason might be related to a large uncertainty of the received satellite time . A more in-depth look into the raw-measurements retrieved from the API would be necessary to clarify this specific issue. However, the aim of this paper is to report on the quality evaluation of the available measurements for ionosphere sensing. In Figure 6, an example from G10 and E30 satellites independently for the data gathered with the smartphone-antenna is given. In this figure, the relation between C/N0 and data quality is clearer, which is less dependent of the elevation. It is worth mentioning that the quality is assessed to provide sTEC measurements, which are a combination of phase observations from two frequencies. Therefore, even if only one of the frequencies at certain epoch is identified as a loss of lock or a cycle slips, the observation for both frequencies are discarded.
Regarding the constellations, there is an apparent correlation between the data quality and the constellation involved. Even though similar C/N0 values on L5/E5a and L1/E1 are observed, between 15% and 25% less observations are discarded from Galileo than from GPS. The importance of high C/N0 , which can be low even at high elevations as seen in  1 (e.g. X. . Furthermore, it is interesting to notice that, despite the C/N0 values being smaller in the smartphone-antenna scenario, no great increase is observed in the percentage of discarded measurements. In the next section, the sTEC measurements from the smartphone are compared to sTEC calculated from the co-located geodetic receiver. It is essential to underline that, in the geodetic-antenna setup (see Figure 7), smartphone and geodetic receiver employ the same geodetic antenna. Therefore, smartphone and geodetic receivers experience the same multipath. However, that is not true for the measurements gathered in the smartphoneantenna scenario (see Figure 8), where the two antennas are separated by roughly 10 m, Figure 4: C/N0 and elevation for valid (green coloured lines) and discarded (red coloured lines) phase measurements gathered in the geodetic-antenna setup with the Mate20x.
Figure 5: C/N0 and elevation for valid (green coloured lines) and discarded (red coloured lines) phase measurements gathered in the geodetic-antenna setup with the Mi8. and are a↵ected by multiple di↵erent surface reflections. Figure 7 depicts a typical lowmultipath-environment, where no large fluctuation can be observed for both geodetic-grade receiver and smartphone. On the other hand, Figure 8 shows a di↵erence in the C/N0 variation between the geodetic receiver and smartphone. In fact, the smartphone's C/N0 indicates a much more perturbed variation with some sharp jumps. Information about the impact of the station dependent errors, e.g. multipath, on the smartphone measurements and on the positioning results in these scenarios can be found in Darugna et al., 2019.

STEC comparison
In this section, sTEC values computed from all the datasets are introduced. For each of the datasets, sTEC measurements are discarded based on the C/N0 threshold and the quality- check, and a combination of both. These results are assessed by comparing them with the unfiltered sTEC retrieved from raw observations. An example of a comparison between sTEC di↵erences from the smartphone after the two-step quality check and geodetic sTEC di↵erences for some GPS and Galileo satellite passes are shown in Figure 9. Geodetic sTEC is re-referenced to the mean value of the arc after each data gap in the smartphone data, as seen in in Figure 9b. Two di↵erent behaviours of smartphone sTEC measurements are presented in Figure 9. One type of behaviour, in panel a), shows no cycle-slip e↵ect in the smartphone data. Another type of behaviour, in panel b), presents sTEC di↵erences from cycle-slips in the smartphone phase observations. These di↵erences can reach values up to ⇠ 100 TECu if the fluctuations in the phase are significant and not properly identified and corrected. Figures 10 and 11 show the comparison of all smartphone and geodetic sTEC di↵erence measurements of GPS and Galileo satellites respectively. The figures are centred around the geodetic values for comparison. The smartphone sTEC measurements that are not close to the geodetic values are therefore excluded from the figure. Nevertheless, later in Table 2 all results are statistically summarised. Figure 10: Scatter plot with all the sTEC measurements from the geodetic and smartphone receivers from the GPS constellation.
The results of the quality check show excellent agreement with the geodetic measurements, emphasising the improvement achieved by checking the quality of the phase observations. However, the e↵ect of some fluctuations is observed in Fig. 10, even after the quality check. Comparing Fig. 10 and Fig. 11, raw measurements from GPS with the Mi8 seem to have a more stable behaviour than Galileo measurements, but they all show a general agreement and similar behaviour.
In Table 2, a summary of all the results is presented. The percentage of valid measurements after the quality check (if any), RMSE and 95 percentile (P 95% ) of the di↵erences between smartphone and geodetic sTEC di↵erence are presented. As each sTEC is normalised to the mean value throughout the arc (see Eq. 3), the di↵erences show a zero-mean distribution. It is worth mentioning that measurements are only considered in this table when both frequencies are available.
The poor quality of the smartphone measurements is evident from the values in the table. Without any validation, the RMSE in the smartphone-antenna dataset can reach up to 78 TECu. The use of the geodetic antenna, even with the attenuator, marks an improvement of ⇠ 70%.
When applying any quality check to the phase observations, these di↵erences decrease dramatically. The use of the C/N0 threshold to discard data below 30 dB Hz is exceptionally e cient for the data gathered with the Mate20 X in the smartphone-antenna scenario, where the P 95% is improved from 55.461 to 0.187 TECu. For the tests with the Mi8 in the same scenario, the C/N0 filtering improves the P 95% from values of 155.018 to 0.240 TECu.
Using the geodetic antenna, the di↵erence between the C/N0 and the two-step quality check is smaller. In the geodetic-antenna + attenuator scenario using the Mi8, where C/N0 values are closer to the reality, the RMSE is 0.186 TECu, only discarding ⇠ 10% of the Figure 11: Scatter plot with all the sTEC measurements from the geodetic and smartphone receivers from the Galileo constellation. Regarding the comparison between smartphones, the Mate20 X shows better and more stable performance than the Mi8, with more valid data and with higher accuracy when applying the two-step quality check.
Overall, a trade-o↵ between data availability and quality is seen when comparing the results from the C/N0 filtering and the data-quality check. The C/N0 threshold provides closer results to the geodetic receiver, but discarding ⇠ 15 60% more of observations. Di↵erences below 0.1 TECu can be achieved even in the rooftop scenario, where the test was performed as in a possible open-sky user-case scenario.
A final comparison can be made w.r.t. the positioning results obtained in Darugna et al. (2019) with the same setups. While for positioning the main quality factor seems to be the multipath, for ionosphere sensing the signal strength has a more remarkable impact on the results. In fact, attenuating the signal going into the RF enclosure marks a decrease of the performance of ⇠ 20%, while it is still possible to get a RTK ambiguity fixed solution.

Conclusions
This study presents a qualitative analysis for measuring the ionospheric total electron content based on more than 100 hours of smartphone phase observations. The novel two-step quality check method introduced in this paper significantly improves the quality of the data by discarding TEC measurements coming from rapidly fluctuating phase observations. Smartphone measurements in di↵erent setups have been gathered. An open-sky scenario on the rooftop and an ideal scenario within an RF enclosure where multipath e↵ects are mostly removed are considered. The results obtained from the comparison between sTEC di↵erences retrieved from smartphone and geodetic receivers in the opensky scenario show excellent agreement with the results obtained with the geodetic-antenna setup, when any of the filtering techniques is applied. The RMSE is below 0.4 TECu in all the cases. However, more observations are discarded in the dataset collected with the smartphone antenna. 7.79% and 14.22% are rejected in the smartphone-antenna scenario for the Mate20 X and Mi8 respectively, compared to 3.90% and 2.83% of observations discarded in the geodetic-antenna scenario.
As seen in the geodetic-antenna + attenuator scenario using the Mate20 X, the two-step quality check improves the performance of the C/N0 filtering. Although similar values of RMSE are obtained, the quality check only discards ⇠ 20% of the data. On the other hand, the C/N0 filtering rejects more than 60% of measurements.
The di↵erence between constellations is also noticeable, with more observations discarded for GPS than for Galileo in all scenarios using both smartphones. This shows the higher quality and phase stability of the Galileo signals.
By looking at the di↵erences in performance between the Huawei and the Xiaomi smartphones, the Mate20 X antenna shows a more stable performance for phase tracking. By comparing the results from all the scenarios using the novel filtering technique, it is found that the Mate20 X performs between 4% and 34% better than the Mi8. Furthermore, while the multipath has a more considerable impact than C/N0 for positioning applications, the analysis indicates that the strength of the signal a↵ects the sTEC sensing significantly. In fact, the addition of the attenuator deteriorates the sTEC di↵erence results of both smartphones by ⇠ 20%.
In conclusion, this study has shown the possibility to use smartphone devices for ionospheric TEC monitoring. Additionally, the importance of C/N0 , multipath, observations gaps and cycle slips for ionospheric sensing has been demonstrated. If these parameters are considered and accounted for, the method described in this paper can be applied to di↵erent smartphones, setups and ionospheric conditions. This could enhance the global availability of low-cost ionospheric monitoring instruments.