Estimating depths and dimensions of gravity sources through optimized support vector classifier (SVC)

By researching and applying new methods we will be able to improve significantly estimation of shapes, dimensions and depths of gravity sources. After shapes estimation of gravity sources through support vector classifier (SVC) in our last research [Hekmatian et al. 2015], in this paper SVC is applied for estimating depths and dimensions of gravity sources. These estimations give us logical and complete initial guesses regarding shapes, depths and dimensions of gravity sources which are needed in more precise interpretations and inversions of gravity sources. Also for better application of SVC, we selected more proper features using the technique called feature selection (FS). In this paper, we trained SVC with 320 synthetic gravity profiles for estimation of dimensions and depths of gravity sources. We tested the trained SVC codes by about 200 other synthetic and some real gravity profiles. The depths and dimensions of a well along with two ore bodies (three real gravity sources) are estimated during the testing process.


Introduction
The dimensions and depths estimation of the gravity sources contains errors. Applying new methods can decrease the estimation errors. In this research, we show that SVC algorithm is a proper method for the mentioned estimations. Support vector classifier (SVC) algorithm is one of the pattern recognition (PR) algorithms and is considered as a supervised learning algorithm (that should be trained) of PR [Van der Heijden et al. 2004]. Regarding producing training data in gravity, it should be mentioned that we need to create several synthetic gravity profiles caused by synthetic gravity sources with different definite shapes, depths and dimensions. Then, values of proper features from each of synthetic gravity profiles will be extracted (see Sections 2.2 and 3). These values are used to train the supervised learning algorithm (SVC) and this trained algorithm would be able to estimate dimensions and depths of gravity sources which their approximate shapes was estimated earlier.
Neural network has been used frequently for gravity interpretation and other geophysical applications [Gret and Klingele 1998, Van der Baan and Christian 2000, Osman et al. 2006] unlike the SVC. SVC algorithm has very suitable properties such as single global solution and ability to use different kernels. It should be mentioned that one of these kernels is Gaussian (or RBF) kernel [see Van der Heijden et al. 2004]. It is proper for separating the objects of one class that its features are distributed in feature space in a nonregular condition (see Figure 1 as an example). More usually the features of the objects related to each shape of gravity sources are distributed in the mentioned manner in the feature space. So this nonlinear kernel has been applied for shape estimation of gravity sources [Hekmatian et al. 2015].
Also according to [Moran and Lavrenko 2011], Minkowski kernel (see Section 2.1) mimics logical AND of variation in feature values, whilst Gaussian (or RBF) kernel is closer to logical OR. Estimating the shapes of gravity sources (separating the class of objects related to each gravity source shape) is a logical OR, since each shape is not related to other shapes.
On the other hand, for estimation of dimensions or depths of different gravity sources, we are obliged to select two different classes of depths or dimensions. For example, suppose that we want to estimate the depths of gravity sources with spherical shapes with a radius of about 20 m, and we know that these gravity sources are located at a depth between 5 m to 1000 m. In these conditions we have two classes: one class is the spherical gravity sources with depth equal to 5 m, and the other class is the same gravity sources but with depth equal to 1000 m. So we should use a proper training set consisting of these two classes. Also we should use a nonlinear kernel of SVC which can estimate the depth of the gravity sources by detecting the probability of each of the two mentioned classes for the gravity sources. For example, if one of the gravity sources is related to class 1 (with a depth of 5 m) with a probability equal to 0.2 and the other gravity source is related to class 2 (with a depth of 1000 m) with a probability equal to 0.8, then its estimated depth is (0.2*5)+(0.8*1000) = 801 m. So a kernel that mimics logical AND could be a proper one for estimating depths and dimensions of gravity sources. Therefore, we applied Minkowski kernel for estimations of the depths and dimensions of gravity sources.
For better application of SVC, we need to optimize SVC by selecting proper features. We have done that by applying FS.

Support vector classifier (SVC) algorithm
2.1.1. Linear SVC Prior to linear SVC description we should mention that support vector machine (SVM) is a concept in computer science for a set of related supervised learning methods that analyse data and recognize patterns used for classification and regression analysis, also the original SVM formulations for classification is named SVC.
SVC is clarified briefly in this and the next section according to Van der Heijden et al. [2004].
Linear (and nonlinear) SVC is considering only 2 classes of information that can be separated by a linear border. One class will be located on the one side of the linear border and the other will be located on the other side of the linear border. If there are more than 2 classes, all classes except one of them should consider as one class and the remaining one as the other class. We should know that in SVC the border between the 2 classes is not a line but a margin and this border line is in the centre of the margin. The linear discriminant function that is used in linear SVC is [Van der Heijden et al. 2004]: where Z n is the matrix of the values of the features (measurement vectors), w is the vector of coefficient and b is a scalar value. The rows of Z n are the features of each object (each gravity profile) and the columns of this matrix show the kind of features in all objects. Two classes (C n =1 and C n =−1 are labels of two classes) are defined by Van der Heijden et al. [2004] as: where w T means transpose of w. The length of the margin between these two classes is shown in Figure 2. For more explanations regarding linear SVC, one of proper references is Van der Heijden et al. [2004].
2.1.2. Nonlinear SVC At first some definitions should be mentioned. It should be mentioned that decision function is a function that maps the measurements space (features space) Figure 2. The linear support vector classifier. The boundary between 2 classes is not only a margin but also is a margin with maximum width and this shows the single global solution of SVC [Van der Heijden et al. 2004]. on to the set of possible classes. It should emphasized that in SVC only 2 classes exist. Therefore in linear and nonlinear SVC, decision function is consist of only 2 discriminant function (g(Z)≥1 and g(Z)≥−1 and notice to relations 1 to 3).
To have nonlinear boundaries in SVC, the discriminant function should be defined as [Van der Heijden et al. 2004]: where k(Z,Z n ) is the kernel (such as polynomial one) and N S is the number of samples in the training set. In relation 4, g(Z) = 1 and g(Z) = −1 show upper limit and lower limit of the margin of nonlinear SVC respectively (see Figure 2 for comparing with linear SVC). Minkowski kernel is a kind of nonlinear kernel [Duin et al. 2007]: where in Equation (5) p is a parameter for increasing or decreasing the power. As we have mentioned earlier, we used this kernel in the depth and dimension estimations.

The features and features selection (FS) algorithm
Feature is a kind of measurement value which is obtained from each object. In this paper, object is a gravity profile which is created by a specific gravity source. Features that can be used for defining the depths and dimensions of the gravity sources are derivable from a principal gravity profile. Gret and Klingele [1998] noted that the principal gravity profile is the gravity profile passing through the maximum value along the anomaly and crossing the anomaly lines perpendicularly. Gravity profiles in this research mean principal gravity profiles.
Some examples of the different features used to characterize the gravity anomalies are defined as [Gret and Klingele 1998]: where in the above equations, gy is the value of gravity (g) at y% of maximum value of gravity g max and X gy is the value of X at gy (for more explanations we should say that in our research X is the horizontal distance in the direction of the profile from the location of g max in the profile). Also X ginf is the value of X at ginf and ginf is the value of gravity (g) at inflection point of the gravity profile. In Figure 3 an example of a gravity profile and some of the features describing the depths and dimensions of gravity sources are shown. Choosing proper features (features selection -FS) is very important while dealing with high misclassification in testing our classifier (more proper features are those features that when are used in classification leads to better separation of the classes). The high misclassification means in our selected features space, the different classes are not separate properly. One of the enable FS algorithms is "inter/intra class distance" (IICD) and we have used this algorithm in our research. IICD algorithm like all other FS algorithms, has a criterion for selection of more proper features. This criterion is called performance measure (PM). PM is increased when the suitability of the features for classification is increased and vice versa in other words selection of more proper features is done when their related PM is bigger. PM of IICD is based on Euclidean distance between each of the two objects in the training set (all objects used for training SVC) in the features space. For more explanation about "inter/intra class distance" (IICD) algorithm, we should mention that as Van der Heijden et al. [2004] noted T s is a training set with N s samples (objects). The classes w k are represented by subsets T k (subsets T k are subsets of objects of T s ), each class having N k samples. Measurement vectors in T s without reference to their class are denoted by Z n (for each object we measure a DEPTH AND DIMENSION ESTIMATION USING SVC This matrix is "the within-scatter matrix" as it describes the average scattering within classes. In addition to this is "the between-scatter matrix" S b that describes the scattering of the class-dependent sample means around the overall average [Van der Heijden et al. 2004]: We know that an individual number is a kind of matrix. So that when "the within-scatter matrix" (S w ) is an individual number, it shows the squared average distance of the location of each object in one class from the average location of all objects of that class in features space for all the classes. In this case we name S w as intraclass distance, and when "the between-scatter matrix" (S b ) is an individual number, it refers to the squared average distance of the average location of all objects of each class from the average location of all objects of all classes in features space. In this case we name S b as interclass distance. The features that are more suitable for classification are the ones that their interclass distance is greater than their intraclass dis-tance. Such that a PM suited to express the separability of classes is the ratio between interclass and intraclass distance. Therefore features with greater PM are preferred for classification (according to IICD algorithm).
Also we should take into consider that the number of possible subsets, q(D), consisting of D features between N features is [ Van der Heijden et al. 2004]: We have used the FS approach as will be described in the following of the paper.

The procedure
To conduct this research, we needed to program a number of software for producing synthetic gravity profiles, extracting features from them, selecting proper features and training SVC for the shape estimation of the gravity sources. There is more explanation in our last research [Hekmatian et al. 2015] regarding these software. Also the needed SVC codes (for the depth or dimension estimation) for this research are implied during the procedure explanations in the following. The overall procedure which we used is as the following: (1) Producing more than 500 synthetic gravity profiles for training and testing the SVC codes which we produced for estimating the depths and dimensions of gravity sources, having one of the six supposed shapes (in our previous research [Hekmatian et al. 2015], we applied SVC in estimating the approximate gravity sources shapes between six possible shapes: sphere, horizontal cylinder, vertical cylinder, rectangular prism, syncline and anticline). From these profiles, about 320 were used for training and about 200 were used for testing.
(2) Extracting 3200 features from the 320 profiles which we used for training. In other word, we extracted 10 features from each of the mentioned 320 profiles. These features are shown in Table 1 (for more explanations regarding the features, see Section 2.2). These features have good characters: first, they are related to the depth and dimension of the gravity sources and are independent from the shape of the sources; second, they consist of all X gy from X g50 to X g85 systematically; third, they are actually obtainable from common real gravity profiles (for example, in real gravity profiles usually X g50 exists but X g10 does not exist).
(3) Selecting the most proper 2 features between the mentioned ten features (by inter/intra class distance FS algorithm; see Section 2.2). We used the most proper 2 features of the training sets to train our SVC codes (for depths or dimensions estimations). According to the shape of gravity sources and also the kind of depth and dimension estimations respectively, the most proper 2 features were different. This is shown in Table  2. Here it should be mentioned that it is possible to select the most proper other number than 2 features (for example the most proper 3 features) between the mentioned ten features by inter/intra class distance FS al-gorithm. But by selecting the most proper 2 features, we have received good and proper results (as will be showed in the following of this paper). Therefore it was not needed to select the most proper other number than 2 features. In any way, we have showed the procedures for selecting any number of features and other researchers can select any number of features in their researches.
(4) Extracting the most proper 2 features (related to Table 2) from each of about 200 synthetic gravity profiles which were produced for testing our trained SVC codes (for depths or dimensions estimations). We tested our trained SVC codes by the mentioned most proper 2 features extracted from testing sets. Using the mentioned most proper 2 features, depth and dimension estimation was done properly. As you see in Table 2 the number of different kinds of depth or dimension estimations of the gravity sources is 16. It should be mentioned that for testing each of these 16 different kinds, we produced 10 to 20 synthetic gravity profiles and for training each of the mentioned 16 different kinds, we produced about 20 synthetic gravity profiles too.
(5) Testing our trained SVC codes (for depth or dimension estimations) by some real gravity profiles and the proper features that we extracted from them.

Synthetic data (and some other discussions)
For depth estimation through SVC, we should know the approximate shape and the range or approximate dimensions of the gravity source; it is the same case for dimension estimation. Different kinds of depth and dimension estimations that we did using SVC are listed in Table 2. We show some examples of these different kinds of depth and dimension estimations (with synthetic data in the following).
Suppose we have estimated the shape of the gravity source to be a vertical cylinder. Also regarding its minimum depth using other methods we have reached to a range of 2 m to 9 m and a range of 2 m to 14 m regarding its height. Its density difference (with the background) is 2.5 gr/cm 3 . In this condition, we can train a SVC code by proper features extracted from a training set. This training set could consist of 20 synthetic vertical cylinder gravity sources which have density difference (with the background) the same as the gravity source and have the mentioned ranges of minimum depths and heights. The specifications of this training set are shown in Table 3.
As it has shown in Table 2, the best 2 features for training SVC code for radius estimation of vertical cylinder gravity sources are feature 1 (X g50 ) and feature 9 (g max ).

Number of the extracted features for depth and dimension estimation
Extracted features Integral of g(x) in central 100 m of profile The kind of depth or dimension estimation of the gravity source Name or number of 2 proper features (see Table 1) Radius of sphere 8, 9 Minimum depth of sphere 7, 9 Radius of horizontal cylinder 9, 10 Minimum depth of horizontal cylinder 8, 9 Radius of vertical cylinder 1, 9 Minimum depth of vertical cylinder 7, 9 Height of vertical cylinder 9, 10 Minimum depth of rectangular prism 6, 9 Vertical extension of rectangular prism 9, 0 Horizontal extension of rectangular prism 5, 9 Minimum depth of anticline 8, 9 Vertical extension of anticline 9, 10 Horizontal extension of anticline 9, 10 Minimum depth of syncline 1, 9 Vertical extension of syncline 9, 10 Horizontal extension of syncline 9, 10  For testing the mentioned trained SVC code, we used a testing set with 12 synthetic gravity profiles created by vertical cylinder gravity sources which have density difference (with the background) equal to 2.5 gr/cm 3 and with minimum depths and heights equal to 5 m and 7 m respectively. Table 4 shows the real radii, feature 1 (X g50 ) and feature 9 (g max ) of the mentioned synthetic gravity profiles and the radii which are estimated by the trained SVC code (proper for radius estimation of vertical cylinder gravity sources) and the errors of the radii estimations.
If we want to estimate the radius of vertical cylinder gravity sources with other depth, height, density difference and other range of radius, we should use other training set. The results of training the mentioned SVC code and testing it in graphic style are shown in Figures  4 and 5.
The radii of vertical cylinder sources are estimated using Minkowski kernel (Equation (5) with setting its parameter to 0.1) based on the following formula: ((The probability of having radius equal to 10m calculated by the trained SVC)*10) + ((The probability of having radius equal to 1m calculated by the trained SVC)*1).
We should mention that the range of radius that can be estimated by the mentioned trained SVC is from 1 m to 10 m. Also, as it has been noted before, estimating the radius of vertical cylinder gravity sources with other range of radius needs using other training set for proper training SVC.
Estimation of the depth and dimension of sources  Table 3. The specifications of the training set which we used for training SVC code for radius estimation of vertical cylinder gravity sources. Table 4. The real radii, feature 1 (X g50 ) and feature 9 (g max ) of the synthetic gravity profiles (of the testing set) and the radii which are estimated by the trained SVC code (proper for radius estimation of vertical cylinder gravity sources) and the errors of the radii estimations.
with other shapes (see Table 2) were, in general, in the same manner with little differences. In continuation, three other examples including, minimum depth and height of vertical cylinder source estimation in addition to estimation of minimum depth of rectangular prism source by their related trained SVC on their testing sets are presented.
Regarding the minimum depth of vertical cylinder estimation, the training set includes 20 synthetic gravity profiles which are created by vertical cylinder gravity DEPTH AND DIMENSION ESTIMATION USING SVC   sources. 10 of these profiles have sources at minimum depth equal to 3 m and 10 other profiles have sources at minimum depth 15 m. The radii of all the 20 profiles are equal to 5 m or 6 m and their height range is 1 m to 36 m. The density difference with background for all of them is equal to 2.5 gr/cm 3 . With such training set, we would be able to estimate minimum depth of those vertical cylinder gravity sources which their minimum depths are between 3 m and 15 m and have the mentioned ranges or values of height, radius and density difference. The estimated minimum depths of vertical cylinder sources of the testing set are shown in Table 5. The training set regarding the height of vertical cylinder sources consists of 20 synthetic gravity profiles with 2 classes. One class is related to height equal to 2m and consists of 10 synthetic gravity profiles created by vertical cylinder sources with height equal to 2 m and the other class is related to height equal to 25 m and consists of 10 other profiles with the height of their sources equal to 25 m. Radii of the mentioned 20 profiles are 5 m or 6 m and the range of their depths are between 5 m to 7 m. Their density difference with background is 2.5 gr/cm 3 . The estimated heights of vertical cylinder sources of the testing set are shown in Table 6.
According to the 3 mentioned testing sets (see Table  4 to Table 6), if the radii, depths and heights of vertical cylinder sources are about in the middle of the used 2 classes of each of them, then they can be estimated with low errors. For example the lowest error pertaining to radius estimation is related to radius equal to 5 m or 6 m (see Table 4). The reason is that as we mentioned in Section 1 according to [Moran and Lavrenko 2011], Minkowski kernel (relation 5) mimics logical AND of variation in feature values.
Regarding estimation of minimum depth of rectangular prism sources, the training set consists of 20 rectangular prism sources with density difference with background equal to 2.5 gr/cm 3 . Ten of them have the minimum depths equal to 2 m and depths of other ten are equal to 25 m. Horizontal and vertical thickness of all of the 20 rectangular prism sources is between 14 m and 18 m and between 20 m and 30 m respectively. After training by the mentioned training set, we examined the trained SVC codes with the testing set. The results are shown in Table 7.

Real data
We have two sets of real gravity data. The approximate location of these 2 sets is shown in Figure 6 by dark spots.
The first set of real data belongs to the small grid gravity network at Institute of Geophysics of University of Tehran. For gravity data acquisition, a Scintrex (CG3M) gravimeter with 1 micro-gal resolution is used and the coordinates of the points were measured by the Total Station Trimple 5601 with accuracy about 1 cm in x, y and h coordinates. The goal was to detect and model the tunnels of an old buried channel. The grid space of 2*2 square meters is used for measuring the data. In the measured data, by using the gravity cor-  6. The approximate location in Iran of the 2 sets of real data is specified by dark spots. The geographic map of Iran is extracted from Ardestani [2008]. Table 7. The real minimum depths of gravity profiles of the testing set and minimum depths estimated by the trained SVC code (proper for minimum depths estimations of rectangular prism gravity sources) and the errors of minimum depths estimations.
rections due to standard formulas, the Bouguer gravity anomalies are computed and the trend effects are removed by polynomial fitting method, and finally the residual anomalies are calculated and shown in Figure  7. We should confirm that contour interval of 1 microgal is the default presentation of the software used for producing Figure 7 (geosoft) and it is only for better presentation of the figure and does not show the real accuracy of the mentioned figure.
A gravity profile (line AB in Figure 7) is selected for testing. The mentioned profile is shown in Figure 7-1.
This profile was collected above a well i.e. the gravity source of this gravity profile was a well full of air. We knew that the well was very similar in shape to a vertical cylinder or a long vertical rectangular prism.
The second set of real data belongs to the grid  gravity network carried out in a mining site close to Zanjan (a city in Iran). The grid space of 10*10 square meters is used for measuring the data. For this set of real data, a Scintrex (CG3M) gravimeter with 1 micro-gal resolution is used and the coordinates of the points in the second set of real data were measured by the Total Station (Leika750) with accuracy to a few cm in x, y and h coordinates. After gravity corrections due to standard formulas, the Bouguer gravity anomalies are computed.
After removing the trend effect by polynomial fitting method, the residual anomalies are computed and for the second set of real data shown in Figure 8. Three real gravity profiles that we used in this set of real data are presented in Figure 8 as the profiles CD, EF and GH. CD, EF and GH profiles are shown in Figures 8-1  to 8-3.
Finally the estimated shapes of the gravity sources of the mentioned real gravity profiles by our different trained SVC codes (the shapes estimations have been explained in detail in our last research: Hekmatian et al. [2015]) are given in Table 8.
It has been shown in the last research [Hekmatian et al. 2015] that the estimation of the shapes of the real gravity sources with SVC is compatible with the reality.
After estimating shapes of the real gravity profiles, now we can estimate their minimum depths and dimensions.
As we mentioned before, the source of AB profile is a well that can be seen in the surface of the Earth, so that we know its depth and dimensions. Regarding the radius of the source of AB profile (see Figure 7 and Table 8), now we know that the shape of this source is DEPTH AND DIMENSION ESTIMATION USING SVC a vertical cylinder. For radius estimation, we should not only know the shape of the source, but also the range of the minimum depth and the height of the vertical cylinder. The minimum depth of this vertical cylinder according to the prior knowledge is less than 2 m (we take this range between 0 m and 2 m). Also we knew that the height is less than 8 meters (we take the range of height between 2 m and 7 m). In addition, the source is a well full of air, so that after measuring the density in the field, we take its density difference with the back-ground equal to −2.0 gr/cm 3 . By these range of possible specifications, we trained the proper SVC code for this case. By the mentioned trained SVC code, we estimated the radius of the source of AB profile approximately equal to 1.05 m. The real radius of the mentioned source is at maximum 1 m (is between 0.6 m to 1 m and therefore the average radius is about 0.8 m).
Similarly, for minimum depth estimation of the source of AB profile, we should know shape, radius and height of the source. So far, we estimated shape and radius; we also know the range of height of the source. So we produced proper trained SVC by which we estimated the minimum depth approximately equal to 0.3 m which is about the same as real minimum depth (zero meter).
Finally, we estimated height of the mentioned source approximately equal to 5.8 m. The real height is 5 m.
Regarding EF profile, the estimated shape of the gravity source is a vertical cylinder (see Figure 8 and Table 8), and its radius, height an minimum depth are estimated by SVC equal to 6 m, 17 m and 6 m respectively.
For GH profile, we estimated its shape as a rectangular prism (see Figure 8 and Table 8), and its minimum depth, horizontal thickness (in the direction of the profile) and vertical thickness are estimated by SVC equal to 16 m, 17 m and 18 m respectively.
Regarding the sources of the profiles EF and GH, we have no real values of their depths and dimensions. But regarding their depths, we should mention that they are consistent with the results of Euler estimations (see Figure 9).
Regarding their horizontal thickness (or diameter), they are consistent with the analytic signals map (see Figure 10).

Conclusion
Estimations of depths and dimensions of gravity sources through SVC are possible. Therefore, estimations by SVC give us initial guesses regarding shape [Hekmatian et al. 2015], depth and dimension of gravity sources which is needed in more precise interpretation and inversion of gravity sources.
Finally, we propose this approach to be applied for geo-scientific investigations as a new approach for the estimation of the depths and dimensions of the gravity sources in exploration projects. We suggest that the above mentioned approach can be used in other geophysical branches.