Difference between revisions of "Detecting the unknown - novelty detection of exceptional water reflectance spectra"

From Coastal Wiki
Jump to: navigation, search
Line 1: Line 1:
 
Introduction
 
Introduction
Synoptic monitoring of large areas in coastal waters can be performed by remote sensing using multispectral sensors on-board satellites. Many methods are in use which enable the detection and quantification of ‘standard algae’ or specific algae blooms using their known spectral response. The ‘novelty detection’ aims to find spectra outside the known range  which are referred to as exceptional spectra.  
+
Synoptic monitoring of large areas in coastal waters can be performed by remote sensing using multispectral sensors on-board satellites. Many methods are in use which enable the detection and quantification of ‘standard algae’ or specific algae blooms using their known spectral response. The ‘novelty detection’ aims to find spectra outside the known range  which are referred to as exceptional spectra.
 
[[Image:Detecting_unknown_donno.jpg]]
 
[[Image:Detecting_unknown_donno.jpg]]
 
 
 
 
 
 
This order given to the hero of a Russian fairy tale could be used to summarise the request received to detect exceptional water reflectance spectra.
 
This order given to the hero of a Russian fairy tale could be used to summarise the request received to detect exceptional water reflectance spectra.
 
A recent review of the novelty detection techniques available is presented by (Markon and Singh, 2003).
 
A recent review of the novelty detection techniques available is presented by (Markon and Singh, 2003).

Revision as of 09:59, 1 April 2009

Introduction Synoptic monitoring of large areas in coastal waters can be performed by remote sensing using multispectral sensors on-board satellites. Many methods are in use which enable the detection and quantification of ‘standard algae’ or specific algae blooms using their known spectral response. The ‘novelty detection’ aims to find spectra outside the known range which are referred to as exceptional spectra. Detecting unknown donno.jpg This order given to the hero of a Russian fairy tale could be used to summarise the request received to detect exceptional water reflectance spectra. A recent review of the novelty detection techniques available is presented by (Markon and Singh, 2003). The present study aimed to develop a novelty detection scheme for water reflectance data which could be used operationally. The basic idea behind this approach is to use a large number of reflectance spectra from one complete year for a defined region, here the North Sea, so as to include the annual cycle of plankton species (including their blooms etc.) to teach a learning system ‘what is normal.’ Only distinctive deviations from these known spectra should be classified as novel. Thus, the algorithm should accept all reflectance spectra from the training period as being normal but should still be sensitive enough to detect novel situations. The Algorithm The selection of the appropriate algorithm is one central issue of novelty detection. It is intimately connected with the type and structure of the data to be tackled. The water reflectances are measured at nine different wavelengths, all of which will be used in the present algorithms. Bio-optical models often exclude several wavebands or give other wavelengths more weight. Examples of water reflectance spectra are shown in Fig. 1. Since our aim is to detect novel situations with unknown spectral signature we use all the information available. However, due to the resulting high dimensionality of the data set, nine dimensions of a point correspond to the nine wavelengths, the performance of different algorithms was carefully analysed. Different approaches of novelty detection, namely: a) a simple statistical scheme, b) an auto-associative Neural Net, c) a Neural Net classifier, and d) a tessellation scheme were analysed. Fig. 2 shows the enclosement of the different methods for a 2-dimensional case. The tessellation showed to be the optimal novelty detection scheme. Scheme The tessellation starts by choosing a radius R in the 9-dimensional space (dimensions for 9 wavelengths).

• The first point (spectrum) becomes the first centre. • For each of the remaining points the smallest of the distances to all centres is compared with R. If the distance is larger than R the point is included in the set of centres. • In the second step each point is assigned to that centre to which it is closest (Voronoi tessellation).

For each of the patches obtained by the tessellation the mean (centre of gravity) and the covariance matrix is calculated and finally for a given point (spectrum) the minimal Mahalanobis distance to all centres of the tessellation is determined. This minimal Mahalanobis distance to all centres of the tessellation is the main parameter to find new, unknown points i.e. spectra. A flowchart of the tessellation scheme is shown in Fig. 3. To minimize 'false alarms' and to optimise the discrimination power two additional conditions have to be fulfilled. The pixel must have a minimal distance to clouds of 3 pixels and only pixels within larger patches are considered.