Artificial Neural Networks and coastal applications

From Coastal Wiki
Jump to: navigation, search

Artificial Neural Networks (ANNs) are powerful prediction tools that establish empirical relationships between predictor variables representing possible influencing factors and a target variable (the output variable) whose value is to be predicted. They consist of a network of nonlinear relationships with coefficients that are empirically determined by minimizing the difference between the predicted and the measured output for a large set of input training data.

ANNs are designed to analyze large data sets representing observations of natural phenomena and possible influencing factors. They can identify and quantify hidden patterns and correlations without prior information about the dynamics of the underlying causes. Once these patterns and correlations are established, the neural network can be used as a prediction tool for unobserved situations. ANNs can also be used to investigate the sensitivity of predictions to different predictor variables and thereby identify the variables that contribute most strongly to predictive performance. The practical importance of neural networks has been strongly stimulated during the past decade by the availability of large data sets and data repositories (see Marine data portals and tools) and by the sharp increase in available computing power.


Background

According to Kingston (2003[1]), the origin of artificial neurons may be traced back to McCulloch and Pitts (1943[2]). Following this work, several researchers started to study interconnected neurons as an information processing system, developing the theory of artificial neural networks. The term 'perceptron' was introduced by Rosenblatt (1958[3]) to refer to an artificial neuron, rather than a natural one. In the following, the term neuron is also used for artificial neuron; other publications sometimes use the term 'node'. A major step in the operationalization of neural networks is due to Rumelhart, Hinton and Williams (1986[4]) who described the backpropagating algorithm that allowed neural networks to adjust efficiently to training data. Many subsequent refinements have contributed to make neural networks very powerful tools for applying knowledge to assist people in almost any task, including checking and streamlining this article.

Neural feedforward network design

Fig. 1. Logistic function (blue) and its derivative (red).

Artificial neural networks are loosely inspired by biological neural networks, in which neurons receive signals from other neurons and transmit signals when sufficiently stimulated. In an artificial neural network, each neuron combines the inputs received from other neurons and transforms them into an output through an activation function. In a feedforward network, this output is passed on to neurons in the next layer.

Sigmoidal output neurons are often modeled by a sigmoidal function [math]f(z)[/math], for example the logistic function (see Fig. 1) [math]\; f(z)= 1 / ( 1+e^{-z}) .\;[/math] They are mainly used when the output represents a probability, for example in classification problems.

For regression, a linear output neuron, [math]Y = z[/math], is usual, so that the output [math]Y[/math] can take any value. The number of inputs and the form of the activation functions [math]f(z)[/math] may vary, and the precise form of the output may vary accordingly.

Fig. 2. Schematic representation of a simple feedforward neural network with 2 inputs (predictor variables) and 2 hidden layers.

The aim of an artificial network is to predict the value [math]Y[/math] of a particular target property knowing the values [math]X_i, \; i=1, .., n[/math] of a set of [math]n[/math] predictor variables representing influencing factors. An artificial neural network therefore consists of the following elements:

  • A set of input data [math]X_i, \; i=1, .., n[/math] provided by the user, each of the input data corresponding to a factor that the output is assumed to depend on or to be related to;
  • A user-defined configuration of one or more interconnected layers of neurons (so-called 'hidden layers') that convert the input they receive into an output according to a user-defined algorithm based on weight parameters;
  • The output [math]Y[/math], the value of the targeted property which is generated by the neural network based on weight parameters determined by previous adjustment to a set of training data.

The first hidden layer receives input from the user-provided data; the next hidden layers get input from other hidden layers. In a typical feedforward network, all neurons of subsequent layers receive all input from all neurons in the preceding layer, see Fig. 2. The output [math]y_k[/math] of a neuron [math]k[/math] is given by

[math]y_k = f_k(z_k), \quad z_k = \sum_{i=0}^{n_k} w_{ki} x_{ki} , \qquad (1)[/math]

where [math]f_k[/math] is the activation function of neuron [math]k[/math], [math]n_k[/math] is the number of inputs [math]x_{ki}, \; i=1, .., n_k[/math] received by neuron [math]k[/math]. The neuron has weight parameters [math]w_{ki}, \; i=0, ..,n_k[/math]. By convention [math]x_{k0}=1[/math] and [math]w_{k0}[/math] is the so-called bias. The neurons in the first layer receive the set of given inputs [math]X_i, \; i=1, ..n[/math]. Connections with a positive weight are called excitatory, the ones with a negative weight are called inhibitory. The activation function transforms the sum nonlinearly into a number which is transmitted as input to the next layer. This feature of the activation function is essential for the capability of the neural network to simulate nonlinear processes.

For a simple network with one hidden layer of [math]n_1[/math] neurons with activation function [math]f[/math], input [math]X_i, \; i=1, .., n[/math] and output [math]Y[/math] we have

[math]Y \equiv y_{\rm out} = z_{\rm out} , \quad z_{\rm out} = w_{{\rm out},0} + \sum_{k=1}^{n_1} w_{{\rm out},k} \, y_k , \quad y_k = f(z_k), \quad z_k = w_{k0} + \sum_{i=1}^n w_{ki} X_i . \qquad (2)[/math]

Training, validation and test data
A crucial condition is that data used to assess the predictive skill have not been used for training. The available dataset is therefore split into three parts: a training set used to determine the weight parameters, a validation set used to choose the network design (number of layers and neurons) and to decide when to stop training, and a test set that is used only once, at the end, to assess the predictive skill of the final network. A split of roughly 60-70% training, 15-20% validation and 15-20% test data is usual. For time series and spatially correlated data, for example in coastal applications, the split should be made in contiguous blocks of time or space rather than by random sampling. Otherwise, successive and strongly correlated observations end up in both the training and the test set, which leads to an overestimate of the predictive skill.

Network training

Gradient descent procedure and backpropagation

The performance of a neural network depends on the values of the [math]n_k+1[/math] weight parameters [math]w_{ki}, \; i=0, .., n_k, \; k=1, .., K[/math] for each of the [math]K[/math] neurons. For a deep network with many inputs and many layers the number of weight parameters can be very large. Network training consists of finding values of the weight parameters from training data. Training data [math]X_{ji}[/math] consist of a set of training conditions for which the outcome [math]\hat{Y_j}[/math] is known. The first index [math]j=1, .., m[/math] indicates one of the [math]m[/math] training examples and the second index [math]i=1, .., n[/math] one of the [math]n[/math] corresponding input parameters. The training process consists of the iterative updating of the weight parameters in order to stepwise reduce the difference between the network output [math]Y_j[/math] and the known training value [math]\hat{Y_j}[/math] for all [math]j=1, ..,m[/math], which we call the error [math]E[/math]. The error is usually defined as the sum over all training examples of the squared deviations between network output and training data (equivalent to maximum likelihood estimation if the deviations [math]\hat{Y_j} - Y_j[/math] are normally distributed),

[math]E = \tfrac12 \sum_{j=1}^m (\hat{Y_j} - Y_j)^2 . \qquad (3)[/math]

The error is smallest when any change in one of the weight parameters [math]w_{ki}[/math] increases the error, implying that at minimum all partial derivatives [math]\partial E / \partial w_{ki}[/math] are zero. Gradient descent searches for such a minimum by repeatedly updating the weights [math]w_{ki}[/math] in the direction opposite to the error gradient, which is the direction of steepest local decrease,

[math]w^{t+1}_{ki} = w^t_{ki} - \eta \; \partial E / \partial w^t_{ki} \; , \qquad (4)[/math]

where [math]w^t_{ki}[/math] is the value of the weight parameter [math]w_{ki}[/math] at iteration step [math]t[/math], [math] \; \partial E / \partial w^t_{ki}[/math] the partial derivative of the error with respect to the weight [math]w_{ki}[/math] at iteration step [math]t[/math] and [math]\eta[/math] is a user-defined hyperparameter that determines the magnitude of the steps towards the minimum. Equation (4) implies that the weight parameters are updated in proportion to their contribution to the error gradient; this minimization procedure is called 'gradient descent'.

To evaluate the partial derivatives in Eq. (4), an additional first index [math]j[/math] is introduced to indicate a particular training example, i.e. a set of input values [math]X_{ji}, \; i=1, .., n[/math] with the corresponding observed training value [math]\hat{Y_j}[/math]. For training example [math]j[/math], Eq. (1) reads

[math]y_{jk} = f_k(z_{jk}), \quad z_{jk} = \sum_{i=0}^{n_k} w_{ki} \, x_{jki} , \quad x_{jk0}=1 , \qquad (5)[/math]

where [math]x_{jki}[/math] is the [math]i[/math]-th input of neuron [math]k[/math]: the input value [math]X_{ji}[/math] for neurons in the first hidden layer, and the output [math]y_{ji}[/math] of neuron [math]i[/math] in the preceding layer for the other neurons. As in Eq. (2), the inputs of a neuron are numbered by the neurons they come from, so that [math]w_{ki}[/math] is the weight given by neuron [math]k[/math] to the output of neuron [math]i[/math]. The derivative of the activation function of neuron [math]k[/math] is denoted by [math]f_k'(z) \equiv df_k/dz[/math].

The error [math]E[/math] (Eq. 4) depends on the weight [math]w_{ki}[/math] only through the sums [math]z_{jk}[/math]. The partial derivatives in Eq. (4) can therefore be written as

[math]\dfrac{\partial E}{\partial w_{ki}} = \sum_{j=1}^m \delta_{jk} \, x_{jki} , \quad \delta_{jk} \equiv \dfrac{\partial E}{\partial z_{jk}} . \qquad (6)[/math]

For the output neuron, with [math]Y_j \equiv y_{j,\rm out}[/math], Eq. (3) gives

[math]\delta_{j,\rm out} = -(\hat{Y_j} - Y_j) \, f_{\rm out}'(z_{j,\rm out}) , \qquad (7)[/math]

which for the linear output neuron of Eq. (2) reduces to [math]\delta_{j,\rm out} = -(\hat{Y_j} - Y_j)[/math]. Hidden neurons have no target value. Their contribution to the error follows from the chain rule, because [math]z_{jk}[/math] influences [math]E[/math] only through the summed inputs [math]z_{jl}[/math] of the neurons [math]l[/math] in the next layer, which receive the output [math]y_{jk}[/math] with weight [math]w_{lk}[/math]:

[math]\delta_{jk} = \sum_l \dfrac{\partial E}{\partial z_{jl}} \, \dfrac{\partial z_{jl}}{\partial z_{jk}} = f_k'(z_{jk}) \sum_l w_{lk} \, \delta_{jl} . \qquad (8)[/math]

Eq. (8) follows from Eq. (5): the summed input [math]z_{jl} = w_{lk} \, y_{jk} + \sum_{i \ne k} w_{li} \, x_{jli}[/math] of neuron [math]l[/math] depends on [math]z_{jk}[/math] only through [math]y_{jk} = f_k(z_{jk})[/math], so that [math]\partial z_{jl} / \partial z_{jk} = w_{lk} \, f_k'(z_{jk})[/math]. For a hidden neuron, the input [math]x_{jki}[/math] in Eq. (6) is the output [math]y_{ji}[/math] of neuron [math]i[/math] in the preceding layer, so that the weight update requires the outputs of the preceding layer and the [math]\delta[/math] values of the next layer:

[math]\dfrac{\partial E}{\partial w_{ki}} = \sum_{j=1}^m y_{ji} \, f_k'(z_{jk}) \sum_l w_{lk} \, \delta_{jl} .[/math]

Starting from the output neuron, Eq. (8) is applied layer by layer back through the network, after which all weights are updated according to Eqs. (4) and (6). This procedure, called backpropagation, yields the gradient for all weights in a single backward pass and is an essential feature of feedforward neural networks.

For large datasets, evaluating the sum over all [math]m[/math] training examples in Eq. (6) at every iteration step is computationally expensive. In practice, the sum is therefore taken over a small, randomly selected subset of the training examples (a mini-batch), with a different subset at each step. This procedure is called stochastic gradient descent; one pass through all training examples is called an epoch, and training usually requires many epochs. The random variation between mini-batches also helps the minimization process to escape from shallow local minima. Most applications use adaptive variants, such as the Adam algorithm[5], in which the step size [math]\eta[/math] in Eq. (4) is adjusted separately for each weight parameter, based on the magnitude of the preceding gradients.

Vanishing gradient issue

According to Eq. (8), each hidden layer that the error passes on its way back from the output contributes a factor [math]f_k'(z_{jk})[/math] to [math]\delta_{jk}[/math]. The values of [math]\delta_{jk}[/math], and hence the weight updates (7), in the first layers of a network with many hidden layers are therefore products of many such factors. Some activation functions are less suitable than others for this reason. A step function to simulate a neuron excitation threshold is impractical because its derivative is zero or not defined. The logistic function (Fig. 1) is also less suitable because its derivative never exceeds [math]f'(0)=\tfrac14[/math]. Along each path through [math]L[/math] hidden layers, the derivatives of the logistic activation functions contribute a factor of at most [math](\tfrac14)^L[/math] to [math]\delta_{jk}[/math], so that, unless this is compensated by large weights, the error is not well propagated back through networks with many layers. Therefore other activation functions are generally used, in particular the so-called rectified linear unit (ReLU) function,

[math]f(z) = \max(0, z) , \qquad (9)[/math]

whose derivative equals 1 for [math]z \gt 0[/math].

Multiple minima issue

The error [math]E[/math] (Eq. 4) is a nonlinear function in the (huge) multidimensional space of all weight parameters (including biases). Through the backpropagation procedure the steepest gradient towards the minimum error in the space of weight parameters is determined at each iteration step. In general however, there is not a single minimum in this multidimensional space, but a large number of valleys at different levels, where the minimization process can get stuck instead of continuing to the lowest minimum. For relatively small networks, as used in many coastal applications, a practical approach is to repeat training with different random initial weights and select the network that performs best on independent validation data. For example, van Maanen et al. (2010) and Bujak et al. (2021) (see below) selected the best of 10,000 networks with different random initial weights. Alternatively, the outputs of several networks with different initial weights can be averaged (an ensemble), which also gives an indication of the prediction uncertainty. In highly overparameterized networks, many different weight configurations may achieve similarly low training errors, so finding the global minimum is generally less important than achieving good predictive performance on independent data (LeCun et al., 2015[6]).

Overfitting issue

By introducing additional neurons the neural network will be able to fit the training data more closely. However, this does not necessarily improve the predictive power of the network and may even reduce the predictive power. The latter occurs when the network output becomes more strongly determined by weak correlations (noise) in the training data than by the stronger ones. Overfitting is detected with the validation set: during training, the error [math]E[/math] for the training set keeps decreasing, whereas the error for the validation set first decreases and then starts to increase. Several strategies are used to prevent overfitting:

  • Early stopping: training is stopped when the validation error starts to increase;
  • Weight decay: a penalty term is added to the error, [math]E_\lambda = E + \tfrac12 \lambda \sum_{k,i} w_{ki}^2[/math], where the user-defined hyperparameter [math]\lambda[/math] determines the strength of the penalty. Large weights, which are needed to fit noise, are thereby suppressed;
  • Dropout: neurons are randomly dropped (along with their connections) from the network during each training step, to prevent neurons from co-adapting too much[7];
  • Noise injection: random noise is added to the input data during training, so that the network becomes insensitive to small input variations;
  • Bayesian neural networks (see below), which reduce overfitting by considering a probability distribution of weights instead of a single optimal set of weights.

Other neural network designs

Feedforward networks are one type of a broad family of artificial neural networks. A few other types are mentioned below.

Fig. 3. Schematic representation of a recurrent neural network.

Recurrent network

Feedforward networks are not well suited for analyzing sequential phenomena where the data at each time step depend on the data at previous time steps. Recurrent networks, i.e. networks with internal memory, have therefore been developed for the analysis of time series data. In a recurrent network, the output of the hidden neurons at each time step is buffered and fed back as additional input to the same neurons at the next time step, so that each new input is processed using the memory of previous steps (Fig. 3). Training is done by backpropagation through the successive time steps (backpropagation through time). The error is then propagated back through as many network layers as there are time steps, and according to Eq. (8) it is multiplied by a factor [math]f_k'(z_{jk})[/math] at each step. Simple recurrent networks therefore suffer from the vanishing gradient problem and cannot learn dependencies over many time steps.

This problem is greatly reduced by the long short-term memory (LSTM) network[8]. Each LSTM neuron contains a memory cell, whose content is passed on from one time step to the next by addition rather than by multiplication with activation function derivatives. Three so-called gates (neurons with logistic activation functions, giving values between 0 and 1) control which information is added to the memory cell, which information is forgotten, and which information is passed on to the output. In this way, information can be retained over hundreds of time steps. LSTM networks are the most widely used recurrent architecture and have been applied, for example, to the prediction of wave heights[9], water levels and shoreline evolution. A simplified variant with fewer gates, the gated recurrent unit (GRU), is also frequently used.

Transformer network

In recurrent networks, information from earlier time steps can only reach the current time step by being passed on step by step. Transformer networks[10] instead use a so-called attention mechanism: for each element of a sequence, the network computes weights that express the relevance of every other element of the sequence, and combines the information of all elements according to these weights. Transformer networks permit more parallel computation than recurrent networks and can efficiently represent long-range dependencies. However, their computational costs can be high for long sequences, and large transformer models generally require substantial training data or pretraining. Transformer networks were developed for language processing and form the basis of today's large language models, but they are increasingly used for other sequence data as well, including environmental time series.

Convolution network

A convolution network is a neural network that contains a subset of neurons sharing the same weights and whose combined receptive fields cover an entire input. By applying the same weights at different locations, convolutional networks can recognize similar local patterns throughout an image. They are therefore particularly suited to identifying spatial features in satellite images (see Satellite-derived shoreline and nearshore bathymetry) and other gridded coastal datasets[6].

Bayesian neural network

Bayesian neural networks are networks with stochastic weights and biases. Bayesian inference is the learning process of finding (inferring) the full probability distribution over all weight parameters in the weight parameter space, the so-called posterior distribution. A Bayesian Neural Network (BNN) is the application of posterior inference for each weight (and bias) in a neural network architecture. This contrasts to the learning process for feedforward networks, i.e. trying to find the optimal weights using optimization through gradient descent. Determining the posterior weight distribution is at the core of Bayesian neural networks. This can only be done in an approximate way, by using techniques that try to mimic the posterior using sampling methods such as Markov Chain Monte Carlo (MCMC), or variational inference. Explanations can be found, for example, in Jospin et al. (2022[11]). Bayesian neural networks can reduce overfitting through prior constraints on the weight distributions and provide estimates of uncertainty associated with imperfectly determined model parameters (epistemic uncertainty). When combined with a probabilistic model of the observations, they can also represent uncertainty arising from inherent variability or measurement noise (aleatoric uncertainty).

Fig. 4. Flowchart of a genetic neural network.

Neural network optimization by genetic algorithms

Genetic algorithms are evolutionary optimization methods that can be used to determine neural network architectures, connection weights or other model parameters. They generate populations of candidate networks and progressively improve them through selection, crossover and mutation (Fig. 4). Unlike gradient descent, genetic optimization does not require derivatives of the error function, although both methods can be combined. To perform these operations, the network specifications are usually encoded in binary strings ("chromosomes"). In each subsequent round of selection, the new best performing networks are selected and in turn used to create the next generation, and so on, until reaching a network with satisfactory performance. Diversity of the population is important for convergence to an optimal solution. The genetic algorithm avoids some problems related to gradient descent by exploring a much wider range of network architectures and a very wide domain of the weight parameter space. However, the final convergence to an optimal solution is slow. Therefore, genetic neural networks are often used with gradient-based training by creating an initial architecture and set of weight parameters that are already close to the optimum.

Although models for artificial neural networks were originally developed to understand the dynamics of brain cells, there are now many applications that use artificial neural networks to analyze other types of data. Artificial neural networks have been applied to various aspects of coastal processes.

Limitations of artificial neural networks

One important limitation of ANNs is their 'black box' nature. ANNs do not explain the causal steps that lead to the results they produce. Their primary aim is to make predictions, not to provide better understanding of the simulated phenomena.

A second important limitation is the low reliability of predictions for situations far beyond the range covered by the input training data. This is similar to the limitation that typically applies to regression models. ANNs are generally not good at predicting situations for which they have not been trained. This limitation is particularly relevant when ANNs trained on historical observations are used to predict coastal responses under changing sea levels, wave climates or sediment supply conditions.

The performances of ANN models are highly dependent on the selection of appropriate predictor variables that represent the most relevant causative factors. Significant efforts are required to determine significant predictor variables of the model because their effects on the output are often not known a priori. Information on the sensitivity of the result to certain predictor variables can be gained by comparing model runs in which different predictor variables are omitted or included. The weights of a trained network provide another indication of the sensitivity to certain predictor variables[12]. For the network of Eq. (2) with one hidden layer, the local sensitivity of the output [math]Y[/math] to an predictor variable [math]X_i[/math] is given by [math]\partial Y/\partial X_i=\sum_k w_{{\rm out},k}f'(z_k)w_{ki}[/math]. This sensitivity depends on the input values through the activation functions and may therefore vary throughout the input domain. An alternative measure of variable importance is the change in predictive performance when an predictor variable is omitted or perturbed. Such measures indicate predictive relevance but do not establish causal influence. This limits the potential of ANNs especially in tasks involving decision-making, as there are no clear explanations of the results obtained[13].

Uncertainty in ANN predictions arises from measurement errors, unresolved variability, limited training data and imperfect model specification. The sensitivity of predictions to the training dataset can be investigated by repeatedly training the network on different subsets of observations. The propagation of known input uncertainties can be estimated by Monte Carlo simulations in which input values are varied according to their uncertainty distributions[14].

Uncertainty may also arise from the particular choice of an ANN and its architecture (the transfer functions between the layers, the number of layers and the number of neurons in each layer). There is no clear methodology for choosing an optimal algorithm. One method consists of trial-and-error simulations with different ANNs in which the number of layers and the number of neurons are varied. Alternative methods include the use of Bayesian neural networks or genetic neural networks to quantify uncertainty and to optimize the network algorithm[15].

Coastal applications of ANN

Artificial neural networks are currently used for analysis and prediction of various coastal processes[16] related to, for example, suspended sediment concentrations[17][18], seabed ripples[19], nearshore sandbank dynamics[20], beach planform [21], coastal erosion[22], salt marsh sedimentation[23], wave forecasting[24] and tidal forecasting[25]. A few typical examples are discussed below.

Littoral drift

Van Maanen et al. (2010[26]) developed an artificial feedforward neural network (ANN) with backpropagation algorithm to predict the depth-integrated alongshore suspended sediment transport rate using 4 predictor variables (water depth, wave height and period, and alongshore velocity). The ANN was trained and validated using a dataset obtained from the intertidal beach of Egmond aan Zee, the Netherlands. 1522 data were used for training and 784 observations for validation. The best result was selected from 10,000 ANNs with different initial random weights. The ANN was shown to outperform the commonly used Bailard formula for littoral drift[27], even when this formula was calibrated. The alongshore component of the velocity, by itself or in combination with other predictor variables, had the largest explanatory power.

Nearshore sandbars

Pape and Ruessink (2011[28]) used linear and nonlinear recurrent networks to study the cross-shore motion of the outer longshore sandbar as a function of wave height. The recurrent network architecture is displayed schematically in Fig. 3. Two study sites with long-term observations were selected, the Gold Coast (GC, Australia) and Hasaki Oceanographic Research Station (HORS, Japan). The datasets covered 5600 daily-observed cross-shore sandbar locations and daily-averaged wave forcings. The time-series of sandbar data were divided into partitions of observed sandbar presence, some partitions being used for training and others for prediction. Five different random initializations of the neural network weights for each training dataset resulted in slightly different optimal values of the weights, thus providing an estimate of the network accuracy. The results showed that many aspects of sandbar behavior, such as rapid offshore migration during storms, slower onshore return during quiet periods, seasonal cycles and annual to interannual onshore-directed trends can be predicted from the wave height with data-driven neural network models. Predictions were more accurate for offshore sandbar migration during high energy conditions than for onshore migration during periods of low waves, suggesting that onshore migration is less strongly correlated with wave height. The results also showed that sandbar migration correlates with wave height over a time scale of several days and not a single day. The linear and nonlinear networks had similar skill, the former performed better for HORS and the latter better for GC. The comparable performance of linear and nonlinear networks illustrates that more complex neural network architectures do not necessarily provide better predictions.

Beach nourishment

Bujak et al. (2021[29]) trained an ANN on shoreline evolution data of 160 beaches on the Croatian coast in order to predict beach nourishment requirements for 68 other Croatian beaches. The predictions were tested against the observed shoreline evolution of these other beaches. To circumvent the issue of convergence to spurious local minima in the optimization process, 10 thousand ANNs were created with random initial weights. The tests revealed a strong correlation (coefficient 0.87) between the observed spatial variability of nourishment volumes and the ANN’s output. The results showed that fetch length was the most important predictor variable for ANN’s prediction ability, apart from the basic information derived from maps such as beach length, beach area and beach orientation.

Extreme sea levels

Bruneau et al. (2020[30]) used the Global Extreme Sea Level Analysis database of quasi-global coastal sea level water information (from around 1070 tide gauges) and an ensemble of hourly physical predictors (wind, wave height and period, precipitation, ..) extracted from the high-resolution atmospheric reanalysis ERA5 of ECMWF to estimate global sea level extremes using neural networks. Each gauge was modelled independently using artificial neural networks (ANNs) composed of 3 hidden layers of 48 neurons. The input layer had 33 neurons (26 predictors plus 7 tide time steps), and the output layer had a single neuron providing the non-tidal residual target. The ANN had about 6400 trainable parameters. An ensemble of 20 ANNs was trained at each gauge location to generate a probabilistic forecast. Each ANN was fitted using 50% of the training set, randomly sampled. The most extreme skew surges were generally underestimated, illustrating the difficulty of predicting rare events from limited observational records. Nevertheless, the ANN ensemble reproduced a substantial part (over 2/3) of the extreme surge signal and outperformed multivariate linear regression.


Related articles

Data analysis techniques for the coastal zone
Linear regression analysis of coastal data
Analysis of coastal processes with Empirical Orthogonal Functions
Wavelet analysis of coastal processes
Data interpolation with Kriging
Random Forest Regression
Support Vector Regression


Further reading

An accessible introduction to neural network methodologies is 'Deep learning' by J.D. Kelleher 2019[31].


References

  1. ↑ Kingston, K.S. 2003. Applications of Complex Adaptive Systems Approaches to Coastal Systems, PhD Thesis, University of Plymouth
  2. ↑ McCulloch, W.S. and Pitts, W.H. 1943. A Logical Calculus of Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics 5: 115-133
  3. ↑ Rosenblatt, F. 1958. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65: 386–408
  4. ↑ Rumelhart, D., Hinton, G. and Williams, R. 1986. Learning representations by back-propagating errors. Nature 323: 533–536
  5. ↑ Kingma, D.P. and Ba, J. 2015. Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego. arXiv:1412.6980
  6. ↑ 6.0 6.1 LeCun, Y., Bengio, Y. and Hinton, G. 2015. Deep learning. Nature 521(7553): 436-444
  7. ↑ Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. 2014. Dropout: a simple way to prevent neural networks from overfitting. J. Machine Learning Res. 15: 1929–1958
  8. ↑ Hochreiter, S. and Schmidhuber, J. 1997. Long short-term memory. Neural Computation 9: 1735-1780
  9. ↑ Fan, S., Xiao, N. and Dong, S. 2020. A novel model to predict significant wave height based on long short-term memory network. Ocean Engineering 205, 107298
  10. ↑ Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L. and Polosukhin, I. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30: 5998-6008
  11. ↑ Jospin, L.V., Laga, H., Boussaid, F., Buntine, W. and Bennamoun, M. 2022. Hands-on Bayesian Neural Networks – a Tutorial for Deep Learning Users. IEEE Computational Intelligence Magazine 17(2): 29–48
  12. ↑ Lee, J.W., Kim, C.G., Lee, J.E., Kim, N. W. and Kim, H. 2020. Medium-Term rainfall forecasts using artificial neural networks with Monte-Carlo cross-validation and aggregation for the Han river basin, Korea. Water 12, 1743
  13. ↑ Borrego, C., Monteiro, A., Ferreira, J., Miranda, A.I., Costa, A.M., Carvalho, A.C. and Lopes, M. 2008. Procedures for estimation of modelling uncertainty in air quality assessment. Environ. Int. 34: 613–620
  14. ↑ Coral, R., Flesch, C.A., Penz, C.A., Roisenberg, M. and Pacheco, A.L.S. 2016. A Monte Carlo-based method for assessing the measurement uncertainty in the training and use of artificial neural networks. Metrol. Meas. Syst. 23: 281–294
  15. ↑ Portillo Juan, N., Matutano, C. and Negro Valdecantos, V. 2023. Uncertainties in the application of artificial neural networks in ocean engineering. Ocean Engineering 284, 115193
  16. ↑ Goldstein, E.B., Coco, G. and Plant, N.G. 2019. A review of machine learning applications to coastal sediment transport and morphodynamics. Earth-Science Reviews 194: 97-108
  17. ↑ Oehler, F., Coco, G., Green, M.O. and Bryan, K.R. 2012. A data-driven approach to predict suspended-sediment reference concentration under non-breaking waves. Continental Shelf Research 46: 96-106
  18. ↑ Yoon, H-D., Cox, D.T. and Kim, M. 2013. Prediction of time-dependent sediment suspension in the surf zone using artificial neural network. Coastal Engineering 71:78–86
  19. ↑ Yan, B., Zhang, Q. and Wai, O.W.H. 2008. Prediction of sand ripple geometry under waves using an artificial neural network. Computers & Geosciences 34: 1655-1664
  20. ↑ Pape, L., Ruessink, B.G., Wiering, M.A. and Turner, I. L. 2007. Recurrent neural network modeling of nearshore sandbar behavior. Neural Networks 20: 509–518
  21. ↑ Iglesias, G., Lopez, I., Carballo, R. and Castro, A. 2009. Headland-bay beach planform and tidal range: a neural network model. Geomorphology 112: 135–143
  22. ↑ Tsekouras, G. E., Rigos, A., Chatzipavlis, A. and Velegrakis, A. 2015. A neural-fuzzy network based on Hermite polynomials to predict the coastal erosion. In Engineering Applications of Neural Networks (pp. 195-205). Springer International Publishing
  23. ↑ Coco, G., Ganthy, F., Sottolichio, A. and Verney, R. 2011. The use of Artificial Neural Networks to predict intertidal sedimentation and unravel vegetation effects. Proceedings of the 7th IAHR Symposium on River, Coastal, and Estuarine Morphodynamics, Beijing, China
  24. ↑ Deo, M.C. and Sridhar Naidu, C. 1999. Real Time Wave Forecasting using Neural Networks. Ocean Engineering 26: 191-203
  25. ↑ Tsai, C.-P. and Lee T.-L. 1999. Back-Propagation Neural Network in Tidal-Level Forecasting. Journal of Waterway, Port, Coastal and Ocean Engineering 125: 195-202
  26. ↑ van Maanen, B., Coco, G., Bryan, K.R. and Ruessink, B.G. 2010. The use of artificial neural networks to analyze and predict alongshore sediment transport. Nonlinear Processes in Geophysics 17: 395-404
  27. ↑ Bailard, J. A. 1981. An energetics total load sediment transport model for a plane sloping beach. J. Geophys. Res. 86: 10938–10954
  28. ↑ Pape, L. and Ruessink, B.G. 2011. Neural-network predictability experiments for nearshore sandbar migration. Continental Shelf Research 31: 1033-1042
  29. ↑ Bujak, D., Bogovac, T., Carevic, D., Ilic, S. and Loncar, G. 2021. Application of Artificial Neural Networks to Predict Beach Nourishment Volume Requirements. J. Mar. Sci. Eng. 9, 786
  30. ↑ Bruneau, N., Polton, J., Williams, J. and Holt, J. 2020. Estimation of global coastal sea level extremes using neural networks. Environ. Res. Lett. 15, 074030
  31. ↑ Kelleher, J.D. 2019. Deep learning. The MIT Press, Cambridge, Massachusetts.


The main author of this article is Job Dronkers
Please note that others may also have edited the contents of this article.
Citation: Job Dronkers (2026): Artificial Neural Networks and coastal applications. Available from http://www.coastalwiki.org/wiki/Artificial_Neural_Networks_and_coastal_applications [accessed on 17-10-2026]
Retrieved from "https://www.coastalwiki.org/w/index.php?title=Artificial_Neural_Networks_and_coastal_applications&oldid=82648"