1 Introduction

In order to increase the degree of autonomy and better ensure navigation safety, unmanned surface vehicle (USVs) should not only be able to acquire their own accurate and reliable navigational data, but to also perceive the surrounding environment to avoid collision risks. Normally, static obstacles, such as small islands and coastlines, can be determined from commercial nautical charts with sufficient accuracy. Detecting dynamic obstacles, such as moving target ships (TS), is a more dynamic challenge. Automatic Identification System (AIS) can provide reasonably accurate navigational data of TSs, and a simple AIS receiver can be powered at similar low-voltage levels that are also adequate for the navigation sensor system of an autonomous USV.

However, the application of AIS may become compromised while the USV is operating at sea encountering with multiple targets. Tracking all the surrounding targets to analyse the collision risks is essential to ensure its safety. Although, an increasing number of vessels are installing AIS devices, only large ships over 300 gross tonnage are required to instal transponders. Small vessels are normally equipped with AIS receivers, so that they would only be aware of other target ship’s information instead of sending their own information at the same time. In addition, AIS is broadcast on VHF radio waves that travel in straight lines. When a USV encounters a complex environment surrounded by multiple target ships, especially in harbour, AIS data are prone to be lost due to the electromagnetic influence. The location of AIS transceivers or the types of the AIS transceivers and weather conditions could also affect the quality of the AIS signal. As a consequence, relying solely on AIS to detect targets is unlikely to prove satisfactory for autonomous USV navigation.

Marine radar has been regarded as a prime solution to perceive the surrounding environment in maritime vessel navigation for many decades. It measures the relative distance and bearing by calculating the transmission time of the echo of an electromagnetic wave pulse. This feature of a marine radar could enable the USV to detect all the targets surrounding the USV within radar detection range, which is typically 48 nautical miles, but associated with a large degree of uncertainty. The target detection can be difficult while using either the AIS or the marine radar alone in harsh environments with an unknown number of targets that varies with time. To improve system reliability, both sensors are employed as complementary devices to perceive the surrounding dynamic environment. A fusion algorithm is therefore required to merge the measurements from the two different sources.

Multitarget tracking presents two main challenges: state estimation and data association. For the former, it has been well regarded that Interactive Multiple Model (IMM) [1] is a highly effective method for estimating manoeuvring target. IMM is a suboptimal recursive filter and its ability to adaptively switch between different kinematic models using Markovian coefficient appeals to many practical tracking scenarios. More recently, Variable Structure Multiple Model (VSMM) was introduced in [2] as an evolution of the IMM. Development of the VSMM originated from IMM’s degrading performance with increasing number of models. Provided a set of possible models, VSMM is able to select admissible models suitable for current event. Such adaptable system offers better computational expense compared to IMM only when large bank of models is required. In terms of accuracy, VSMM at this point indicates no significant superiority to IMM. [3,4,5,6].

Data association problem corresponds to correctly identifying multiple measurements to its target. Poor match between a measurement and its target will in turn lead to poor estimation. Nearest neighbour (NN) was initially proposed to tackle this problem as demonstrated in [7, 8], but was soon replaced with more robust methodologies especially in highly cluttered environments. Widely recognised probabilistic approaches include Multiple Hypothesis Tracking (MHT) and Joint Probabilistic Data Association (JPDA) where various probable association hypotheses are considered instead of direct individual assignment [9,10,11]. By spending more computational effort, these algorithms perform reliably even when observations are likely to agree onto more than one target.

Many combinations of state estimation and data association algorithms have been put forth to yield robust multitarget tracking. IMM-JPDA, introduced by Y. Bar-Shalom et al. in [12], has been broadly accepted due to the blending of IMM’s renowned performance and JPDA’s dependable association in cluttered environments; e.g. see [13,14,15,16,17] for further work in IMM-JPDA. Monte Carlo methods, also known as particle filters, is another technique gaining attention in multitarget tracking [3, 4, 18,19,20]. But despite their proven results, algorithms mentioned are computationally demanding and impose unaffordable compromise in pragmatic applications.

Relevant studies on maritime situational awareness by Braca et al. [21, 22] proposes using UKF-JPDA to sense surrounding objects and successfully presented satisfactory multitarget tracking in cluttered environments. This maritime-oriented work was then extended further in [23] and [24, 25], where radar-based IMM-JPDA and VSMM-JPDA are employed in multitarget scenario, respectively, a substantial improvement in the state estimation aspect. However, data association aspect here still indicates room for improvements in terms of reducing computational expense by using alternative techniques. Additionally, JPDA-based tracking tends to merge tracks together when separation between objects are close [9]. Simultaneously overlapping target estimation is unwelcome as it would represent unrealistic behaviour of an operating surface vehicles in close proximity. Another related work was also done by Liu et al. [26] in using IMM with AIS for ship tracking. Although it complements UKF-JPDA’s limitation in state estimation by using multiple model system, the work was focused on single-target tracking using AIS.

This paper proposes a master fusion algorithm using radar and AIS in multitarget tracking for USV purposes. Radar tracking system within the master fusion consists of a linear Kalman filter (KF) and IMM to represent different manoeuvring modes. Predictions made is then paired with received measurements by a two-stage fuzzy logic association method. Dissimilar to work in [27], proposed fuzzy system is designed to completely disregard JPDA to dramatically enhance processing speed. Regular updates from AIS allows occasional adjustments to radar-based estimation while also keeping targets unequipped with AIS to be tracked. The highlights of this paper can be summarised as:

  1. 1.

    By using a Chi square based multi-factor manoeuvre detector, a new interacting multi-model Kalman filter (IMMKF) has been designed. This can greatly deal with the task of efficiently tracking target ships that have complex motion models, which will facilitate a real-time autonomous navigation capability;

  2. 2.

    With the help of the new two-stage fuzzy logic data association method, an accurate data correlation process can be achieved, which enables a rapid multi-sensor data fusion functionality;

  3. 3.

    By employing the proposed IMMKF as the underlying filtering method, the developed multi-sensor data fusion algorithm can provide a reliable tracking performance of multiple target ships in maritime environments.

  4. 4.

    The proposed multi-sensor data fusion algorithm enables ships to be equipped with multiple sensors, which improves the robustness of the navigation systems especially when some sensors encounter malfunctions.

The organisation of the rest paper is as follows. Section 2 describes the detection and prediction of moving target using constant velocity model, which is the fundamental for multi-model-based manoeuvring detection and prediction that has been introduced in Sect. 3. Section 4 specifically explains the proposed multi-sensor data fusion algorithm and Sect. 5 provides the simulation results. Section 6 concludes the paper and provides future work.

2 Moving Target Detection and Prediction Using Constant Velocity Model

Prior to the consideration of a complex maritime environment, this section focuses on detecting and predicting the navigational data of a single target ship that is equipped with an AIS transponder to broadcast its own navigational data, i.e. position, speed over ground (SOG) and course over ground (COG). In general, a vessel over the sea is not designed for both rapid and precise manoeuvring and its operation is associated with constant velocity and course unless manoeuvring is required to eliminate collision risks or correct drifted trajectory. The speed or course changing is often slow to maintain the vessel steady. Therefore, a constant velocity (CV) model can be used to describe the state of the target [28]. The state vector is defined to include essential navigational data to assess the collision risk between the target and USV:

$$ \varvec{x}_{T} = \left[ {p_{Tx} \,\,p_{Ty} \,\,v_{Tx} \,\,v_{Ty} \,\,\varphi } \right]^{\text{T}} , $$
(1)

where \( p_{Tx} \) and \( p_{Ty} \) represent the target’s positions, \( v_{Tx} \) and \( v_{Ty} \) are the target’s SOG in the x and y directions, and \( \varphi \) is the COG of the target ship. The system state equation based on a constant velocity model is denoted as below:

$$ \varvec{x}_{T} \left( k \right) = \left[ {\begin{array}{*{20}c} 1 & 0 & T & 0 & 0 \\ 0 & 1 & 0 & T & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right]\varvec{x}_{T} \left( {k - 1} \right) + \varvec{w}_{T} \left( {k - 1} \right). $$
(2)

The observations are provided by the dynamic t6 AIS messages, which give the absolute positions, SOG and COG of the detected target. Therefore, the system measurement model can be defined as:

$$ \varvec{z}_{A} \left( k \right) = \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \\ \end{array} } \right] \varvec{x}_{T} \left( k \right) +\varvec{\nu}_{T} \left( k \right). $$
(3)

The Kalman filter (Eqs. 4 to 9) is then applied to reduce AIS signal noise and provide predicted navigational data during long AIS data-transmitting intervals. As shown in Fig. 1, the algorithm first takes the prior states including target’s position, SOG and COG to make predictions of the navigational data for the next time step using Eqs. 4 and 5. It then calls the system to check whether there is an updated AIS message. If so, the system will enter the estimation stage using the updated data to correct the predicted target’s navigational data by Eqs. 6 to 8. Otherwise, the system will output the predicted navigational data and use it as the next state to enter the next prediction–estimation process.

Fig. 1
figure 1

AIS data pre-process prediction and estimation

Full size image

The predicted state of the target’s navigational data is computed by Eq. 4 using the constant velocity model and the predicted error covariance \( \varvec{P}_{T}^{ - } \) is defined in Eq. 5, where \( \varvec{Q}_{T} \) is the processing error covariance:

$$ \widehat{\varvec{x}}_{T}^{ - } \left( k \right) = \left[ {\begin{array}{*{20}c} 1 & 0 & T & 0 & 0 \\ 0 & 1 & 0 & T & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right] \widehat{\varvec{x}}_{T} \left( {k - 1} \right), $$
(4)
$$ \varvec{P}_{T}^{ - } \left( k \right) = \left[ {\begin{array}{*{20}c} 1 & 0 & T & 0 & 0 \\ 0 & 1 & 0 & T & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right] \varvec{P}_{T} \left( {k - 1} \right) \left[ {\begin{array}{*{20}c} 1 & 0 & T & 0 & 0 \\ 0 & 1 & 0 & T & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right]^{\text{T}} + \varvec{Q}_{T} . $$
(5)

The Kalman filter gain \( \varvec{K}_{T} \) to correct the prior target’s navigational data by reducing the mean square error is computed by Eqs. 6 and 7:

$$ \varvec{K}_{T} \left( k \right) = \varvec{P}_{T}^{ - } \left( k \right) \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \\ \end{array} } \right]^{\text{T}} \varvec{S}_{T} \left( k \right)^{ - 1} , $$
(6)
$$ \varvec{S}_{T} \left( k \right) = \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \\ \end{array} } \right] \varvec{P}_{T}^{ - } \left( k \right) \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \\ \end{array} } \right]^{\text{T}} + \varvec{R}_{T} . $$
(7)

As demonstrated in Fig. 1, if there is an updated AIS message with the observation \( \varvec{z}_{A} \left( k \right) \), the system state \( \widehat{\varvec{x}}_{T} \left( k \right) \) can be computed by applying the calculated Kalman filter gain \( \varvec{K}_{T} \) to the prior target’s navigational data as shown in Eqs. 8 and 9. If there is no updated AIS message, the predicted system state \( \widehat{\varvec{x}}_{T}^{ - } \left( k \right) \) will be treated as the current state of the target to assess the collision risk:

$$ \widehat{\varvec{x}}_{T} \left( k \right) = \widehat{\varvec{x}}_{T}^{ - } \left( k \right) + \varvec{K}_{T} \left( k \right) \left[ {\varvec{z}_{A} \left( k \right) - \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \\ \end{array} } \right] \widehat{\varvec{x}}_{T}^{ - } \left( k \right)} \right], $$
(8)
$$ \varvec{P}_{T} \left( k \right) = \left( {I - \varvec{K}_{T} \left( k \right) \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \\ \end{array} } \right]} \right) \varvec{P}_{T}^{ - } \left( k \right). $$
(9)

3 Multi-model-Based Manoeuvring Target Detection and Prediction

In a maritime environment, although a vessel conducts the mission to adhere to a straight-line trajectory at a constant speed, the influences caused by water currents and winds would alter its trajectory. The vessel normally makes manoeuvres to correct its course to its destination or the next waypoint. Thus, the system state equations, based on a constant velocity model are inaccurate and would generate inaccurate predictions when the target is manoeuvring. Therefore, multiple models have been integrated into the system to describe the target’s motions more veraciously to provide more accurate detection and prediction results.

3.1 Interacting Multi-model (IMM)-Based Target Detection

Interacting multi-model (IMM) filtering has been widely used in manoeuvring target detection. According to the International Maritime Organization (IMO), 2002, vessels maintain as steady as possible while operating over the sea. Turning at constant angular velocity is a common manoeuvre of vessels. Therefore, a coordinate turn (CT) model is normally used to model the target’s manoeuvre. The transition matrix of a CT model is expressed in Eq. (10) and the system dynamic equations are demonstrated in Eq. (11):

$$ \varvec{A}_{\text{CT}} = \left[ {\begin{array}{*{20}c} 1 & 0 & {\frac{\sin \omega T}{\omega }} & {\frac{{ - \left( {1 - \cos \omega T} \right)}}{\omega }} \\ 0 & 1 & {\frac{1 - \cos \omega T}{\omega }} & {\frac{\sin \omega T}{\omega }} \\ 0 & 0 & {\cos \omega T} & { - \sin \omega T} \\ 0 & 0 & {\sin \omega T} & {\cos \omega T} \\ \end{array} } \right], $$
(10)
$$ f\left( {x\left( k \right)} \right) = \left\{ {\begin{array}{*{20}l} {p_{x} \left( {k - 1} \right) + \frac{{\sin \left( {\omega \left( {k - 1} \right)T} \right)}}{{\omega \left( {k - 1} \right)}} \times v_{x} \left( {k - 1} \right) - \frac{{\left( {1 - \cos \left( {\omega \left( {k - 1} \right)T} \right)} \right)}}{{\omega \left( {k - 1} \right)}} \times v_{y} \left( {k - 1} \right)} \hfill \\ {p_{y} \left( {k - 1} \right) + \frac{{1 - \cos \omega \left( {\left( {k - 1} \right)T} \right)}}{{\omega \left( {k - 1} \right)}} \times v_{x} \left( {k - 1} \right) + \frac{{\sin \left( {\omega \left( {k - 1} \right)T} \right)}}{{\omega \left( {k - 1} \right)}} \times v_{y} \left( {k - 1} \right)} \hfill \\ {\cos \left( {\omega \left( {k - 1} \right)T} \right) \times v_{x} \left( {k - 1} \right) - \sin \left( {\omega \left( {k - 1} \right)T} \right) \times v_{y} \left( {k - 1} \right)} \hfill \\ {\sin \left( {\omega \left( {k - 1} \right)T} \right) \times v_{x} \left( {k - 1} \right) + \cos \left( {\omega \left( {k - 1} \right)T} \right) \times v_{y} \left( {k - 1} \right)} \hfill \\ {\theta \left( {k - 1} \right) + \omega \left( {k - 1} \right)T.} \hfill \\ \end{array} } \right. $$
(11)

The state of the target ship can be predicted if the angular velocity is known. However, AIS cannot provide the measurement of the target’s angular velocity. Therefore, the angular velocity should be considered as a parameter rather than a variable to generate multiple models and an interacting multiple model estimator has been integrated to the KF-based target detection and prediction algorithm to model the target’s manoeuvres. The system state equation of the CT model can be then defined in Eq. 12:

$$ \varvec{x}_{T} \left( k \right) = \left[ {\begin{array}{*{20}c} 1 & 0 & {\frac{\sin \omega T}{\omega }} & {\frac{{ - \left( {1 - \cos \omega T} \right)}}{\omega }} & 0 \\ 0 & 1 & {\frac{1 - \cos \omega T}{\omega }} & {\frac{\sin \omega T}{\omega }} & 0 \\ 0 & 0 & {\cos \omega T} & { - \sin \omega T} & 0 \\ 0 & 0 & {\sin \omega T} & {\cos \omega T} & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right]\varvec{x}_{T} \left( {k - 1} \right) + \left[ {\begin{array}{*{20}c} 0 \\ 0 \\ 0 \\ 0 \\ {\omega T} \\ \end{array} } \right] + \varvec{w}_{T} \left( {k - 1} \right). $$
(12)

The interacting multiple model Kalman filter (IMMKF) has been proposed to calculate the possibilities of each of the predefined models and generate the fused navigational data accordingly. First, a set of fixed values of the angular velocities (\( \omega 1, \omega 2, \omega 3, \ldots ,\omega j) \) are defined to generate different coordinate turn models \( {\text{CT}}1, {\text{CT}}2,{\text{CT}}3, \ldots {\text{CT}}j \) as \( M \) using Eq. 12:

$$ M = \left\{ {{\text{CT}}1\left( {\omega 1} \right), {\text{CT}}2\left( {\omega 2} \right),{\text{CT}}3\left( {\omega 3} \right), \ldots {\text{CT}}j\left( {\omega j} \right)} \right\}. $$
(13)

The model at each time step \( k \) can be expressed as:

$$ m_{j} \left( k \right) \triangleq \left\{ {M\left( k \right) = m_{j} } \right\}. $$
(14)

Then the predicted probability \( \mu_{j}^{ - } \) of each model at time step \( k \) can be computed as in Eq. 15:

$$ \mu_{j}^{ - } \left( k \right) \triangleq P\left\{ {m_{j} \left( k \right)|z\left( {k - 1} \right)} \right\} = \mathop \sum \limits_{i} \pi_{ij} \mu_{i} \left( {k - 1} \right). $$
(15)

The probabilities are then used to generate each model’s mean \( \widehat{x}_{0j} \) and the spread of the means \( X_{j} \) and calculate and covariance \( P_{0j} \) of each model by Eqs. 16 to 19:

$$ \mu_{i|j} \triangleq P\left\{ {m_{i} \left( {k - 1} \right) |m_{j} \left( k \right),z\left( {k - 1} \right)} \right\} = \frac{{\pi_{ij} \mu_{i} \left( {k - 1} \right)}}{{\mu_{j}^{ - } }}, $$
(16)
$$ \widehat{\varvec{x}}_{0j} \left( {k - 1} \right) \triangleq E\left[ {x\left( {k - 1} \right)|m_{j} \left( k \right),z\left( {k - 1} \right)} \right] = \mathop \sum \limits_{i} \widehat{\varvec{x}}_{i} \left( {k - 1} \right)\mu_{i|j} , $$
(17)
$$ X_{j} \triangleq \mathop \sum \limits_{i} \left[ {\widehat{\varvec{x}}_{i} \left( {k - 1} \right) - \widehat{\varvec{x}}_{0j} \left( {k - 1} \right)} \right]\left[ {\widehat{\varvec{x}}_{i} \left( {k - 1} \right) - \widehat{\varvec{x}}_{0j} \left( {k - 1} \right)} \right]^{\text{T}} \mu_{i|j} , $$
(18)
$$ P_{0j} \left( {k - 1} \right) = \mathop \sum \limits_{i} P_{i} \left( {k - 1} \right)\mu_{i|j} + X_{j} . $$
(19)

The predicted mean of system state \( \widehat{\varvec{x}}_{j}^{ - } \) and covariance \( P_{j}^{ - } \) are computed using Eqs. 20 and 21:

$$ \widehat{\varvec{x}}_{j}^{ - } \left( k \right) = A_{CTj} \left( {k - 1} \right)\widehat{\varvec{x}}_{0j} \left( {k - 1} \right), $$
(20)
$$ P_{j}^{ - } \left( k \right) = A_{j} \left( {k - 1} \right)P_{0j} \left( {k - 1} \right)A_{j} \left( {k - 1} \right)^{'} + Q_{j} \left( {k - 1} \right). $$
(21)

The measurement residual of each model is calculated as Eq. 22 and gives the covariance of the residual in Eq. 23:

$$ \nu_{j} \left( k \right) = z\left( k \right) - H_{j} \widehat{\varvec{x}}_{i}^{ - } \left( k \right), $$
(22)
$$ S_{j} \left( k \right) = H_{j} P_{j}^{ - } \left( k \right)H_{j}^{'} + R_{j} . $$
(23)

The Kalman filter gain can then be computed and the estimated state vectors \( \widehat{x}_{j} \) and error covariance \( P_{j} \) of each model are obtained from Eqs. 24 to 28:

$$ K_{j} \left( k \right) = P_{j}^{ - } \left( k \right)H_{j}^{'} S_{j} \left( k \right)^{ - 1} , $$
(24)
$$ \widehat{\varvec{x}}_{j} \left( k \right) = \widehat{\varvec{x}}_{j}^{ - } \left( k \right) + K_{j} \left( k \right)\nu_{j} \left( k \right), $$
(25)
$$ P_{j} \left( k \right) = P_{j}^{ - } \left( k \right) - K_{j} \left( k \right)S_{j} \left( k \right)K_{j} \left( k \right)^{'} , $$
(26)
$$ L_{j} = {\mathcal{N}}\left[ {\nu_{j} ;0,S_{j} } \right] = \frac{1}{{\sqrt {2\pi S_{j} } }} \times { \exp }\left( { - \frac{1}{2}\nu_{j}^{T} S_{j}^{ - 1} \nu_{j} } \right), $$
(27)
$$ \mu_{j} = \frac{{\mu_{j}^{ - } L_{j} }}{{\mathop \sum \nolimits_{i} \mu_{i}^{ - } L_{i} }}. $$
(28)

The final estimation of the state vector and error covariance can be computed by combining all the data from each model based on its probability:

$$ \widehat{\varvec{x}}\left( k \right) \triangleq E\left[ {x\left( k \right)|z\left( k \right)} \right] = \mathop \sum \limits_{j} \widehat{\varvec{x}}_{j} \left( k \right)\mu_{j} , $$
(29)
$$ P\left( k \right) \,\triangleq \,E\left[ {\left( {x\left( k \right) - \widehat{\varvec{x}}\left( k \right)} \right)\left( {x\left( k \right) - \widehat{\varvec{x}}\left( k \right)} \right)^{'} |z\left( k \right)} \right] = \mathop \sum \limits_{j} P_{j} \left( k \right)\mu_{j} + X, $$
(30)
$$ X \triangleq \mathop \sum \limits_{i} \left( {\widehat{\varvec{x}}_{i} \left( k \right) - \widehat{\varvec{x}}\left( k \right)} \right)\left( {\widehat{\varvec{x}}_{i} \left( k \right) - \widehat{\varvec{x}}\left( k \right)} \right)^{'} \mu_{i} . $$
(31)

This process is repeated in each integration of the Kalman filter-based target tracking algorithm and the most probable model is determined to express the target’s manoeuvre.

3.2 Chi Square-Based Multi-factor Manoeuvre Detector

The computational cost of multiple models becomes high with the increasing number of the models, which introduces a degree of non-practicability to real-time systems. Manoeuvres represent a change in the target motion pattern, therefore detecting the manoeuvre of the target first offers a solution to reduce such computational cost since the multiple model-based data fusion algorithm will only be employed when manoeuvring of the target is detected. Chi square-based detectors are widely used in manoeuvring target detection [29]. For an \( n \)-dimensional Gaussian distributed vector \( x\sim {\mathcal{N}}\left( {\widehat{\varvec{x}}, P} \right) \), its covariance is Chi square distributed. Therefore, the proposed detector employs the covariance of system residuals in the proposed IMMKF target detection and prediction algorithm to compare with the Chi square-defined thresholds (Eqs. 32 to 33). The thresholds are listed in Table 1 (Lancaster, 1965), where \( \alpha \) is the probability and \( 1 - \alpha \) is the level of confidence, which is typically set at 95% or 99.5% by the system. The detector identifies whether the target is making manoeuvres by Eq. 34. This procedure will save a significant amount of the computational cost generated by the multiple model filter:

Table 1 Chi square distribution \( \chi_{n}^{2} \)
Full size table
$$ \varvec{\varepsilon}\left( k \right) = \varvec{z}\left( k \right) - \varvec{H}\widehat{\varvec{x}}\left( k \right), $$
(32)
$$ dc\left( k \right) = \left| {\varvec{cov}\left( {\varvec{\varepsilon}\left( k \right)} \right)} \right| = \varvec{\varepsilon^{\prime}}\left( k \right) \varvec{S}_{T} \left( k \right)^{ - 1}\varvec{\varepsilon}\left( k \right), $$
(33)
$$ dc\left( k \right) > \eta = \chi_{n}^{2} \left( \alpha \right). $$
(34)

Once the target is detected as manoeuvring, the above interacting multiple model algorithm is applied to determine the system states.

4 Multi-sensor Data Fusion for Moving Target Detection and Tracking

Most of the current studies on radar and AIS data fusion are concerned with synchronising, associating and fusing the different measurements from each. In this research, raw radar and AIS measurements will not be associated and fused directly. They will be associated with each detected target track individually. The system states are then updated by the proposed manoeuvring target detection and prediction algorithm from the previous sections using the associated sensor measurements, respectively, and the final fusion algorithm generates the estimated target’s navigational data by fusing the updated estimations. The system structure is demonstrated in Fig. 2.

Fig. 2
figure 2

System structure of multitarget detection using AIS and radar measurements

Full size image

4.1 Multi-sensor Data Association Algorithm

Generally, a complete marine radar system comes with an automatic radar plotting aid (ARPA) to provide a visual map for the mariner to identify the surrounding target ships. Most of the NMEA 0183-supported radar systems also generate NMEA0183 sentences to provide the information of the detected targets, which can be extracted by using the correct converter. In this research, the measurements obtained from NMEA0183 sentences are used. The main data obtained from a marine radar are the dynamic information of the target, such as the relative distance and bearing between radar platform and the target, as well as target’s true speed and course.

After obtaining raw sensor data, a data association algorithm is then required for the autonomous system to determine the number of targets and allocate each sensor measurement to the related target. In a real-time target tracking system, the data collected from sensors should have some similar physical characteristics to the related targets. Therefore, the data can be associated according to the designed rules that express their similarities. Figure 3 gives a simple approach of the data association using the nearest neighbours. At each fusion time step, the green circle denotes the predicted target generated by the IMMKF algorithm and the orange star represents a sensor measured target (AIS or radar). The sector formed within the dashed line gives the thresholds of both the position and bearing of the target. If both targets are inside the threshold, the sensor measured target can be treated as related to the predicted target.

Fig. 3
figure 3

Target validation: measured target and predicted target

Full size image

However, such a simple approach is not efficient and may generate error correlations when the number of targets increases. In this study, a two-stage multi-factor fuzzy integration decision-making algorithm has been proposed to associate measurements from AIS and radar with detected targets indirectly for real-time multiple target tracking with the intention of reducing computational time. As mentioned before, a marine radar can provide the relative range, relative bearing, course and speed of the target while AIS provides absolute position in latitude/longitude, course and speed. With the knowledge of the USV’s own absolute position, the relative range and bearing can also be calculated from AIS measurements. Therefore, the four characters from radar and AIS measurements can be compared with the detected targets to determine whether the measurements are related to the same target. As shown in Fig. 4, at the first stage, the differences in the relative range and bearing to the USV between the sensor target and system-predicted target are evaluated by the fuzzy decision-making system to determine whether the target detected by the sensor is in a similar location to that of the system-predicted target. However, it is yet to make a decision whether the two targets are related at this stage, although the opposite fact that the sensor target is related to a different target is obvious if the differences in the range and bearing are large. The second stage to compare the course and speed of the two targets will be enabled if the system requires further evaluation to make a final decision. Instead of inputting all the four characters of all sensor measurements, the proposed algorithm uses a two-stage structure that is able to reduce the computational cost significantly, especially in an environment with a large number of targets.

Fig. 4
figure 4

Two-stage fuzzy multi-factor integration data association algorithm

Full size image

Assume there are \( i \) measurements obtained by a sensor, denoted as \( {\text{SE}}\left( i \right) \), and \( j \) system-predicted targets, denoted as \( {\text{TS}}^{ - } \left( j \right) \). The fuzzy set at the first stage is defined as the respective differences between the two targets in the relative range \( \varvec{\delta}_{R} \) and bearing \( \varvec{\delta}_{B} \) to the USV:

$$ \varvec{\delta}_{1} \left( {i,j} \right) = \left[ {\begin{array}{*{20}c} {\varvec{\delta}_{R} } \\ {\varvec{\delta}_{B} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {\left| {R_{\text{SE}} \left( i \right) - R_{\text{TS}} \left( j \right)} \right|} \\ {\left| {B_{\text{SE}} \left( i \right) - B_{\text{TS}} \left( j \right)} \right|} \\ \end{array} } \right], $$
(35)

where \( R_{\text{SE}} \) and \( B_{\text{SE}} \) are the relative range and bearing obtained from the sensor measurements \( {\text{SE}}\left( i \right) \); \( R_{\text{TS}} \) and \( B_{\text{TS}} \), are from the system-predicted states \( {\text{TS}}^{ - } \left( j \right) \). A Gaussian membership function is employed to compute the correlation grade of each input:

$$ g_{1} \left( {i,j} \right) = \left[ {\begin{array}{*{20}c} {g_{R} \left( {i,j} \right)} \\ {g_{B} \left( {i,j} \right)} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {\exp \left( { - \tau_{R}\varvec{\delta}_{R}^{2} \left( {i,j} \right)/\sigma_{R}^{2} } \right)} \\ {\exp \left( { - \tau_{B}\varvec{\delta}_{B}^{2} \left( {i,j} \right)/\sigma_{B}^{2} } \right)} \\ \end{array} } \right], $$
(36)

where \( \tau_{R} \) and \( \tau_{B} \) are the predefined adjustment coefficients, \( \sigma_{R} \) and \( \sigma_{B} \) are the related sensor measurement errors that can be obtained from sensor specifications.

The integrated association grade \( G_{1} \left( {i,j} \right) \) can then be computed by distributing the weight to each correlation grade of each character:

$$ G_{1} \left( {i,j} \right) = \left[ {\begin{array}{*{20}c} {w_{R} } & {w_{B} } \\ \end{array} } \right] \times \left[ {\begin{array}{*{20}c} {g_{R} \left( {i,j} \right)} \\ {g_{B} \left( {i,j} \right)} \\ \end{array} } \right]. $$
(37)

A threshold is then designed and the initial decision as to whether the two targets \( {\text{SE}}\left( i \right) \) and \( {\text{TS}}^{ - } \left( j \right) \) are correlated can be made by comparing the integrated association grade \( G_{1} \left( {i,j} \right) \) to the designed threshold according to the following rules:

  • If \( G_{1} \left( {i,j} \right) \ge {\text{threshold}} \), the two targets \( {\text{SE}}\left( i \right) \) and \( {\text{TS}}^{ - } \left( j \right) \) are related in the similar position and the second stage enables;

  • If \( G_{1} \left( {i,j} \right) < {\text{threshold}} \), the two targets \( {\text{SE}}\left( i \right) \) and \( {\text{TS}}^{ - } \left( j \right) \) are different.

4.2 Multi-sensor Target Detection and Tracking Algorithm

In order to detect multiple targets in a maritime environment, moving tracks that are associated to each target are formed to determine each target’s real-time positions. Unlike the AIS, the sampling time of a marine radar is fixed. It is about 1.25 s to 2.5 s as the rotation rate of its antenna is normally 24 or 48 rpm (round per minute). The sampling time of the radar is used as the system’s sampling time. The proposed target detection and prediction algorithm based on the IMMKF with manoeuvre detector is used to form the tracks of each target. Therefore, the state vector of each target is defined as following:

$$ {\mathbf{TS}}_{m} = \left[ {p_{xm} p_{ym} v_{xm} v_{ym} \varphi_{m} } \right]^{T} , $$
(38)

where \( m \) is the number of detected targets.

When the target is operating at a constant speed without manoeuvring, its motion model is

$$ {\mathbf{TS}}_{\varvec{m}} \left( k \right) = \left[ {\begin{array}{*{20}c} 1 & 0 & T & 0 & 0 \\ 0 & 1 & 0 & T & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right]{\mathbf{TS}}_{\varvec{m}} \left( {k - 1} \right) + \varvec{w}\left( {k - 1} \right). $$
(39)

When its manoeuvre is detected, the motion model of the target based on the coordinate turn model is described in Eq. 40:

$$ {\mathbf{TS}}_{\varvec{m}} \left( k \right) = \left[ {\begin{array}{*{20}c} 1 & 0 & {\frac{\sin \omega T}{\omega }} & {\frac{{ - \left( {1 - \cos \omega T} \right)}}{\omega }} & 0 \\ 0 & 1 & {\frac{1 - \cos \omega T}{\omega }} & {\frac{\sin \omega T}{\omega }} & 0 \\ 0 & 0 & {\cos \omega T} & { - \sin \omega T} & 0 \\ 0 & 0 & {\sin \omega T} & {\cos \omega T} & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} } \right]{\mathbf{TS}}_{\varvec{m}} \left( {k - 1} \right) + \left[ {\begin{array}{*{20}c} 0 \\ 0 \\ 0 \\ 0 \\ {\omega T} \\ \end{array} } \right] + \varvec{w}\left( {k - 1} \right). $$
(40)

The radar measurements are in a polar frame and have to be converted to a Cartesian frame. A debiased conversion algorithm [30] has been employed to compensate for errors that might occur during the conversion as below:

$$ p_{Rxm} = r_{m} \times cos\theta_{m} - \mu_{xm} , $$
(41)
$$ p_{Rym} = r_{m} \times sin\theta_{m} - \mu_{ym} , $$
(42)
$$ \mu_{xm} = E\left[ {\widehat{p}_{x} |r_{m} ,\theta_{m} } \right] = \left[ {r_{m} \times cos\theta_{m} \times \left( {e^{{ - \sigma_{{\theta_{m} }}^{2} }} } \right) - e^{{ - \sigma_{{\theta_{m} }}^{2} /2}} } \right], $$
(43)
$$ \mu_{ym} = E\left[ {\widehat{p}_{y} |r_{m} ,\theta_{m} } \right] = \left[ {r_{m} \times sin\theta_{m} \times \left( {e^{{ - \sigma_{{\theta_{m} }}^{2} }} } \right) - e^{{ - \sigma_{{\theta_{m} }}^{2} /2}} } \right]. $$
(44)

where \( p_{Rxm} , p_{Rym} \) are the position coordinates of the target; \( r_{m} \) is the range from the \( m \) th target to the USV; \( \theta_{m} \) is the bearing of the target;\( \mu_{xm} , \mu_{ym} \) are the estimated bias that will be removed during conversion.

The measurements obtained by radar are converted to the following:

$$ \varvec{z}_{\text{TSRm}} = \left[ {p_{Rxm} p_{Rym} v_{Rxm} v_{Rym} \varphi_{Rm} } \right]^{\text{T}} . $$
(45)

And, the measurements obtained by AIS are expressed as Eq. 46:

$$ \varvec{z}_{{{\text{TSA}}m}} = \left[ {p_{Axm} p_{Aym} v_{Axm} v_{Aym} \varphi_{Rm} } \right]^{\text{T}} . $$
(46)

Unlike the single target detection, all the measurements and predictions are associated from a known target, so that they can be used to form a moving track of the target directly. For a multiple targets problem, the proposed data fusion algorithm at each time step should first determine the number of the targets and their relationships to those detected targets from the previous time step. The following flowchart demonstrates the whole target tracks formation and association and multi-sensor data fusion process. The system first predicts the next state of each of the detected targets that are associated with \( m \) tracks from last time step \( k \) using system state models. The obtained radar measurements are then investigated to determine how many targets (\( j \)) are detected at this time step \( k + 1 \). The predictions of each detected target \( {\text{TS}}_{m}^{ - } \left( {k + 1} \right) \) are compared with the radar measurements \( R_{j} \left( {k + 1} \right) \) using the proposed two-stage fuzzy association decision-making algorithm to associate the radar measurements with the known targets’ tracks. If \( m < j \), then the radar detects a new target, a new track is then formed that makes \( m = j \). The targets’ tracks can then be updated by the proposed data fusion algorithm to obtain radar estimations \( \widehat{\text{TS}}_{\text{RM}} \left( {k + 1} \right) \). The system then calls AIS measurements to check whether there is an update. If not, the system will make a new prediction \( {\text{TS}}_{\text{Am}}^{ - } \left( {k + 1} \right) \) based on the last AIS estimation that is also used as the updated AIS data \( \widehat{\text{TS}}_{\text{Am}} \left( {k + 1} \right) \). Otherwise, the system will decode and convert new AIS measurements \( A_{i} \left( {k + 1} \right) \) to associate them with known target tracks using the two-stage fuzzy association decision-making algorithm. After associating the AIS measurements, the AIS prediction of each target \( {\text{TS}}_{Am}^{ - } \left( {k + 1} \right) \) is equal to the system-predicted states \( {\text{TS}}_{m}^{ - } \left( {k + 1} \right) \) and updated by the associated AIS measurements to generate AIS estimations \( \widehat{\text{TS}}_{AM} \left( {k + 1} \right) \).

After obtaining the system estimates \( \widehat{\text{TS}}_{Rm} \left( {k + 1} \right) \) and \( \widehat{\text{TS}}_{Am} \left( {k + 1} \right) \) by applying the radar and AIS measurements, respectively, these two estimations, rather than raw AIS and radar measurements, are then fused to obtain the master fusion results. The radar and AIS estimations belong to Gaussian distributions. Therefore, the two distributions for each target track \( m \) can be fused by:

$$ P\left( {x |\widehat{\text{TS}}_{Am} ,\widehat{\text{TS}}_{Rm} } \right) \propto P\left( {\text{TS}} \right)L\left( {{\text{TS}};\widehat{\text{TS}}_{Am} } \right)L\left( {{\text{TS}};\widehat{\text{TS}}_{Rm} } \right) = 1 \times { \exp }\frac{1}{2}\left( {\frac{{{\text{TS}} - \widehat{\text{TS}}_{Am} }}{{\sigma_{A} }}} \right)^{2} \times { \exp }\frac{1}{2}\left( {\frac{{{\text{TS}} - \widehat{\text{TS}}_{Rm} }}{{\sigma_{R} }}} \right)^{2} , $$
(47)
$$ \begin{aligned} & \widehat{\text{TS}}_{Fm} \left( {k + 1} \right) = { \arg }\mathop {\hbox{max} }\limits_{x} P\left( {x |\widehat{\text{TS}}_{Am} ,\widehat{\text{TS}}_{Rm} } \right) = \\ & { \arg }\mathop {\hbox{min} }\limits_{TS} - \log P\left( {x |\widehat{\text{TS}}_{Am} ,\widehat{\text{TS}}_{Rm} } \right) = \\ & { \arg }\mathop {\hbox{min} }\limits_{x} \left\{ {\frac{1}{2}\left( {\frac{{{\text{TS}} - \widehat{\text{TS}}_{Am} }}{{\sigma_{A} }}} \right)^{2} + \frac{1}{2}\left( {\frac{{{\text{TS}} - \widehat{\text{TS}}_{Rm} }}{{\sigma_{R} }}} \right)^{2} } \right\}, \\ \end{aligned} $$
(48)

where \( \widehat{\text{TS}}_{Fm} \) expresses the fused data, \( \sigma_{A} \) and \( \sigma_{R} \) are the error covariance obtained from the estimation process with AIS and radar updates, respectively. The flowchart of the multi-sensor target detection and tracking algorithm is shown in Fig. 5.

Fig. 5
figure 5

Flowchart of the multi-sensor target detection and tracking algorithm

Full size image

An improved weight distribution fusion algorithm has been proposed to deal with practical AIS sensor signal loss. It defines the relationship between the absence time of the AIS signal and the weights assigned to the AIS estimations. During the absence of AIS messages, the weight of AIS estimations reduces. A two-phase linear relationship is designed to describe the ratio of the weighting change and absence time as shown in Fig. 6. \( t1 \) represents the safe time margin. If the duration of the loss of AIS signal is less than \( t1 \), the change of the weight of AIS estimation is relatively small. The weight then drops rapidly to zero at \( t2 \) since the AIS estimations are no longer reliable without AIS updated messages.

Fig. 6
figure 6

Relationship between the weight of AIS estimation and the time without AIS update

Full size image

5 Simulation Results and Discussions

This section provides simulation results for validating the proposed algorithms. Two different sets of simulations have been designed to evaluate:

  1. 1.

    The performance of the proposed interacting multi-model (IMM)-based manoeuvring target detection and prediction algorithm using AIS data. Comparison with conventional KF has been undertaken to show that the proposed new filtering algorithm has a better capacity in dealing with complex movements of target ships;

  2. 2.

    The performance of the IMM-based multi-sensor data fusion algorithm in dealing with multitarget tracking mission. Specifically, the algorithm’s capability in handling sensor failure scenario (AIS signal is assumed to be lost during certain time period) has been tested to demonstrate that a fault-tolerance functionality can be achieved.

5.1 Simulations of the AIS-Aided Target Detection and Prediction Algorithm

In this section, AIS measurements are simulated to determine a single dynamic target’s navigational data as well as to make predictions during the long AIS data-transmitting intervals. The target ship is treated as a single point without considering its actual size. Portsmouth Harbour (Fig. 7a) is used to simulate a practical environment for the target. It has first been converted into a binary map (Fig. 7b), which has the dimension of 800 pixels * 800 pixels representing a 1.2 km * 1.2 km area (1 pixels = 1.5 m). The simulated target is assumed to be operating at constant and initially adheres to a straight-line trajectory. Additionally a current vector with the speed of 0.3 m/s at 155° is simulated that has the effects of pushing the target towards the southeast. The trajectory of the target is therefore altered and the target has a constant angular velocity of 3°/s when manoeuvring to correct its course, which is presented in Fig. 7b. The initial speed of the target is 7 knots on a course of 160°, while the updating intervals of the AIS measurements are 10 s under normal condition and 2 s when manoeuvring. The tracking start point is (450 m, 1200 m) and the end tracking point is (850 m, 64 m). The sampling time between each time step is 2 s. The target starts to manoeuvre after time step \( k = 140 \). Eight angular velocities from − 4°/s to 4°/s that cover most frequently used angular velocities of a vessel are chosen to generate 8 models.

Fig. 7
figure 7

Simulation scenario 1: a testing environment in Portsmouth harbour with a constant current and the simulated straight trajectory of the target; b the binary map and the altered true trajectory of the target

Full size image

Figure 8 shows the simulation results of the conventional KF-based AIS-aided target detection and prediction algorithm using the CV model. When the detected target is following its trajectory, 4 possible positions (red dots) are predicted by the proposed algorithm during each AIS data update interval and all the predictions are along the simulated trajectory (black line), which proves that the algorithm is able to provide effective estimated positions without AIS measurement updates during the time period. From the enlarged inset in Fig. 8, it is evident that the proposed algorithm performs creditably at improving AIS data accuracy since the estimated positions (green circles) are closer to the actual trajectory when the target is operating on a straight-line trajectory.

Fig. 8
figure 8

Simulation scenario 1: the simulation results of conventional KF-based target detection and prediction algorithm

Full size image

Figure 9a demonstrates the same simulation results as Fig. 8 with an enlarged inset detailing the end of the trajectory, where the target is conducting manoeuvres. It can be seen that the AIS data (blue squares) are updated more frequently when the target is approaching the end of its trajectory since it is correcting its course to get to its end point. However, the estimated positions (green circles) of the target are driven to an incorrect direction when the target is manoeuvring. The simulation results confirm the effectiveness of the constant velocity model-based conventional KF target detection and prediction algorithm when the target is not manoeuvring, but it is incapable of estimating the correct course of the target during manoeuvring although the AIS data updates more frequently. Figure 8b demonstrates the simulation results of the proposed IMMKF AIS-aided manoeuvring target detection and prediction algorithm. The manoeuvring target detection algorithm performs better at estimating the positions and courses of the detected target. It can be seen from the enlarged inset of Fig. 8b that the estimated positions (green circles) adhere to the true trajectory (black line) while the target is manoeuvring. Further numerical evidence is demonstrated in Figs. 10, 11, 12, 13, and 14.

Fig. 9
figure 9

Simulation scenario 1: the simulated AIS-measured positions and the predicted and the predicted and estimated position results using standard KF and IMMKF algorithms

Full size image
Fig. 10
figure 10

Simulation scenario 1: ideal course, AIS-reported cour, KF- and IMMKF-estimated courses

Full size image
Fig. 11
figure 11

Simulation scenario 1: the probabilities of each manoeuvring model generated by the IMM filter

Full size image
Fig. 12
figure 12

Simulation scenario 1: RMSEs of the target’s positions

Full size image
Fig. 13
figure 13

Simulation scenario 1: RMSEs of the target’s velocities

Full size image
Fig. 14
figure 14

Simulation scenarios 1: RMSEs of the target’s courses

Full size image

Figure 10 provides a comparison of the estimated target’s courses by conventional KF-based algorithm and the proposed IMMKF AIS-aided manoeuvring target detection and prediction algorithm. The actual course of the target is denoted as the black line, the AIS-reported course is shown as the blue line, the KF-estimated course is presented as the green line and the IMMKF-estimated course is denoted as the red line. This figure also supports the findings from Figs. 8 and 9 since the green line and the red line are very similar and closer to the black line than the blue line before the target starts manoeuvring around step \( k = 140 \)., but the green line starts to deviate from the other three lines from that point while the red line is still close to the black line.

The probability of each model shown in Fig. 11 expresses how the proposed IMMKF-based algorithm determines which model is the correct. Before time step \( k = 140 \) when the target is not manoeuvring, all the probabilities of the 8 models (mu1, mu2, mu3, mu4, mu5, mu6, mu7 and mu8) remains at 0. It can be seen that at the beginning of the manoeuvring period, the probabilities of mu3, mu4 and mu6 are peaked and return to 0 in a short time. It is caused by the insufficient data obtained by the manoeuvre detector algorithm at initial stage. After extracting enough data, the proposed algorithm determines the correct model mu7 that represents the angular velocity of 3°/s and its probability becomes the largest and tends to 1 during the target’s manoeuvring, which is the same as the target’s actual angular velocity. The results prove the effectiveness of the designed manoeuvre detector.

The rooted mean square errors (RMSEs) of the target’s positions, velocities and courses that are detailed in Figs. 12, 13 and 14 further support the improvement made by the proposed IMMKF AIS-aided manoeuvring target detection and prediction algorithm. In each figure, the blue line indicates the RMSE of the AIS raw measurements, the green line denotes the RMSE of the conventional KF-based estimations and the red line represents the RMSE of the IMMKF-based estimations. Around time step \( k = 140 \), the target starts to manoeuvre and the RMSEs of the KF-estimated positions, velocities and courses increase while the RMSEs of the proposed IMMKF estimations remain lower than those of both the KF estimations and AIS raw measurements. This is clearly evident in all the aspects of the targets navigational data, especially the course in Fig. 14. The RMSE of KF-estimated courses steadily increases and eventually exceeds the error of the raw AIS measurement. The comparisons of RMSEs provide numerical evidence that estimation of the target’s positions in the x and y directions are improved by 4 m and 3 m, respectively, and the RMSE of the courses are reduced by approximately 50% by the IMMKF algorithm. All the evidence indicates the proposed IMMKF with manoeuvre detector target detection and prediction algorithm based on AIS data is effective for both detecting the target and predicting its positions and courses when the target is manoeuvring.

5.2 Simulation of the Multi-sensor Target Detection and Tracking Algorithm

The dynamic multiple target detection system is implemented by simulating four targets around the USV. Three of them are operating with both AIS and in marine radar detection range and one of them can only by detected by radar. The specific parameters of the simulated targets are listed in Table 2.

Table 2 Simulation scenario 2: simulated unmanned surface vessel (USV) and targets’ initial position, speed and course
Full size table

Assuming the targets 1 to 3 are equipped with AIS transponders and the USV can collect their AIS dynamic information at reporting interval \( t_{A} = 10 {\text{s}} \). Target 2 is set to be disabled for \( k = 100 {\text{s to }}120 {\text{s}} \) and the AIS signal of target 3 is lost during

\( k = 300 {\text{s to }}450 {\text{s}} \). The sampling time of the USV’s radar is 2 s, which is also used as the system’s sampling time and the whole observation time is 900 time steps. During the observation, all the targets are operating at constant speed and constant angular velocity when required, modelled as both CV model and CT model. The RMS error vectors for the AIS signals are 0.01 nautical miles in position, 0.007 knots in speed and 0.5 degree in course and for radar are 0.08 nautical miles in relative range, 1.2 degree in relative bearing, 0.03 knots in speed and 1.0 degree in course. The parameters of the improved weight distribution fusion algorithm are defined as \( w1 = 0.6, t1 = 60 {\text{s }} \) and \( t2 = 300 {\text{s}} \). Figure 15 shows the simulated actual trajectories of the four target ships as magenta, blue, green and yellow lines, respectively, and the unmanned surface vessel (USV) as the black line.

Fig. 15
figure 15

Simulation scenario 2: simulated multiple targets environment surrounding a USV

Full size image

Figure 15 demonstrates the master fusion results of the multiple targets detection system, which are displayed in Figs. 16, 17, 18, and 19. The actual positions, radar measurements, possible AIS measurements and master fused positions are presented for each target. It can be seen that target 4 does not have AIS measurements and its fused results are generated from radar-based estimations. The trajectory results prove that the proposed multi-sensor target detection and tracking algorithm can successfully and efficiently associate each AIS and radar measurement to the related target tracks using the two-stage fuzzy multi-factor integration data association algorithm.

Fig. 16
figure 16

Simulation scenario 2: fused trajectories of target ship 1

Full size image
Fig. 17
figure 17

Simulation scenario 2: fused trajectories of target ship 2

Full size image
Fig. 18
figure 18

Simulation scenario 2: fused trajectories of target ship 3

Full size image
Fig. 19
figure 19

Simulation scenario 2: fused trajectories of target ship 4

Full size image

Figures 20, 21, 22, and 23 illustrate the RMSEs of the positions and courses for each target. The blue solid line represents the RMSEs of raw radar measurements, the blue dashed line denotes the RMSEs of radar-based estimations, the green solid line represents the RMSEs of raw AIS measurements, the green dashed line denotes the RMSEs of the AIS-based estimations and the red line demonstrates the master fused results. In Fig. 20, the AIS is updated every 10 s while the radar provides continuous measurements at 2-s intervals. The master fusion results are much closer to the AIS estimations rather than the radar estimations due to the high accuracy of AIS estimations. Two very small increases of the RMSEs of the fused course occur while target 1 is manoeuvring. In Fig. 21, the RMSEs of the master fusion results of all of target 2’s navigational data are increased from time step \( k = 100 {\text{s}} \) because of the absence of AIS measurements. The increase is then eliminated after the AIS measurements are restored after 20 s. A similar pattern occurs in Fig. 22, where target 3 is set to lose its AIS signal for 150 s. But the RMSEs of the master fusion results are more highly augmented due to the long absence duration of AIS signal. For target 4, the proposed algorithm is still able to reduce the RMSEs of raw radar measurements automatically without AIS integration. These results validate the performance of the proposed multi-sensor target detection and tracking algorithm. Although the RMSE of the master fusion results are increased by a small amount when the targets are making manoeuvres, the algorithm is able to compensate for such increase before the error increases to a magnitude greater than the error of raw radar measurements, which confirms the effectiveness of the fusion algorithm.

Fig. 20
figure 20

Simulation scenario 2: the RMSEs of target ship 1’s positions and courses

Full size image
Fig. 21
figure 21

Simulation scenario 2: the RMSEs of target ship 2’s positions and courses

Full size image
Fig. 22
figure 22

Simulation scenario 2: the RMSEs of target ship 3’s positions and courses

Full size image
Fig. 23
figure 23

Simulation scenario 2: the RMSEs of target ship 4’s positions and courses

Full size image

6 Conclusions and Future Work

This paper was dedicated to developing intelligent and reliable data fusion algorithms for both single and multiple target detection, prediction and tracking. Instead of using the constant velocity model alone, a manoeuvring target detection and prediction algorithm based on IMM filtering with different coordinate turn models has been developed to estimate the navigational data of the target. Furthermore, a multi-sensor data fusion algorithm for the AIS and marine radar measurements has been proposed to implement a multiple target detection and tracking system. The raw sensor measurements were pre-processed individually using the developed manoeuvring target detection and prediction algorithm and both outputs were then being associated with related targets to make further fusions. The multi-sensor data fusion algorithm increases system reliability by using two different sensors as the complementary devices. Simulations have been carried out to provide numerical evidence that the proposed target detection, prediction and tracking algorithms are effective in realising their designed purpose.

The improved target ship detection capability achieved in this paper can potentially enable a more reliable autonomous navigation for unmanned ships together with robust control algorithms. To accommodate the complexities that occurred during the control processes, fuzzy logic-based robust controllers (such as the ones proposed in [31,32,33]) can improve the control performance and be seamlessly integrated into autonomous navigation systems. In addition, while navigating in ocean environments, uncertainties generated by system modelling, ocean environments and vessels’ autopilot also require a robust and adaptive tracking control strategy. Methods developed not only for surface vessels, but also for underwater vehicles can be well adopted and integrated. Especially, as underwater environment possesses more challenging constraints than surface one, with appropriate modification a number of most recent autonomous underwater vehicle (AUV) fault-tolerant tracking control methods such as the disturbance-observer-based strategy [34], performance-function-constrained [35] and data-driven adaptive tracking [36] can be implemented on surface vessels for enhanced control performances.

Another important further research direction is towards multi-sensor fusion using a wider spectrum of sensors including vision systems and LiDAR. Although radar can detect target ships over a wide range of areas with a reasonable performance, the radar scatters may provide false sizes of targets due to reflection or disturbance. Moreover, the sampling rate and resolution of radar are relatively low compared to other sensors, such as LiDAR and vision sensors. Therefore, to enhance the awareness capabilities of autonomous ships, especially for detecting high-speed or small size obstacles, additional sensors are necessary to be integrated into the sensing system on board with an improved sensor-fusion algorithm that is capable of processing heterogeneous data. Weather influences challenges such as the fog, haze and sunlight should be especially addressed when vision systems are implemented.