Particle tracking for the assessment of microcirculatory perfusion

Circulatory shock is a widespread clinical condition that affects about 30% of the patients admitted to European intensive care units (ICU) (Sakr et al 2006). In septic shock, which represents the most common form, tissue oxygenation is compromised by the disruption of the physiological regulatory mechanisms (myogenic, metabolic and neurohumoral) that control blood flow at the microcirculation level in order to meet the local requirements of tissue metabolism (Ince 2005). In this pathological condition, therefore, microvascular perfusion can still be deteriorated even after the normalization of inadequate macrohemodynamic parameters: a dysfunctional microcirculation, in fact, may not be reflected by the systemic circulation as a consequence of arteriovenous shunting. In the light of the above, monitoring the peripheral perfusion of ICU patients may provide valuable biomarkers for the early detection of a potential impairment of the oxygenation of vital organs, as the microcirculation of non-vital districts is the first to reflect the perfusion derangements characteristic of circulatory shock. Besides, in patients suffering from septic shock, the possibility to directly evaluate the microcirculation status may help the identification of adequate endpoints of hemodynamic resuscitation that ensure the optimal correction of local blood flow deficit, without compromising the efficiency of oxygen transport due to excessive hemodilution (Boldt and Ince 2010).

Recent years have witnessed the development of subsequent generations of compact, mobile, noninvasive imaging devices (Bauer et al 2007, Goedhart et al 2007, Aykut et al 2015) that, for the first time, have enabled the visualization of the microcirculation directly at the bedside of ICU patients. Clinical studies based on these hand-held microscopy devices found that microcirculatory alterations are best correlated with the occurrence and severity of multi-organ failure (Trzeciak et al 2007) and that persistent alterations in microvascular perfusion at later than 24 h after the onset of shock are a good predictor of ICU mortality (Sakr et al 2004), even in the absence of abnormal values of conventional hemodynamic and oxygen-derived parameters.

However, the subjective evaluation of microvascular alterations based on the visual inspection of the recorded videos is unfeasible due to the complexity of the observed patterns, determining a poor intra- and inter-observer reproducibility. The definition and adoption, in the microcirculation research community, of a consensual scoring framework, comprising several quantitative and semi-quantitative parameters of capillary density and perfusion, laid the basis for a standardized assessment procedure that guarantees improved reproducibility and reliability (De Backer et al 2007). Although such scoring system demonstrated to be sensitive and specific in the assessment of disease severity in critically ill patients, it still entails time-consuming tasks.

Nevertheless, this standard evaluative approach has also provided a reference for the development of technologies able to translate this procedure into the context of critical care, by partially automating the measures through the introduction of adequate image processing algorithms (Dobbe et al 2008). A few dedicated software packages, which support processing and semi-automated analysis of microcirculation videos, are nowadays available. In particular, we focused our attention on two applications currently used in microvascular research: Automated Vascular Analysis (AVA—Microvision Medical) and Cytocam Tools (CCTools—Braedius), both offering automated vessel segmentation and assisted scoring of the microcirculation, including the assessment of the capillary perfusion speed. The main prerequisite step for the quantitative evaluation of perfusion rates consists in vessel segmentation. In AVA, the automatic detection and reconstruction of blood vessels is based on the algorithm described by Steger (1998), while CCTools adopts a similar approach, proposed in Frangi et al (1998). Both techniques require the video frames to be preliminarily stabilized, in order to compensate for potential unwanted displacements of the field of view during the video acquisition, and time-averaged to smooth the gray levels discontinuities caused by the presence of plasma gaps between flowing RBCs. After these steps, both segmentation methods employ a tensor representation of the time-averaged frame that allows the delineation of simple linear structures, characterized by gray levels that mainly change in only one direction, through the eigenvector analysis of the local Hessian matrix.

In the research software currently used for the quantitative analysis of the microcirculation, the estimation of RBC velocities involves the generation of the so-called space-time diagrams (STD) of individual capillaries (Jähne 1991). In this technique, the centerline of the analyzed capillary is initially straightened by an adequate deformation of the image, as shown in figure 1. Then, the diagram is created by arranging the gray level intensities of the centerline pixels, corresponding to the subsequent frames composing the video, into adjacent columns (or rows, depending on the utilized software package), as shown in figure 2. One image coordinate thus assumes a spatial meaning, indicating the position along the straightened vessel centerline, and the second one characterizes the temporal evolution of the intensities: depending on the chosen orientation, columns (rows) represent the spatial distribution of the gray levels along the vessel axis at a specific time point, while rows (columns) show how the intensities vary with time at a specific position.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In the STDs, erythrocyte flow creates characteristic dark, oblique linear structures, that alternate with lighter patterns, associated with the presence of plasma gaps. The higher the velocity of a RBC is, the larger is the distance it travels in the time interval between two consecutive frames, and hence the local slope of the dark track, with respect to the temporal axis. Therefore, an estimate of the mean erythrocyte velocity can be obtained by taking the average of the slopes of the lines detectable by the operator. Also, automated slope estimation may be performed using the Hough transform (Dobbe et al 2008).

It is important to note how the occurrence of linear structures in the STDs requires the presence of an adequate contrast along the direction longitudinal to the vessel: in other words, the analyzed video must offer a proper definition of the RBCs, with acceptable contrast and focus. The measurement conditions and the performance of the microscope devices must therefore allow the detection of the granular nature of the flow through the capillary network. However, regardless from image quality, ambiguities may arise in the STDs generated from vessels large enough to allow the erythrocytes to flow alongside each other, especially at high cell densities. In these diagrams, cells may occlude and pass each other, leading to uncertainty in the physical correspondence of the objects detected in consecutive frames. As a consequence, STDs may not display lines univocally related to separate flowing cells. Besides, space-time diagrams are often irregular, given the uneven spacing of cells, and the speed fluctuations which may occur both over time and along the vessel length. Thus, the operator usually performs a visual best-fit of several lines in different portions of the diagram, manually sampling RBC speed in space and time. This approach is time-consuming, inherently subjective, and does not take into account the variability of the estimates.

The manual estimation of RBC speed from STDs is thus one of the major limitations for a widespread clinical application of these portable microscopes to the study of the mucosal microcirculation. For this reason, much research is ongoing with the aim to provide an automated assessment of microcirculatory parameters but, up to now, no established method is available (Massey and Shapiro 2016). Among the possible strategies, the most recently proposed methods include dynamic time warping (Grisan et al 2009), optical flow estimation (Liu et al 2015), and block matching algorithms (Lin et al 2014).

In our preliminary work (Sorelli et al 2015), we demonstrated that RBC segmentation and object tracking could provide robust estimates for speeds up to 200 μm s⁻¹. Using the same approach, in this study we extended the algorithm toward a fully automated method for the assessment of RBC speed in the microvessels that are detected in microscopy videos of the sublingual mucosal microcirculation. In every frame, erythrocytes are segmented, so as to obtain their position and, thanks to a Kalman filter-based multiple-object tracking, their individual path is reconstructed over consecutive video frames. The outcome of the algorithm provides an estimate of the average RBC speed.

2.1. Preliminary study

We conducted a preliminary study, aimed at identifying the velocity range which can be generally assessed from the analysis of microcirculation videos of state-of-the-art portable microscopes. To this end, we referred to a dataset of 920 STD-based estimates of RBC velocity, collected from videos captured with Cytocam (Aykut et al 2015), in porcine models of circulatory shock. The velocity distribution observed in this dataset is shown in figure 3. The histogram exhibits a large number of values concentrated between 100 and 300 μm s⁻¹, with only a relatively few samples above 500 μm s⁻¹. Statistical analysis revealed a median speed of 187 μm s⁻¹, and an interquartile range given by $307\, {\rm\mu}\text{m}\,{{\text{s}}^{-1}}-114\, {\rm\mu}\text{m}\,{{\text{s}}^{-1}}=193\, {\rm\mu}\text{m}\,{{\text{s}}^{-1}}$. In STD analysis higher speed observations tend to be affected by increased inaccuracy, due to the higher sensitivity of the velocity estimates to small adjustments of the lines traced by the operators: even pixel-size displacements of the reference points produce a large difference in the estimated value. For this reason, a reliable evaluation of RBC velocities above 500 μm s⁻¹ is feasible only in a limited number of cases; in particular, when the operator is able to perform a manual adjustment of the capillary detections, so as to obtain an overall vessel length that allows to visualize RBC cells travelling at fast speed for a sufficient period of time.

Zoom In Zoom Out Reset image size

2.2. Dataset

2.3. Erythrocyte segmentation

Segmentation of erythrocytes is performed on each frame independently from the others. Due to the possibly high flow speed, information about previous locations of the RBCs does not add any actual benefit to the performance of the detector. In dark field microscopy movies, cells appear as dark blobs flowing on a lighter background; however, salt-and-pepper noise generally corrupts the video frames, that need to be properly enhanced to improve RBC segmentation. Four operations constitute the preprocessing module: after an inversion of the image, a median filter, with a $7\times 7$ kernel size, is applied to the original image for noise reduction; next, a top-hat filter, based on a disk-shaped structuring element of radius 16 pixels attenuates background inhomogeneities; then, small artefacts, which may remain after the background suppression, are reduced with a morphological opening using a 4-pixel radius disk as structuring element (Soille 2003). The size of the structuring elements was suitably chosen so as to avoid the undesirable cancellation of true RBCs. Normocytic erythrocytes, in humans, have an average diameter of about 7.5 ${\rm\mu}\text{m}$ (Boron and Boulpaep 2008). Therefore, in the microscopy videos captured with Cytocam, the expected average RBC size is $7.5\, {\rm\mu}\text{m}/{{p}_{s}}=10.7$ pixel. Since top-hat filtering and morphological opening remove bright details that are respectively larger and smaller than the structuring element they employ, the selected parameters suit the requirement.

Figure 4 shows a sample of the intermediate and final results of the preprocessing module. As it can be observed in this sample image, flowing RBCs tend to align along the centerline of the image, corresponding to the vessel axis; consequently, potential blobs that are detected close to the upper and bottom edges, likely correspond to spurious outcomes of the segmentation process. A space-dependent contrast-enhancement, which increases contrast along the horizontal axis of the video frames and decreases it along the top and bottom edges, allows to reduce those spurious detections. More specifically, the enhancement is obtained by multiplying the pixel intensities with a Gaussian mask M(x,y) of the form:

$$\begin{eqnarray}&&M(x,y)=\exp -\frac{1}{2}{{\left(\frac{y-{{y}_{a}}}{{{\sigma}_{y}}}\right)}^{2}}\end{eqnarray} \tag{ 1 }$$

where y_a  =  50 is the vertical coordinate of the vessel axis and ${{\sigma}_{y}}=20$. Finally, connected bright areas are isolated and segmented from the background using the Otsu image thresholding method (Otsu 1979). Results of the contrast enhancement, extraction of connected components, and image thresholding are shown in figure 5. Each blob represents a detected RBC, whose centroid is fed to the tracking algorithm.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

2.4. Kalman filtering

In the context of classic experimental fluid dynamics, the application of Kalman filtering techniques to the problem of particle tracking can be traced back to 20 years ago (Yagoh et al 1992, Takehara et al 1996). In recent years, this approach has been translated to the biomedical field, to address the challenge of estimating RBC speed (Sugii et al 2005, Moon et al 2016) still, to our knowledge, it has never been implemented in vivo nor extended to clinical scenarios.

The Kalman filter deals with the estimation of the state of a linear, stochastic, discrete-time process, governed by the following general difference equation (Welch and Bishop 1995):

$$\begin{eqnarray}&&{{\vec{s}}_{t}}=A{{\vec{s}}_{t-1}}+B{{\vec{u}}_{t-1}}+{{\vec{n}}_{p,t-1}}\end{eqnarray} \tag{ 2 }$$

where ${{\vec{s}}_{t}}$ represents the state vector of the particle at time t, ${{\vec{u}}_{t}}$ is the (optional) control input vector, and ${{\vec{n}}_{p,t-1}}$ expresses the process noise. $A$ and $B$ are the matrices that define the process by relating the previous state and the current control input to the current process state. Furthermore, the general equation describing the measurements performed on the process is given by:

$$\begin{eqnarray}&&{{\vec{z}}_{t}}=C{{\vec{s}}_{t}}+{{\vec{n}}_{m,t}}\end{eqnarray} \tag{ 3 }$$

where ${{\vec{z}}_{t}}$ is the measurements vector, ${{\vec{n}}_{m,t}}$ represents the localization noise, and C is the measurement matrix. In the Kalman filter theory the random processes ${{\vec{n}}_{p,t-1}}$ and ${{\vec{n}}_{m,t}}$ are assumed to be white, reciprocally independent and Gaussian; thus their covariance matrices are diagonal. Moreover, although they could in general be time-varying, process and measurement matrices are here also assumed to be constant. In the adopted approach, a linear motion model with constant velocity is hypothesized for the erythrocytes. Therefore, the specific expression of the state equation of the ith particle is the following:

$$\begin{eqnarray}&&\left[\begin{array}{c} {{x}_{i,t}} \\ v{{x}_{i,t}} \\ {{y}_{i,t}} \\ v{{y}_{i,t}} \end{array}\right]=\left[\begin{array}{c} 1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right]\left[\begin{array}{c} {{x}_{i,t-1}} \\ v{{x}_{i,t-1}} \\ {{y}_{i,t-1}} \\ v{{y}_{i,t-1}} \end{array}\right]+\left[\begin{array}{c} {{\delta}_{p}}{{x}_{i,t}} \\ {{\delta}_{p}}v{{x}_{i,t}} \\ {{\delta}_{p}}{{y}_{i,t}} \\ {{\delta}_{p}}v{{y}_{i,t}} \end{array}\right]\end{eqnarray} \tag{ 4 }$$

where the state of the system at an arbitrary time t is given by the particle position and speed $\left({{x}_{i,t}},v{{x}_{i,t}},{{y}_{i,t}},v{{y}_{i,t}}\right)$. As it can be deduced from the A matrix, the motion model assumes the vertical position of the RBC to be constant and the vertical speed to be null; therefore, any observed variability in the vertical direction is modeled only through the process noise. Besides, since image segmentation and blob analysis return an estimate of the position of the objects being tracked, we contextually express the measurement equation as follows:

$$\begin{eqnarray}&&\left[\begin{array}{c} {{{\bar{x}}}_{i,t}} \\ {{{\bar{y}}}_{i,t}} \end{array}\right]=\left[\begin{array}{c} 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right]\left[\begin{array}{c} {{x}_{i,t}} \\ v{{x}_{i,t}} \\ {{y}_{i,t}} \\ v{{y}_{i,t}} \end{array}\right]+\left[\begin{array}{c} {{\delta}_{m}}{{x}_{i,t}} \\ {{\delta}_{m}}{{y}_{i,t}} \end{array}\right]\end{eqnarray} \tag{ 5 }$$

where ${{\delta}_{m}}{{x}_{i,t}}$ and ${{\delta}_{m}}{{y}_{i,t}}$ are the horizontal and vertical components of the measurement error, respectively.

The implementation of the Kalman filter requires the a priori knowledge of the process and measurement covariance matrices. In this specific application, RBC flow was hypothesized to closely follow uniform motion along the vessel axis (i.e. the x image axis); accordingly, we set the ${{\delta}_{p}}{{x}_{i,t}}$ and ${{\delta}_{p}}v{{x}_{i,t}}$ variances to a very small value: $0.0001~\text{pixe}{{\text{l}}^{2}}$. On the other hand, having assumed that vy_i  =  0, we introduced a higher level of process disturbance for the y coordinate into the model, ${{\delta}_{p}}{{y}_{i}}=100~\text{pixe}{{\text{l}}^{2}}$, in order to take the variability in the RBCs vertical position properly into account, and to make cells tracking more robust. The variance of the localization errors ${{\delta}_{m}}{{x}_{i,t}}$ and ${{\delta}_{m}}{{y}_{i,t}}$ was tailored according to the size of the RBC detections, which were expected to have a radius lower than 4 ${\mu} \text{m}$. Accordingly, we set ${{\delta}_{m}}{{x}_{i,t}}$ and ${{\delta}_{m}}{{y}_{i,t}}$ to ${{\left(2\, {\rm\mu}\text{m}/{{p}_{s}}\right)}^{2}}=8.2\,\text{pixe}{{\text{l}}^{2}}$.

2.5. Particle tracking

Video frame enhancement and segmentation provide the Kalman Filter with a set of noisy observations z_i,t of the motion process of each RBC. However, since segmentation generally produces, in each frame, several detections of varying shape and size, we face a crucial difficulty in obtaining a correct motion reconstruction based on video processing: the correspondence problem, i.e. the problem of ensuring the correct matching between each moving object and the locations associated to it over consecutive frames, in a scenario where there is no self-evident correspondence rule. To this end, we exploit the a priori information inherent to the motion model of the Kalman filter; more specifically, the previous position of each RBC being tracked is projected forward in time, according to the model, and the Euclidean distances between each a priori estimate and the new measured positions are computed. This array of distances between predicted and measured positions provides a cost value for each of the possible associations. The optimal combination between active tracks and new detections is then identified through the minimization of the overall assignment cost, based on the association algorithm described in Munkres (Munkres 1957). The adopted approach maximizes the probability of maintaining the biunivocal correspondence between tracks and detections, provided that the sampling time interval is narrow enough to make the RBC displacement between consecutive frames significantly shorter than the average RBC reciprocal distances. The adopted association algorithm does not necessarily imply the assignment of a new position to each of the currently active tracks, thus allowing the presence of 'unassigned' tracks. These unassigned tracks are not immediately erased; instead, they are marked as invisible and kept active, in order to cope with possible misdetections, and preserve tracking robustness even in the presence of substantial noise. This visibility feature is exploited at a successive tracks selection step. Conversely, new detections, that are not linked to any of the existing tracked objects, are used as seed points for initializing the state of new tracks: the centroid of the detection provides the initial condition for the RBC position $\left({{x}_{i,t}};{{y}_{i,t}}\right)$, while the velocity components are preliminarily assumed to be null; therefore, the initial state of the tracked particle is given by:

$$\begin{eqnarray}&&{{\vec{s}}_{i,t}}={{\left[{{x}_{i,t}};0;{{y}_{i,t}};0\right]}^{\prime}}\end{eqnarray} \tag{ 6 }$$

Ideally, this mechanism is designed to handle the situations where new RBCs come into view for the first time, avoiding the creation of an incorrect link to active tracks associated to different moving cells. In practice, it also proves to be useful to resume the tracking of RBCs which were lost by their original tracks. According to the Kalman filter theory, an a priori error covariance $P_{0}^{-}$ of the estimated state must be set upon track initialization:

$$\begin{eqnarray}P_{0}^{-}=\left[\begin{array}{c} {{\delta}_{p}}{{x}_{i,0}} & 0 & 0 & 0 \\ 0 & {{\delta}_{p}}v{{x}_{i,0}} & 0 & 0 \\ 0 & 0 & {{\delta}_{p}}{{y}_{i,0}} & 0 \\ 0 & 0 & 0 & {{\delta}_{p}}v{{y}_{i,0}} \end{array}\right]\end{eqnarray} \tag{ 7 }$$

The selection of the elements of $P_{0}^{-}$ follows the same rationale adopted for the process noise, with the exception of the variance of the axial velocity component ${{\delta}_{p}}v{{x}_{i,0}}$ that is set to a very high value, so as to allow the filter to rapidly adapt the corresponding state component to the detected displacements, avoiding being trapped in the initial condition:

$$\begin{eqnarray}\begin{array}{*{35}{l}} {{\delta}_{p}}{{x}_{i,0}}={{\delta}_{p}}{{y}_{i,0}} & = & {{\left(4\, {\rm\mu}\text{m}/{{p}_{s}}\right)}^{2}}=32.7\,\text{pixe}{{\text{l}}^{2}} \end{array}\end{eqnarray} \tag{ 8 }$$

$$\begin{eqnarray}\begin{array}{*{35}{l}} {{\delta}_{p}}v{{x}_{i,0}} & = & 10\,000~\text{pixe}{{\text{l}}^{2}} \end{array}\end{eqnarray} \tag{ 9 }$$

$$\begin{eqnarray}\begin{array}{*{35}{l}} {{\delta}_{p}}v{{y}_{i,0}} & = & 0~\text{pixe}{{\text{l}}^{2}} \end{array}\end{eqnarray} \tag{ 10 }$$

The motion model provides an a priori state estimate $\vec{s}_{i,t}^{\,-}$ for each of the active tracks, within the currently processed video frame. All estimates are associated with an error covariance given by:

$$\begin{eqnarray}&&P_{t}^{-}=A{{P}_{t-1}}{{A}^{T}}+Q\end{eqnarray} \tag{ 11 }$$

where P_t−1 represents the a posteriori estimate error covariance at the previous frame, and Q is the covariance of the process noise. In the case of the tracks assigned by the Munkres algorithm, these a priori estimates are integrated with the information carried by the corresponding measurements. In practice, the a posteriori state ${{\vec{s}}_{i,t}}$ is computed as the linear combination of the model prediction $\vec{s}_{i,t}^{\,-}$ and the weighted residual between $\vec{s}_{i,t}^{\,-}$ and the actual measurement ${{\vec{z}}_{i,t}}$:

$$\begin{eqnarray}&&{{\vec{s}}_{i,t}}=\vec{s}_{i,t}^{\,-}+{{K}_{t}}\left({{\vec{z}}_{i,t}}-\vec{s}_{i,t}^{\,-}\right)\end{eqnarray} \tag{ 12 }$$

The weight K_t is the Kalman filter gain at time t which, in the present tracking problem, is given by:

$$\begin{eqnarray}&&{{K}_{t}}=\frac{P_{t}^{-}}{P_{t}^{-}+R}\end{eqnarray} \tag{ 13 }$$

where R represents the covariance of the measurement noise. The covariance of the estimate error is likewise updated as:

$$\begin{eqnarray}&&{{P}_{t}}=\left(I-{{K}_{t}}\right)P_{t}^{-}\end{eqnarray} \tag{ 14 }$$

where I denotes the identity matrix. On the other hand, the state of the tracks left unassigned, for which no measurement is actually available, is updated only on the basis of the motion model prediction, i.e.:

$$\begin{eqnarray}&&{{\vec{s}}_{i,t}}=\vec{s}_{i,t}^{\,-}\end{eqnarray} \tag{ 15 }$$

$$\begin{eqnarray}&&{{P}_{t}}=P_{t}^{-}\end{eqnarray} \tag{ 16 }$$

Once the set of detection/tracking operations described above is completed on the current frame, spurious tracks, identified as those tracks having age less than 5 frames and visibility less than 60% of the track age, are suppressed and discarded.

Besides, a further selection criterion is applied in order to properly deal with RBCs which have flowed out from the analyzed frame. The corresponding 'lost' tracks need to be disabled to avoid an incorrect matching between them and different moving objects, thus preventing the consequent corruption of the reconstructed particle trajectory. In our method, a track is considered lost if marked as invisible for 3 consecutive frames. Lost tracks, differently from spurious tracks, may correspond to a correct tracking process and are thus used for the later average flow velocity estimation. Therefore, although they are removed from the tracking algorithm, the corresponding particle positions are saved.

When the iteration over the video frames is completed, a post-processing module limits the effect of noisy/invalid trajectories produced, for instance, by static artefacts and matching errors (i.e. different RBCs associated to the same track), on the final average axial speed estimate. This module accepts only those tracks that completely fulfil the following list of requirements:

• ratio of the maximum excursion along y to the maximum excursion along x lower than 0.5: cells not moving along the vessel axis are rejected;
• number of changes in the sign of vx_i lower than 3: a large number of changes in the flow direction relates to matching errors;
• vx_i direction consistent with the direction of the majority of flowing cells: at high flow rates, an 'aliasing' effect may be observed, with different cells matched to the same track during the video;
• $|v{{x}_{i}}-v{{x}_{\text{mean}}}|<1.65\centerdot v{{x}_{\sigma}}$, where vx_mean and $v{{x}_{\sigma}}$ are respectively the mean and standard deviation of the estimates (i.e. being inside the 90% confidence interval of the tracks mean velocity).

Finally, the estimated RBC flow speed v, expressed in μm s⁻¹, is computed as the average of the mean axial speeds of each reliable track satisfying the previous inclusion criteria, weighted by their respective length, so as to assign a more important contribution to long-lasting, stable tracks:

$$\begin{eqnarray}&&v=f\centerdot {{p}_{s}}\centerdot \frac{{\sum}_{i=1}^{{{{N}_{t}}}}\,{{N}_{i}}\centerdot v{{x}_{\text{mean}}}}{{\sum}_{i=1}^{{{{N}_{t}}}}\,{{N}_{i}}}\end{eqnarray} \tag{ 17 }$$

where $f=25$ Hz is the video frame rate, N_t is the number of reliable tracks, and N_i represents the tracking time of the ith track.

A preliminary evaluation, based on the STD method, was aimed at comparing the velocity distribution of the selected subset of data with the reference distribution of the larger dataset, derived from the animal models. Figure 6 shows the histogram of the velocities estimated with the STD method. The histogram indicates that the expected velocity is usually below 400 μm s⁻¹, with a large percentage of cases having speed below 300 μm s⁻¹. Therefore, the selected dataset well represents the expected velocity distribution that can be observed in the general population of patients. Thus, the evaluation of the system performance will be focused on this velocity range.

Zoom In Zoom Out Reset image size

In a first phase, a visual assessment of the tracking procedure allowed to evaluate the correspondence between the proposed system and the STD approach. The RBC-tracking tool creates STD-like images, displaying the reconstructed path of each tracked object throughout the video sequence, as shown in figure 7. The figure reports both accepted and rejected lines, thus providing a feedback on the possible problems that may occur during the tracking process. In particular, the picture allows to assess the inclusion of spurious tracks in the calculation of the final velocity estimate. Generally, we may observe a few quasi-horizontal, irregular lines, that correspond to spurious detections of local quasi-static noise in the image. Besides, the visual inspection of these diagrams may occasionally reveal lines which exhibit an opposite orientation with respect to the actual RBC flow. These lines are usually related to some aliasing effect, i.e. the tracking algorithm associates different cells to the same track. Moreover, in correspondence with the end of the recording, several tracks are usually rejected due to the limited visibility time of the tracked objects. Overall, the qualitative assessment of the diagrams shows a good performance of the tracking system, and supports the effectiveness of the tracks rejection method, thus indicating a reliable estimate of the RBC velocity.

Zoom In Zoom Out Reset image size

The quantitative analysis of the system performance consisted in comparing the velocity estimates obtained with the proposed method against the corresponding results provided by the established STD manual technique. The comparison was based on correlation and Bland–Altman analyses. The correlation plot between the manually and automatically derived estimated is shown in figure 8.

Zoom In Zoom Out Reset image size

The resulting sample Pearson's correlation coefficient is r  =  0.951, corresponding to a coefficient of determination r²  =  0.905. This result expresses a strong positive, statistically significant ($p\ll 0.001$), linear correlation between the compared velocity variables, with 90.5% of the variance in the automatically-derived speed that is predictable from the reference STD-based estimate.

Linear least-squares regression of the velocity data produced a best-fitting line of equation y  =  mx  +  p  =  0.953x  −  5.066. Given the almost unitary slope of the regression line (m  =  0.953), the correlation data already suggests a high degree of agreement between the compared methods.

Figure 9 shows the Bland–Altman graph (Altman and Bland 1983) resulting from the sample data; the Bland–Altman analysis highlights only a slight bias between the techniques: the mean difference d between the estimates is  −2.2 μm s⁻¹ (confidence interval $-4.6<d<0.9\, {\rm\mu}$m s⁻¹ with a significance level $\alpha =0.05$). Concerning the determination of the 95% limits of agreement, the analysis gave ${{l}_{\text{low}}}=-49.0\, {\rm\mu}\text{m}\,{{\text{s}}^{-1}}$ (confidence interval $-60.8<{{l}_{\text{low}}}<-37.2\, {\rm\mu}\text{m}$ s⁻¹, $\alpha =0.05$) and ${{l}_{\text{up}}}=44.6\, {\rm\mu}\text{m}\,{{\text{s}}^{-1}}$ ($32.9<{{l}_{\text{up}}}<56.4\, {\rm\mu}$m s⁻¹, $\alpha =0.05$). The calculated values shown here include the contribution of the three outliers, labelled with letters in figure 9, which present a large disagreement with respect to the reference STD method.

Zoom In Zoom Out Reset image size

The maximum theoretical particle speed that allows a reliable tracking depends on the particles size, the device acquisition rate, and the distance between neighbour particles $\Delta x$. The basic rule, i.e. matching a particle with the closest one that was detected in the previous frame, yields a correct match if:

$$\begin{eqnarray} &&\Delta x>2\centerdot {{v}_{x}}/f\end{eqnarray} \tag{ 18 }$$

In the worst case scenario, where RBCs of diameter equal to the lowest extreme of the normal range, ${{D}_{\text{min}}}=6\, {\rm\mu}\text{m}$, are adjacent to each other, the maximum speed that can be reliably estimated for a single moving cell is ${{v}_{x}}=\left({{D}_{\text{min}}}/2\right)\centerdot f=3\centerdot 25\, {\rm\mu}$m ${{\text{s}}^{-1}}=75\, {\rm\mu}$m s⁻¹, that is largely below the range of interest of the observed capillaries. However, in practical situations, the particles are unequally spaced from each other, providing some sort of 'contextual information'. Therefore, the global matching algorithm has a large probability of properly tracking the same particle also at higher speeds. Numerical results indicate that we obtain a correct matching of RBC detections for speeds up to 300 μm s⁻¹, provided that cells are spaced enough. At higher speeds, we observe a few mismatches between tracks and particles, where aliasing occurs, leading to estimates that are significantly lower than the actual velocity. The strategy adopted for the rejection of outliers, applied before the estimation of the average RBC speed, reduces this kind of artefacts.

In 3 out of 50 cases the reported data indicate a rather large disagreement between the tracker and the STD-based manual estimates. A retrospective analysis of the STDs revealed two distinct situations. Samples labelled as B and C (figure 9) correspond to very noisy images, respectively shown in figures 10 and 11. In these cases, the average speed detected by the tracker (represented by the white line) may still constitute a reasonable estimate of the average RBC flow velocity. However, especially for sample B, it is difficult to assess the actual slope of the STD pattern, as it appears too blurred to allow a reliable connection of the traces produced by the RBC flow. The STD associated to sample A is shown in figure 12. RBCs velocity is clearly variable among different parts of the diagram, as indicated by the changes in the slope of the visible lines. Therefore, defining a single speed estimate is almost meaningless, and the calculated average speed cannot fit well the heterogeneous slope of the line pattern. In this case, a more detailed description of the histogram of the particles speed could possibly provide valuable clinical data. According to this discussion, the exclusion of the outliers from the data set is meaningful and leads to a sensible improvement of the algorithm performance. Indeed, after the exclusion of the three outliers, the correlation factor becomes r  =  0.978, and the regression line is y  =  mx  +  p  =  1.01x  −  4.68, meaning an almost perfect agreement with the manually estimated values.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The proposed automatic system is able to provide a quantitative assessment of the erythrocyte flow speed, from videos of the sublingual microcirculation captured at the bedside with incident dark field illumination microscopes. With respect to our previous work described in Sorelli et al (2015), the current system includes improved video enhancement and segmentation algorithms that reduce the number of misdetections, and a more complete motion model underlying the Kalman filter, thus yielding an extended range of reliable speed estimation. Moreover, tracks post-processing effectively limits the number of spurious results that are included in the evaluation of the average speed, thus improving the robustness of the estimates.

The correlation between the calculated velocities and those manually estimated from the STDs, indicates that the Kalman filter-based video-tracking technique is able to assess erythrocyte flow velocities up to 300 μm s⁻¹. The method, however, presents some degree of instability when RBC speed is higher than the reported limit. The major cause of this limited performance is certainly the current video frame rate of 25 Hz. Most errors are related to issues arising in the matching of tracks and RBC detections; therefore, tracking robustness may reasonably benefit from the higher frame rates of the latest available versions of microscopes. The image acquisition rate of Cytocam, in particular, is expected to increase to 60 Hz in the next future, and this improvement could allow to expand the range of measurable erythrocyte velocities up to about 600 μm s⁻¹.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.