Andreas Link,
Irene Luna Pardo,
Bernd Porr and
Thomas Franke*
Division of Biomedical Engineering, School of Engineering, University of Glasgow, Oakfield Avenue, G12 8LT Glasgow, UK. E-mail: Thomas.franke@glasgow.ac.uk
First published on 28th September 2023
The flow dynamics of red blood cells in vivo in blood capillaries and in vitro in microfluidic channels is complex. Cells can obtain different shapes such as discoid, parachute, slipper-like shapes and various intermediate states depending on flow conditions and their viscoelastic properties. We use artificial intelligence based analysis of red blood cells (RBCs) in an oscillating microchannel to distinguish healthy red blood cells from red blood cells treated with formaldehyde to chemically modify their viscoelastic behavior. We used TensorFlow to train and validate a deep learning model and achieved a testing accuracy of over 97%. This method is a first step to a non-invasive, label-free characterization of diseased red blood cells and will be useful for diagnostic purposes in haematology labs. This method provides quantitative data on the number of affected cells based on single cell classification.
Based on viscoelastic models the experimentally observed cells could be reproduced and variation of mechanical parameters (viscoelastic moduli) were determined has also demonstrated that transitions in between these cell shapes can occur. However, in experimental settings it is still challenging to distinguish between populations of red blood cells from different samples, such as diseased cells and healthy cells.
In AI based red blood cell classification most of the prior work11–13 distinguished between populations of cells with the associated problems mentioned above and in addition poses technical problems when employing segmentation techniques to isolate individual blood cells which are error prone and computationally intensive. Because there is a strong interest in the viscoelastic properties of the red blood cells often only their shape has been analysed but not the healthiness of the cells.14,15 The use of a channel to probe the mechanical properties of the red blood cells in conjunction with AI to detect a disease has, to our knowledge, only attempted once.16
Here, we demonstrate an AI based image analysis using TensorFlow that can decide between native, untreated red blood cells and red blood cells with chemically altered mechanical properties to mimic disease.17–24 Unlike the other studies mentioned before, we do not analyse the contour or shape of the cell to model the cell mechanics but train a TensorFlow implemented AI model with a large number of images from native and treated cells. The model is trained for 10 epochs and achieves a high training and validation accuracy already after ∼5 epochs, making is very efficient. Experimentally, we use a microchannel with oscillating width to transiently deform cells to capture both, the elastic, and the viscous properties. Our device presents a method that can distinguish a population of cells which are not obvious by simple inspection by eye. It is ready to be used for disease diagnosis and analysis of the severity of a disease by providing quantitative results on the frequency of affected cells and enables an end-to-end classification of red blood cells without understanding the biomechanical differences between the classified cells in detail.
As an example, we have shown one soft trigger when the cell in the red square has just arrived at a narrow section and has touched the soft trigger. A 2nd soft trigger has also been established (not shown) when the cell has arrived at a wide section and thus, we have two soft triggers: one for taking a snapshot in the narrow section and one for the section. Once the cell has been detected a snapshot is taken of the size of the crop window indicated with “crop” in Fig. 2A. As a next step the background was obtained (Fig. 2B) and subtracted (Fig. 2C) to prevent TensorFlow from learning features of the background instead of that of the red blood cells. Since the cell flows through the image from left to right, we can take the background for the left half from the final frames of a clip and the background from the right half from the first frames of the clip. Splicing these two halves together gives the background which can then be used in Fig. 2C to obtain a background-free cell image.
After training the TensorFlow model was presented with test data which the network has not yet seen: 492 clips of native red blood cells and 111 clips of chemically modified cells. The testing accuracy at the narrow section was 98% and for the wide one 97%. Fig. 3C/D show the testing results for the dataset generated at the narrow section of the channel and Fig. 3E/F at the wide section of the channel. Every row shows three examples of red blood cells and their detection probability against the ground truth: true native or true chemically modified. Remember that training forces the network to clearly output a one for its detection category. For example, in Fig. 3C the ground truth is “native” and the network outputs a one for native, indicated as “N” under the green bar. Then in Fig. 3D the cells were chemically modified, and the network classified them all with very high confidence as chemically modified (“C”). Fig. 3E/F now shows the same as above but for the wide section of the channel. Again, true native ones are detected as native without any doubt and reflects the 97% of the testing accuracy. The truly chemically modified ones were also detected with ease. Having just two categories being native or chemically modified allows a simple comparison which is robust with one category being almost zero and the other almost one.
We punch holed into the cured PDMS mould to connect inlet and outlet to the tubing. The PDMS is covalently bonded onto a microscope slide using oxygen plasma.
For the native red blood cell sample, we used 10 μL of concentrated RBC pellet from the bottom of the reaction tube and incubated in 0.5 mL green-fluorescent calcein-AM solution (5 μM, Invitrogen) for 30 minutes at 37 °C, followed by three washes with PBS.
For the chemically modified RBC experiments, we first incubate the red blood cells in formaldehyde solution before we stain the cells. Briefly, 10 μL of the pellet were incubated in 0.37% formaldehyde solution (Sigma-Aldrich) for 10 min at room temperature. The cell suspension was then washed three times before stained in calcein-AM as described above.
To avoid cell sedimentation during the experiment, the cells were suspended in a density-matched solution using a density gradient solution (OptiPrep Density Gradient Medium, Sigma Life Science). Optiprep is a sterile non-ionic solution of 60% (w/v) iodixanol in water. Additionally, bovine serum albumin (BSA, Ameresco) was used to prevent cell adhesion to each other and the microchannel walls. The stock density matched solution was prepared by mixing 945 μL of OptiPrep solution with 3035 μL of PBS and 40 mg BSA.
For the experiments 5 μL of the native or chemically modified cell suspension were mixed with 995 μL of the stock density matched solution and used as the stock solution with a haematocrit of 0.5%. The experiments were completed within a day of blood collection. The final density of this solution was ρ = 1.080 g mL−1 with an osmolarity of cosm = 317 mOsm L−1. The viscosity of this solution was determined after Ubbelohde to be η = 1.5 ± 0.1 mPa s.
The red blood cell suspension was pumped by a pressure driven flow using a pressure transducer (MarshBellofram, USA) and a pressure of p = 20 mbar at cell velocities in range from 1.3–8 mm s−1. Due to the Poiseuille-like flow cells flowing close to the wall are slower than cells in the centre of the channel. The average velocity of the red blood cells measured is vRBC =5.1 ± 2.0 mm s−1. The flow velocity is not constant and does change along the channel because of the alternating width and across the channel due to the Poiseuille-profile. Similarly, the flow rate is variable. However, to estimate typical shear forces on the cell we estimate an effective shear rate from an average channel width of wav = 15 μm and the average velocity to be eff = 2vRBCwav = 680 s−1. From the shear rate and the viscosity of the density matched solution η = 1.5 mPa s we estimate an effective shear stress to be τeff = η × =1.0 Pa.
To detect the cells at these finishing lines, we used OpenCV background subtractor, a Gaussian Mixture-based background/foreground segmentation, with a history of 100 and varThreshold of 20. We exclusively applied this only to the specific detection area which was determined by the finishing line parameter we had specified. The detection area was defined by a y range of (10110) and an x range from the finishing line to the finishing line plus 250. Then, we used OpenCV function “findcontours” to create contours around the moving cells in that detection area. We defined an area threshold of 20 px2 enclosed by these contours to identify a cell. If a cell passes the detection area and is above the set threshold the frame number is recorded and an image of the cell is extracted. In this way the function “findcontours” is only used to determine the soft-trigger as shown in Fig. 2 and no information on the contour is used in the AI categorization model.
The resulting images were cropped to a size of 120 × 150 pixels, with a cropping range of 0 to 120 for height and from the finishing line minus 80 to the finishing line plus 70 for the width. Therefore, images were of a shape of (120,150,3).
For our neural network, we used Keras sequential model with 10 layers. The first layer is used to rescale and normalize the input data. In the second layer a 2D convolution layer is used with 16 filters to detect different features in the input image. The next layer, the Maxpooling2D layer, reduces the spatial dimensions of the data, controlling overfitting and reduces the computational load. In the fourth layer another 2D convolutional layer is used but with more filters as before, with 64 as the number of output filters. Then another Maxpooling2D layer to reduce the dimensions of the data is used. To prepare the data for the upcoming dense layers the multidimensional output from the previous layers is flattened into a one-dimensional vector. For the two dense layers we used first 128 units per neuron to learn the complex relationship between the flattened features followed by the last layer, another dense layer with two units representing the output classes: “native” and “chemically modified”. To extract all pixel information from the images we used in the 2D convolutional layers a stride parameter of (1,1) and a kernel size of (3,3). Finally for the training, we passed through the entire training dataset ten times (10 epochs) in batch sizes of 32 samples where after each batch the gradients were updated. A schematic representation of the network is shown in Fig. 2D.
In AFM studies an increase in elastic modulus depending on GA concentration has been reported as well as a reduction in deformability (elongation index).23 AFM measurements using formalin at a concentration of 5% have revealed a 10-fold increase in Young modulus as compared to untreated cells.28 The difference in stiffness leads to less deformation and a lower elongation index as obtained from a shape analysis.29
Stiffening of RBC membrane has also been achieved by incubation in diamide.24,30,31 However, diamide has been reported to only rigidify the membrane stiffness and fluidity. It provides disulphide bridges between specific thiol-group containing amino-acids (cystine, methionine) and has a minor effect on the cytosol. Its effect has been studied in microflow analysing the shape changes by using the cell width to length ratio.32
In our experiments, the changes in viscoelastic properties due to chemical treatment with formalin 0.3% are expected to affect the morphology of the RBC in flow. However, we could not detect any apparent shape changes in our micrographs by eye or simple analysis such as analysing the deformation index or projected area.
Instead of analysing the RBC shape and contour as we have done in previous studies,33,34 here we have taken an AI based approach.35
A few other authors have used AI based analysis in microfluidics channels. Lamoureux et al.36 used a microfluidic ratchet channel to sort highly deformable RBCs and trained the machine learning on the isolated cells. Darrin et al.44 have used millimetre sized channels and observed red blood cells near the wall at shear rates of 10 s−1 to study the motion from tumbling to tank treading. The channel size used by Recktenwald et al.37 and Kihm14 et al. have similar dimensions as our geometry and both are operating in the same velocity range of up to 10 mm s−1. However, they use a straight channel and different flow rates to investigate the shape transition of individual red blood cells and therefore their device is mainly sensitive to elastic parameters.
Alkrimi et al.15 have classified RBCs using machine learning principal component analysis to reduce the correlation of features. Yet, the morphological analysis was done on a blood smear and not involving microfluidic channels.
Using a TensorFlow based machine learning we categorize native and chemically treated cells with high precision and with less training data required because we use a pretrained model.
This goes beyond current literature that probe red blood cells in simple capillary flow.16
We examined how the position of cells within an oscillating microchannel affects the outcome of the categorization. Two sets of red blood cell images were used, one in the channel's wider section and the other in the narrower section. Using TensorFlow, an image classifier with two outputs was trained and its objective was to generate one output close to one while maintaining the other output close to zero, thus, effectively distinguishing between “native” and “chemically modified” cells. The training accuracy for images of both section in the microchannel exceeded 99% after only five epochs, demonstrating the fast and efficient training convergence of the model. Similarly, the validation accuracy converges rapidly as shown in Fig. 3A and B. As a second indicator we used the training and validation loss, a parameter that quantifies how bad the prediction of the model is (penalty for poor prediction). Both indicators rapidly converge to small value indicating a very good prediction of the model. In both, the narrow and wide part of the channel, the validation and training accuracies and loss are monotonous increasing and decreasing, respectively and the fluctuation of their values with epochs is small. The value of all indicators plateau within about 5 epochs, again showing a very efficient categorization.
The trained model was then tested with previously unseen data of native and chemically modified cells and yielded a 98% accuracy rate for the narrow section and 97% for the wide section. Apparently, the binary categorization of cells does only weakly depend on the position of the cell and suggests that the selection of the position is not critical to the prediction of the image-based AI model.
Central to red blood cell diagnostics is the analysis of their deformability and, thus, their ability to form different shapes.16,36 For example, Kihm et al. divided the cells in two groups called “Slippers” and “Croissants”,14 implemented a CNN (Convolutional Neural Network) to train 4000 images and classify the RBCs uniquely based on their shape characteristics. Considering the increasing demand for advancements and the potential for significant impact and popularity in this field, Recktenwald et al.37 and its follow-up study38 adopted the approach, proposed by Kihm et al. and Alkrimi et al.,14,15 to benchmark different AI techniques classifying RBCs and similarly to Kihm et al.,14 the classification was based fully on morphology. Lee et al.39 uses not just shape but also texture features to classify normal and abnormal RBCs and similarly to Das et al.,12 they classify cells in more than two categories. However, overall, these studies employ an intermediate step by first or exclusively focussing on morphology and then feeding the pre-processed data into the final classifier for diagnosis. However, this assumes that one knows which deformation, shape or texture relates to a diseased cell and which one to a healthy one. In contrast, we do pixel-to-disease-classification (i.e., end to end) where the deep net learns the distinguishing features by itself without 1st hand-crafting features and only then feeding them into a classifier.
Often raw microscopic images taken contain a large amount of red blood cells which requires a segmentation process to extract individual images.16,39 Due to the segmentation processes and individual image extraction, the quality of the image is often seen reduced and as a consequence degrades the classification accuracy. Other approaches to work with images of many red blood cells are the Circular Hough transform40 or region-based segmentation (ORBS).13 Das et al.12 also studies segmentation of RBCs in image classification and similarly to the other studies40 and the study of Shemona et al.,13 the images analysed also contain many RBCs. In contrast to these studies our work directly captures single red blood cells passing through the microchannel and is not affected by image degradation, low resolution after cropping or wrong segmentation.
AI can be used to classify RBCs into more than two categories,12,39 for example Malaria, Thalassaemia, other abnormal and normal.11 In our study we have used two categories as a proof of concept but the TensorFlow classifier can take any number of classes where the only limit is the available computing power and GPU memory. Future work will use more than two categories.
Confounding factors are a significant challenge in any AI based learning algorithm which is at risk of learning just trivial and superficial features from the training data. For instance, Zech et al. explains that CNNs may not effectively identify disease-specific finings but rather exploit confounding information.41 This crucial issue is only addressed by Rizzuto et al. which tries to eliminate any confounding structures cropping the video to extract only a limited area of interest.16 No study has explored the effect of confounding factors such as focal plane on image classification of RBCs. In this study we discovered that AI would use the focal plane to distinguish between native and chemically modified RBCs when using one focal plane for the native ones and one for the diseased ones. To overcome this problem, we mixed images captured at different focal planes and, in addition, removed the background from the images.
Footnote |
† Electronic supplementary information (ESI) available: Supplemental figure of microfluidic channel with dimensions and description and location of the Python code with companion files. See DOI: https://doi.org/10.1039/d3ra04644c |
This journal is © The Royal Society of Chemistry 2023 |