Tetiana
Orlova
a,
Anastasiia
Piven
a,
Darina
Darmoroz
a,
Timur
Aliev
a,
Tamer Mahmoud
Tamer Abdel Razik
a,
Anton
Boitsev
b,
Natalia
Grafeeva
b and
Ekaterina
Skorb
*a
aInfochemistry Scientific Center, ITMO University, 9 Lomonosova Street, Saint-Petersburg 191002, Russia. E-mail: skorb@itmo.ru; Tel: +7 999 210 39 77
bThe Higher School of Digital Culture, ITMO University, 9 Lomonosova Street, Saint-Petersburg 191002, Russia. E-mail: dc@itmo.ru; Tel: +7 921 997 97 91
First published on 20th February 2023
This review discusses three types of soft matter and liquid molecular materials, namely hydrogels, liquid crystals and gas bubbles in liquids, which are explored with an emergent machine learning approach. We summarize specific examples of the use of machine learning technique to study the structure and properties of soft matter at the molecular, microscopic and macroscopic levels. The approaches of artificial intelligence have greatly improved the prediction of material properties, stimulated the progress in modeling methodologies capable of revealing physical phenomena, and opened up new perspectives in the design and use of soft material devices. For this reason we also provide guidance on machine learning methods and recommendations on best practices for data understanding.
The field of soft and liquid molecular materials can also be such a playground (Fig. 1). In this article, we consider the main categories of ML problems and algorithms and their applicability in the field. Among the tasks to be solved by ML methods, several categories are particularly in demand. Let us briefly introduce them to you.
The purpose of dimensionality reduction1 is as follows. A large number of data features are reduced to fewer for future data visualization or application of other ML algorithms in real time. The anomaly detection2 is also a very important field. Atypical objects (which are found in any industry) are separated from the usual (standard) ones based on various statistical characteristics of the entire sample or other ML algorithms. The regression3 is the task of predicting the possible characteristics of new objects based on a sample of existing objects with a set of features, while the classification4 is the task of assigning objects to predefined groups based on a set of features (there can be 2 or more classes). Finally, the clustering5 is the task of dividing the analyzed objects into an unknown number of groups based on a set of features, highlighting the structure in the data.
The tasks mentioned above are solved by various methods, but perhaps the most well-known in this field are the following.
The Principal component method6 (PCA) is basically an orthogonal transformation that allows to translate observations of interrelated variables into a set of principal components or linearly uncorrelated values. PCA is used to provide visualization of the source data, as well as to minimize them and facilitate the learning process itself.
The Nearest neighbor method7 (k-NN) is a very popular classification method, sometimes used in regression problems. This is one of the most intuitive approaches to classification. The essence of the method is as follows: new objects are added to objects already divided into classes according to the principle of “the nearest neighbor”. Hence, the new object falls into the class that is closer to it by attributes. The distance (proximity) between neighbors is determined by various metrics.
Decision trees8 are a method that implements decision–making based on the use of a tree graph. Such a tree is formed from the minimum possible number of questions with an unambiguous answer (either “yes” or “no”) based on existing labeled data. After entering the sequence of answers, the user comes to the right choice. As a rule, the method is used in classification tasks.
The Support Vector Machine9 (SVM) is an algorithm that is actively used to solve classification problems. The main idea of the algorithm is iterative partitioning of N-dimensional objects by hyperplane of dimension (N – 1).
Clustering algorithms5 deal with the distribution of analyzed objects into clusters, in which similar elements should appear. Various algorithms based on probability, density, dimension reduction, etc. are used for clustering.
Neural networks10 are an apparatus based on a mathematical model of the interaction of neurons in the human brain. To apply the model, the network is pre-trained on the basis of existing “marked up” data. Currently, the neural networks approach is one of the most popular branches of machine learning. Unlike classical ML algorithms, their advantage is that complex and important work on the formation of features before applying the algorithm (feature extraction) is alienated from the researcher and left to the algorithm.
In the next section of the article, we focus on the machine learning methods already applied in liquid molecular materials tasks and to what extent this application is justified.
Recently, there has been increased interest in hydrogel because of its unique properties that enable it to spread in several applications. Additionally, numerous books and articles on hydrogel materials have been published over the past few decades and offer more comprehensive and varied perspectives. There has been a fast growth of novel hydrogel materials and correlated research between 2000 and 2021, which has not been reviewed frequently. Fig. 1b shows a recent number of publications, including hydrogel and machine learning.
The significantly increased ability of materials scientists to collect, share, and analyze large volumes of data in recent decades has led to what many are calling the fourth paradigm of materials science, also known as data-driven materials discovery or materials informatics (MI).15,16
MI is a discipline in which correlation relationships can be suggested or validated by examining massive data sets of materials using statistical techniques, many of which use machine learning. An informatics strategy uses these enormous data sets to alter more standard correlation relationship development methods. For example, a more traditional technique might utilize a theoretical model based on our understanding of chemistry and physics to predict a material's attributes based on its structure and composition. Experimental measurements of the structures and expected properties can then be used to validate or alter the theoretical model. Instead, an informatics method develops a model using data from the inputs (structure) and the measured responses (properties). This model can then predict responses to similar or only slightly different inputs, and cross-validation can test and improve its ability to do this.
A wide range of technological applications are enabled by functional hydrogels, which are made up of crosslinked 3D macromolecules and molecular building blocks that self-organize into complex structures as a result of their adjustable connections. Inverse approaches allow designers to navigate their intrinsically high-dimensional design areas in order to generate materials with specific qualities. While a number of physically driven inverse techniques have been effectively applied in some circumstances, their application to directing experimental materials discovery has been confined to a few proof-of-concept investigations thus far.17 We highlight recent improvements in inverse methods for hydrogel design that address two issues: (1) methodological limits that prevent such approaches from satisfying design requirements and (2) computing challenges that limit the pore size and network structure that may be addressed. Methods to identify order parameters that characterize complicated structural motifs, as well as approaches to effectively compute macroscopic features from the underlying structure, have proven to be particularly effective. We also talk about promising ways to improve the accuracy and computational efficiency of models that are relevant to experiments, such as finding materials that work in more than one thermodynamic state, making protocols for externally directed assembly that are easy to use in experiments, and coming up with other ways to improve the accuracy and computational efficiency of models that are relevant to experiments.
Drug delivery systems,18 wound dressing membranes,19 cell culture,20 tissue engineering21 and other critical applications have all benefited from tailored hydrogels. The use-inspired behaviors of these materials are caused by the physicochemical properties of their constituent components and their internal spatial organization (i.e., structure). Synthetic polymers, polysaccharides, and proteins can serve as powerful material building blocks for hydrogel formation because their mutual interactions, which help determine the system's favored equilibrium state, can be systematically varied through, for example, their size, shape, charge, composition/sequence, and surface functionalization.
This opens up many design possibilities. It is a big job to determine which building components may consistently self-assemble a material with a given structure or desired macroscopic features. Innovative approaches to the search for new self-assembling materials are routinely used. In such approaches, an initial set of material building blocks is synthesized, and strategies are developed to make self-assembly easier in an experiment or a computer simulation. The structure and qualities of the final material are next investigated. These processes are repeated (typically numerous times) to hunt for materials with superior attributes using other building blocks or protocols. Instead of framing this process as an inverse problem, framing it as a methodology and amenable to meeting stated design objectives can be advantageous. For example, a figure of merit (FOM) can be set up based on the desired structure or macroscopic property, and constrained optimization methods can move through the multidimensional design space and determine which building blocks, interactions, or protocols are best for making material.
Kalasin and his coworkers developed The Remote Artificial Intelligence-Assisted Epidermal Wearable Sensing for Environmental Heat-Stress Sweat Creatinine Monitoring based on Satellite-Based Sensor. His group develop coated nylon with poly(vinyl alcohol) (PVA)-Cu2+-poly(3,4-ethylenedioxythiophene) polystyrene sulfonate (PEDOT:PSS) and cuprous oxide nanoparticles. The system was equipped with heart rate monitoring and a satellite communication device to locate wearers, and incorporates machine learning to predict the levels of environmental heat stress. Electrochemical impedance spectroscopy (EIS) was used to investigate different charge-transfer resistances of PVA and PEDOT:PSS with cuprous and cuprite ions induced by single-chain and ionic cross-linking.22
To improve the performance of ultrasound-mediated chemical sensing using titanium dioxide (TiO2) nanoparticles-embedded hydrogel, Islam and his colleagues used machine learning. A new wireless pH sensing technology has been developed using ultrasound transmission through titanium dioxide (TiO2) nanoparticle-embedded hydrogels. An ultrasonic transmitter/receiver pair and a TiO2-embedded hydrogel implanted under the skin make up the sensing system (or any other body area that requires pH monitoring). Filling hydrogel with TiO2 nanoparticles improves ultrasonic wave backscattering, obviating the need for complex readout devices like ultrasound imaging. The physical behavior of ultrasonic waves changes as they flow through the hydrogel, depending on the thickness of the hydrogel. The ultrasonic receiver, located on the other side of the body, captures the changed ultrasonic wave caused by the hydrogel. Ultrasound activities were used to investigate the volumetric transition of hydrogel wirelessly. The delivered ultrasonic signals are gathered with feature extraction for machine learning applications in mind. This approach uses ultrasonic waves to measure pH information with low reflection and noise effects. Despite having many scopes, no machine learning-enabled direct pH measurement systems have been published previously. The measurement errors of today's state-of-the-art pH sensors (with analyte) are within 0.1 to 0.01 pH. So, this research aims to find ways to use machine learning to make the error rate even lower.23
For 3D printed bioink, Lee and his colleagues use a machine learning-based design technique. They used a model system using naturally produced biomaterials to build a machine learning-based method for designing a 3D-printable bioink. First, they showed that, compared to native collagen, atelocollagen (AC) had better physical qualities for printing (NC). NC gel had highly crosslinked and temperature-responsive irreversible behavior, resulting in brittleness and high yield stress. In contrast, AC gel had weakly elastic and temperature-responsive reversible activity, generating a soft cream-like structure with low yield stress. Then, using machine learning, they identified a universal relationship between the mechanical parameters of ink and printability: a high elastic modulus increases shape fidelity, and extrusion is possible below the threshold yield stress. Based on this relationship, they were able to use multiple regression analysis to make a lot of different formulations of bio-inks made from natural materials with great shape fidelity. Finally, using a framework of high form fidelity bioink, a 3D model of a cell-laden hydrogel was created, revealing that the cells are incredibly viable and proliferative in the 3D structures.24
Liu and his colleagues created a system based on photography and machine learning, which they used to hydrogel pressure distribution sensors. The team proposed and built a hydrogel pressure distribution sensor that can monitor pressure distribution across the entire hydrogel component. This is performed via a technique called electrical impedance tomography (EIT), which involves putting electrodes just around the hydrogel (EIT) (Fig. 2). Meanwhile, PAAm/PAA-Fe3+ double-network hydrogels were developed as hydrogel pressure-sensitive substrates, and mechanical and electrical tests confirmed their suitability as sensitive elements for hydrogel pressure distribution sensors. A machine learning method based on the hydrogel pressure distribution sensor was also used to create a pressure distribution reconstruction model. Finally, the hydrogel pressure distribution sensor was used to test the viability of the EIT strategy-based hydrogel pressure distribution sensor by applying forces of known position and magnitude. The real sensor data was then reassembled and compared to the applied force.25
Fig. 2 (a) Hydrogel pressure distribution sensor consisting of two parts, an electrode array board where the hydrogel can be placed, and a driver board, tested with a force gauge. (b) Predicted maps of plantar pressure distribution during the simulated walking process, starting with heel under pressure, then with whole foot under pressure, next with main force on the forefoot, and finally with big toe under pressure while the foot is lifted. (c) The maximum predicted impedance value of the compressed area (MPICA) shows a linear correlation with the actual pressure applied to the sensor. All figures are adapted with permission from ref. 25. |
Li and his co-authors use chemical characteristics to examine the design of self-assembly dipeptide hydrogels and machine learning (Fig. 3). They built a peptide-like chemical library for screening chemicals that can produce hydrogels based on a Ugi four-component reaction. A rheometer and transmission electron microscopy (TEM) evaluated selected hydrogels, which were then grown with an adherent cell line. The machine learning method was designed to recognize these chemical properties and forecast whether a chemical structure may form a hydrogel at neutral pH without any divalent or trivalent metal ions. In addition, the molecular structure and gelation property connection was summarized.26
Fig. 3 (a) A typical method of peptide-based hydrogel preparation and the question of hydrogel formation as a binary classification problem. (b) The precision–recall (PR) and receiver operating characteristic (ROC) curves obtained for the random forest and gradient boosting models of hydrogel formation. All figures reprinted with permission from ref. 26. |
The researchers examined several polysaccharide hydrogels to find functional properties that could predict antibiotic accumulation in Gram-negative bacteria. A model composed of starch hydrogel was evaluated in the Gram-negative model bacterium E. coli and shown to be exceptionally capable of discriminating high from low-accumulating antibiotics. The rapid penetration of porin-dependent antibiotics in the starch gel matches Gram-negative specific porin-mediated absorption. However, findings on nalidixic acid permeation and structure-permeability connections suggest that this approach can also discover high-accumulating medicines with good porin-independent outer membrane penetration. Whether manually pipetted or printed, model preparation is simple, reproducible, cost-effective, and risk-free. Membrane-permeation tests can be done automatically and give accurate results in as little as 10 minutes. This makes them great for high-throughput screening of compounds with different physical and chemical properties.
Modern machine learning approaches to in vitro data offer evidence of the influence of previously revealed molecular characteristics in bacteria. Based on in vitro permeation data, it was determined that a small set of seven features was all that was needed to build a reliable machine learning model that predicted well. In testing the ability of different medicines to pass through cells in a lab, the first evidence of bacterial accumulation of aminoglycosides and sulfonamides, which are essential antibiotics for treating Gram-negative infections, was found. By optimizing the composition of the alginate formulation or using biological hydrogels, the experiment could be changed to study the permeability of biofilms, exopolysaccharides, or mucus on a larger scale.27
Fig. 4 (a) Catechol-chitosan-diatom hydrogel as triboelectric nanogenerators for collecting energy from human movements to monitor the health of a person with Parkinson's disease using machine learning methods. Reprinted with permission from ref. 29. (b) Predictive cell-state classification model based on computed quantitative 3D nuclear metrics for splicing factor SC-35. Reprinted with permission from ref. 31. (c) Using the random forest algorithm to determine the presence of encephalitis antigen in an electrochemical system. Reprinted with permission from ref. 32. |
Hydrogels are being used in conjunction with machine learning for biological stem cell research. A group of scientists from Rutgers University created a system to analyze the high content of the true three-dimensional organization of SC-35 cells with machine learning approaches to classify the resulting cell states when cells are cultured in three-dimensional scaffolds. This system makes it possible to study cells without destroying them, since PCR, flow cytometry and immunoassays are time consuming and require cells to be extracted from the system (differentiation, apoptosis, transformation). Decision tree models for three types of cells were trained in the WEKA software package.30 Each of the models showed an accuracy above 70% with an average error of no more than 0.4 (ref. 31) (Fig. 4b).
In an article by a research group from the ITMO University's ISC Infochemistry Scientific Center, hydrogels were used to collect a database using cyclic voltammetry. The obtained data was used to create a random forest model in the WEKA software package.30 Four different hydrogels were used with different combinations of encephalitis antibody and antigen. The accuracy of the model was 93%32 (Fig. 4c).
Hierarchical machine learning with small datasets has made it possible to create a pipeline that allows to analyze and predict the parameters required for the 3D printing method of soft hydrogel molds – freeform reversible embedding of suspended hydrogel (FRESH).33 During the study, 48 seals were analyzed. They were analyzed based on a statistical comparison of CAD files and, obtained from them, real samples. LASSO regression with the parametrization of the middle layer to the upper one showed good values of the coefficient of determination R2 = 0.643 34 (Fig. 5a).
Fig. 5 (a) Hierarchical machine learning to predict the best parameters for 3D printing of elements from alginate hydrogel. Reprinted with permission from ref. 34. (b) Use of support vector machine for binding energy prediction in supramolecular chemistry with hydrogel depot release. Reprinted from ref. 37 with permission from the Royal Society of Chemistry. (c) The use of hydrogels based on collagen, fibrils and hyaluronic acid for the development of 3D biofilms using machine learning methods. Reprinted with permission from ref. 38. |
Hydrogels are used in supramolecular chemistry to study and analyze the process of release of supramolecular assemblies. The support vector machine was used to predict the binding energies of supramolecular assemblies with various molecules. Supervised learning was applied to create classification and regression models. In the training dataset, the data obtained by the DFT method were used, such as: geometry from optimized orientation, eigenvalues condensed to atoms, all electrons, condensed to atoms, geometry accompanying electronic data, electrostatic properties, electric field gradient, gradient eigenvalues. Also in the dataset, environmental data was used, such as: temperature, buffer and salt concentration and pH. In the course of the study, it was possible to train models that showed their effectiveness by correctly predicting the binding energy of cucurbit[7]uril with promising drugs against low-grade gliomas in children: the RAF type II inhibitor TAK-580 (ref. 35) and the MEK inhibitor selumetinib.36 The models predicted that RAF would have a high binding energy and MEK would have a low binding energy, which was confirmed experimentally. RAF and MEK were incorporated into a hydrogel based on cucurbit[7]uril with the same release kinetics of both guest structures for the needs local drug delivery37 (Fig. 5b).
Hydrogels based on collagen, fibril and hyaluronic acid are considered for 3D printable bioinks. The correlation between rheological properties and printability was analyzed using the relative least general generalization algorithm. The input parameters were the concentrations of three initial components of the hydrogel. Two outputs were obtained. The first output is the rheological parameters, and the second output were printing results such as shape fidelity. Combining it with multiple regression analysis, Lee with co-authors showed operating window maps to determine printability of natural hydrogels38 (Fig. 5c).
Research is also using machine learning to automate the hydrogel manufacturing process. Hydrogels in which bacteria were introduced were used as an experiment. Differential dynamic microscopy was used to determine the gelation rate. For the selection of parameters, Bayesian methods of machine learning were used. A machine learning pipeline was obtained, which independently selected parameters for gelation, and after the experiment, estimated the speed and selected more efficient parameters.39
Various classification algorithms, such as multi-linear regression analysis and neural networks, were tested aiming to predict liquid-crystalline property from molecular structures of individual chemical compounds encoded by different sets of numerical descriptors along with the clearing temperatures.41–44 Despite the large number of compounds used for to train neural networks by Kränz with co-authors,41 the clearing temperatures were predicted by the best with a standard deviation of 13°, which is a rather high error, especially in the case of LC materials with a clearing temperature close to room temperature. Improvements in neural network model descriptors have reduced the RMS error to just a few degrees42 along with successful encoding of clearing temperature trends into homologous trends.43 A comparison of three classes of machine learning algorithms such as eager learners, lazy learners, and neural networks has shown that no heuristic is better than another on the set of all possible problems.44 All classification algorithms represented good methods for predicting the liquid-crystalline property, but their efficiency depended on a specific problem.
The quantitative structure–property relationship (QSPR) methodology, combined with a specific type of feed-forward artificial neural network, has been applied to predict the liquid crystallinity and phase transition temperature of bent-core molecules.48 (Fig. 6a) The authors turned to nonlinear QSPR models and for the first time used a group method of data handling type neural network, testing several machine learning models with different sets of molecular structure descriptors. The developed neural network models demonstrate an improvement in determining the clearing temperatures compared to the results obtained in a previous study.49 using the multivariate adaptive regression splines technique. Key structural features that affect the transition temperature of five-ring bent-core aromatic compounds have also been identified.
Fig. 6 (a) Phase transition temperature prediction for binary LC systems using machine learning methods (random forest, linear regression, ridge regression, elastic network regression, and support vector regression). Reprinted with permission from ref. 45. (b) Analysis of phase transitions in liquid crystal 12BBAA (4-bromobenzylidene-4′-dodecyloxyaniline) using the k-means cluster analysis of infrared spectra. Reprinted with permission from ref. 46. (c) Chemical structure of the two lipids and the twenty unsaturated and ten saturated fatty acids studied, where blue arrows indicate the location of the double bond in the hydrocarbon chains. Bottom part shows experimentally revealed (left) and predicted (right) phase diagrams for monoolein (top) and phytantriol (bottom) under the effect of different unsaturated fatty acids for samples. The mesophase type is indicated as the inverse bicontinuous diamond cubic (QDII), inverse bicontinuous primitive cubic (QPII), inverse hexagonal (HII), microemulsion (L2), micellar cubic (I2), lamellar crystal (Lc), and a combination of Lα+L2+L3 phases. Reprinted with permission from ref. 47. |
Recently, a general methodology was presented by Chen and coworkers50 to identify appropriate machine learning algorithms and molecular descriptors for predicting a wide variety of liquid crystal behavior of organic compounds based on QSPR. Almost a dozen machine learning algorithms were compared using a dataset from the LiqCryst 5.2 database51 with 3786 entries, of which 2780 compounds exhibit liquid crystal behavior. The most accurate for predicting the liquid-crystalline behavior was the random forest algorithm, while the molecular descriptor took into account the mesogen and wings of the chemical structure. Other advantages of the classifier included no pre-processing, quick training, simplicity and versatility for different descriptor inputs. This extensive study can serve as the basis for constructing a multipurpose QSPR model for each type of mesogenic molecule, including the prediction of desired LC properties for unknown or not yet synthesized compounds.
After a number of the above-mentioned studies predicting the properties of monodisperse molecular systems, a similar attempt was made for polydisperse systems.45 (Fig. 6b) A dissipative particle dynamics simulation method, developed specifically for soft matter and complex liquids,52 was used to analyse self-assemble structures and phase transitions in the binary LC system. The order parameter and the phase transition temperature were predicted using several different machine learning algorithms, among which the random forest method showed the highest predictive ability. This study is an important step forward since many of technologically significant liquid crystals are multi-component mixtures, for instance, the widely popular thermotropic nematic LC E7 (Merck).53
Lipid-based lyotropic LCs are of considerable interest as potential delivery systems for drugs and in vivo imaging contrast agents. Therefore, Le and Tran extended machine learning to predict the complex phase behavior of monoolein and phytantriol based lyotropic LC nanomaterials.54 Robust models were developed for seven different mesophases considering the effects of two types of lipids, 20 unsaturated fatty acids, 10 saturated fatty acids, a range of fatty acid/lipid ratios, and temperature. The phase behavior prediction was obtained with high accuracy using a Bayesian regularized artificial neural network. In addition, the developed models were able to interpolate data for the same fatty acids at temperatures that have not yet been tested, as well as extrapolate data for new lipid nanomaterials, thus elucidating rules that will be useful for the future development of advanced lipid systems for therapeutic delivery. It is worth noting that earlier Le with co-authors reported on the same Bayesian regularized neural network capable of predicting with high accuracy the complex non-stationary behaviour of amphiphilic nanostructured mesophases over time and under the influence of various crystallization screens.47
More than 50 new LC phases have been discovered during extensive studies of bent-shaped molecules over the last 20 years.55 Among them, one of the most fascinating is the twist-bend nematic phase, when the director follows an oblique helicoid at a constant oblique angle to the helix axis. Recently,56 it was demonstrated that a machine learning protocol can describe the helical trajectories of hard curved spherocylinders57 that form the twist-bend phase in dynamic Monte Carlo simulations. The pitch and radius of the trajectories of diffusing hard particles are determined by the pitch and conical angle of twist-bend nematic phase, thereby relating the structural and dynamic properties of this complexly ordered LC. Such studies are important not only for fundamental science, but also for the industry of LC-based devices, for instance, optoelectronic elements with switching times that are determined by the diffusion rate.
The machine learning-based analysis of molecular descriptors or molecular dynamics data is not the solely way to predict complex phase behaviour. For example, the cluster analysis method was applied to the temperature-dependent infrared spectra of a mesogenic chemical compound 12BBAA (4-bromobenzylidene-40-dodecyloxyaniline)46 (Fig. 6c). Changes in the FT-IR spectra are associated with the alkyloxy chain melting phenomena. Thus, phase transitions from an isotropic liquid to smectic A, crystalline smectic B, and a crystalline phase were successfully predicted from spectroscopy data on the characteristics of vibrational bands.
The group led by Ribeiro showed that machine learning methods are able to capture many fundamental characteristics of liquid crystals in an equilibrium state directly from optical images of LC textures.60,62,63 Convolutional neural networks (CNNs) and k-nearest neighbors algorithm trained on simulated optical images of nematics and cholesterics have successfully predicted the LC phase (nematic or isotropic), the order parameter, and the pitch of the cholesteric helix.60,62 (Fig. 7a) Furthermore, these algorithms demonstrated ability to predict the temperature of the nematic phase, the phase transition and its order, and the chiral dopant concentrations from experimentally obtained optical microscopy images(Fig. 7b). Newly,63 it has been shown that ordinary networks of only 24 nodes encode enough optical information that, when combined with a simple machine learning method, is enough to identify and classify mesophase transitions with high accuracy, determine concentrations of chiral molecular dopants, and predict sample temperature.
Fig. 7 (a) Prediction of the pitch length from numerically simulated optical images of the cholesteric LC phase with convolutional neural networks. Reprinted with permission from ref. 60. (b) Prediction of the sample temperature from experimental micro-photographs of E7 nematic LC textures. The network architecture is modified by including additional convolutional layers before each max-polling layer for high prediction accuracy. Reprinted with permission from ref. 60. (c) Extracting hydrodynamic parameters from a libraty of simulated LC director fields using a supervised neural network, and demonstrating the capabilities of a neural network as surrogate model of time evolution. Reprinted with permission from ref. 61. |
Another useful technique is to apply machine learning technique for estimating the macroscopic parameters of physical models describing complex LC states. This has been demonstrated for active nematic hydrodynamics.61,64 Neural networks predicted multiple hydrodynamic parameters using only movies of the director field (Fig. 7c) and forecasted the chaotic dynamics of these systems including point defect nucleation, splitting and annihilation.
An approach of analyzing spatiotemporal variations in the parameters of nonequilibrium systems, combined with machine learning methods, could explain the nucleation and behavior of dynamic supramolecular patterns in chiral nematics.65 As the authors suggested, the continuous rotation of chiral patterns in a photoactive LC under constant illumination with a focused ligh beam is sustained by the reaction-diffusion process of embedded light-driven chiral molecular motors coupled to the long-range director field due by molecular diffusion. Despite this hypothesis and the analysis of the rotation period on the size of chiral pattern, the rotation equation depending on the physicochemical parameters of the system has not been explicitly obtained. Machine learning methods could reveal hidden connections between the macroscopic organization of the director field and molecular photoisomerization along with diffusion, but this task remains to be explored.
Since all LCs are highly sensitive materials, the problem of analyzing sensor readings becomes especially important for chemical and biosensors, when precise agent detection at minimum concentration is crucial. Machine learning has great potential for filtering a useful optical signal caused by a change in the LC orientation structure under the action of a measured stimulus from random fluctuations in the director field or the influence of side physicochemical factors.68 In other words, it is necessary to accurately and quickly solve the problem of pattern recognition and classification, for which machine learning was originally developed.69
In response to the global COVID-19 pandemic, Xu with co-workers developed a SARS-CoV-2 point-of-care detection kit based on the response of LC films to femtomolar concentrations of single-stranded ribonucleic acid (ssRNA).70 For this purpose, the LC layer was decorated with a cationic surfactant and a complementary probe of 15-mer single-stranded deoxyribonucleic acid (ssDNA). The minimum concentration of SARS-CoV-2 RNA led to the ordering transition, causing changes in the optical response. Then micrographs were captured by a support vector machine for statistical classification into two categories, positive or negative. Furthermore, in order to obtain a reliable reading of test results for non-expert users, a smartphone application has been developed based on SVM(Fig. 8a).
Fig. 8 (a) SARS-CoV-2 point-of-care detector kit for smartphone app with a reliable readout of the test result by machine learning based algorithm when adsorption of SARS-CoV-2 RNA at the water-LC interface results in an optical response of the LC film. Reprinted with permission from ref. 70. (b) Textural changes of nematic LC-based hybrid gels and corresponding signals during VOC vapor exposure and subsequent recovery along with a collective accuracy prediction plot for 12 VOCs. Reprinted with permission from ref. 72. |
Roque with co-authors applied CNNs and support vector machines to recognize more than 10 different volatile organic compounds (VOCs) using LC sensors.71,72 (Fig. 8b) In various multicompartment gel films containing nematic or smectic droplets, variations in optical textures were registered due to orientational transitions of LC molecules in the presence of VOC vapors. Functional and structural differences in the selected VOCs were small. Depending on the LC material, VOC, and droplet diameter, the prediction accuracy ranged from moderate 50–60% to near 100%. The change in optical texture also depended on the concentration of VOCs, for example, the mean absolute error in determining the concentration of acetone was below 0.25%.
The research group led by Abbott and Zavala has made significant contributions to the field of machine learning methods for LC sensors.73–75 They developed LC-based sensors that exhibited optical responses to N2-water (30% relative humidity) and N2-DMMP (10 ppm) gaseous environments.73,74 AlexNet, the CNN used in their first study,73 showed a classified accuracy of 99% based on grayscale micrographs. However, such a high level of accuracy required the use of a large number of features, so the more compact VGG16-CNN using color micrographs was tested and demonstrated a perfect classification accuracy while the number of features was reduced to less than 100.74
The most original study by the group led by Abbott and Zavala is aimed at identifying bacterial agents and quantifying the concentration of bacterial endotoxins.75 Instead of taking optical microphotographs, the authors for the first time used flow cytometry with measuring the intensity of side-scattered and forward-scattered light by LC droplets. The ratio of these intensities depended on the droplet orientational structure, the specific changes in which the authors attributed with the molecular structure of the lipid A domain of bacterial endotoxins. Endonet-CNN predicted endotoxin sources directly by the ratio of scattered light intensities and estimated endotoxin concentrations over a wide range of eight orders of magnitude from 1 μg mL−1 to 0.01 pg mL−1.
Image formation is affected by each layer of the LCD. ITO electrode defects result in permanent bright dots or black pixels on the screen. The brightness of the backlight affects the sharpness and contrast of the image. Therefore, it is important for manufacturers to evaluate the quality of LCDs directly at the factories in automatic mode without inspection by human eyes and at high speed. Clearly, machine learning could serve as a highly efficient method for solving this task.
The early attempts to apply various ML algorithms to assess the quality of LCDs were made more than 10 years ago aiming to detect and classify defects of TFT-LCD arrays.77–80 Both assessments are crucial for LCD manufacturing, as identifying the type of defect allows the necessary corrective actions to be taken for its elimination and prevent future failures. In an early study,77 different types of defects were determined with an accuracy of about 86% to almost 93% using various ML algorithms such as a support vector machine-based classifier and a backpropagation neural network. The data of the analyzed samples were obtained from a real manufacture process. Later, it was shown that modified support vector data descriptions78,79 provide a high defect detection rate of 90–96% and capable of defect detection on an LCD image within 60 ms.78 A method that combines K-means clustering with a backpropagation neural network machine learning algorithm outperforms others in the specific task of predicting the heights of thin film transistor-liquid crystal display photo-spacers.80
ML methods, primarily neural networks, have also been tested to control the local backlight dimming, which is important for less loss of image detail, higher contrast ratio, and low power consumption of LCDs.81–83 A comparative study of local backlight dimming prediction accuracy applied to the subjective evaluation of video quality, impacted by ambient light exposure and peak white (maximum display brightness), revealed that Elastic Net algorithm performed best compared to partial least squares regression and support vector regression.81 The CNN-based algorithm made it possible to control the backlight intensity along with reducing the loss of detail while achieving a high contrast ratio by taking into account the diffusion property of light and leakage property of liquid crystal.82 However, this ML method required statistical information of pixel values in each local block. The local dimming algorithm based on the U-net convolutional network enabled the compensation of pixel data transferred to a panel directly from an input image, without any information about dimming levels of the backlight unit sub-blocks.83
Machine learning can be successfully used to predict the quality of a whole product through each process data.84 Usually, the overall assessment of product quality in industry is carried out by a sampling method that is not comprehensive and has no timeliness. The random support vector machine method, which combines the support vector machine and random forest, showed the mean square error 0.6 percent lower than that of random forest, despite the fact that the random forest is considered the best traditional machine learning algorithm. A comprehensive three-stage data science framework has also been developed, consisting of variable selection, metrology prediction, and process control.85 At each stage, different ML methods were used to identify the key factors, for instance, the decision tree, stepwise regression, and random forest. The proposed data science framework, applied to an empirical study of a leading TFT-LCD manufacturer, allowed to determine the variables affecting yield, predict the photo spacer thickness with higher performance than the company's method, and proposed the process control in the color filter manufacturing process. All this helps to reduce the cost for process monitoring and quality in TFT-LCD manufacturing.
Specific tasks, such as dead pixels and mura detection during manufacturing process were also adequately addressed by ML.86,87 Typically, such defects are detected by an operator, which is not always performed reliable and increases the cost of production. The support vector machine algorithm proved to be the most effective in detecting dead pixels with an accuracy of about 92% compared to random forest.86 In contrast, mura (a variation in local brightness with no distinct contour on a uniform LCD surface) was predicted using the random forest algorithm with a detection rate above 99% and a processing time of 27 ms per image, which is a competitive result for industrial systems.87
Parameters such as bubble size and shape statistics generally determine bubbly flow. Using the analysis of cavitation bubbles and machine learning methods, it is possible to predict the structure of the flow. A similar method is demonstrated in a number of articles. Using graph files as inputs, Gao et al. classified samples by flow structure via convolutional neural networks.88 Two convolutional neural networks were used to classify 6 flow structures and void fraction (Fig. 9a). The accuracy was more than 92%, the root mean square error was 0.0038 for the flow structure forecast.
Fig. 9 (a) Convolutional neural network architecture with two inputs for stream structure classification and void fraction measurement. Reprinted with permission from ref. 88. (b) Bubble analysis based on image pre-processing and convolution neural network. Reprinted with permission from ref. 89. (c) Image processing algorithm based on PCNN. Reprinted with permission from ref. 96. (d) Classification of alcohol concentration based on cavitation bubble images and CNN. Reprinted with permission from ref. 103. |
Not only graph files can be used as machine learning inputs. Bubble images are a common inputs to cavitation bubbles analysis. Even images of micro-size bubbles can be processed successfully. CNNs can be also used for microbubble analysis. Qaddoori et al. performed a similar approach to determine the size of microbubbles using pre-processed images.89 The images were preliminary converted into HSV format and then processed with CNN (Fig. 9b). According to this approach, the microbubble size was determined both by the number of pixels and luminosity intensity. The method showed 100% accuracy in microbubble size determination. Bubble' image analysis via CNNs also enables identification of the flow patterns.90,91
High-speed cameras are widely used to capture cavitation bubbles, but the resulting images are often of a low quality. Machine learning can also be used to improve defects of input images. Poletaev et al. used deep learning, specifically CNN, to build a model capable of detecting and tracing overlapping, blurry, and non-spherical cavitation bubbles, and demonstrated its further analysis.92 Various models and training parameters were performed, and the proposed approach was found to be effective, as the accuracy was about 97% on the test dataset. CNNs showed high efficiency in a wide range of studies with defective images, as well as deep neural networks.93,94 CNNs were also applied to reconstruct the bubble pattern and its further identification.95 He et al. further developed the approach and used pulse-coupled neural networks (PCNN) for bubble segmentation.96 The proposed scheme turned to be effective also for image enhancement (Fig. 9c). The segmentation accuracy was about 90%.
Deep learning and convolutional neural networks can be successfully replaced with classical machine learning methods such as decision trees, support vector machines, linear regression, etc. Srivastava et al. demonstrated an approach to determine the size of bubbles depending on environment change. Cavitation bubble data was extracted from images using ImageJ scripts. Various regression models were used based om multilayer perceptron, decision tree, support vector machine. Performed models showed the same high efficiency as artificial neural network (ANN).97 Classic ML methods were successfully applied to determine the size of bubbles in real column reactors.98,99
The discussed approaches require a relatively large dataset of input images. To solve the small dataset problem, Fu and Liu used generative adversarial networks to create a BubGAN model.100 The performed model can generate image files that can later be used to create other models for cavitation bubble analysis. The algorithm showed high efficiency and image detection. The root mean square error (RMSE) was about 2–3%. This method is effective when it is necessary to create the intended applications of cavitation bubbles in the deficit of the liquid in which they were formed. This approach is proposed for both statistical machine learning methods, and is also applied to identify bubbles with high efficiency.29
Generative adversarial network (GAN) was also used for dataset enhancement and bubble image synthesis, while models for bubble segmentation were developed.101 The combination of image synthesis and the U-Net model provided more accurate bubble segmentation in comparison with previous models.
Cavitation bubbles can interact with each other, which complicates the analysis of bubbles flow analysis and image defects. Bubbles' interaction is hardly described by 2D images. Shao et al. performed a 3D reconstruction via CNN with U-net architecture.102 The result was a model that can be used to create a 3D graph with the distribution of bubbles.
Cavitation bubbles can transform not only via interactions with each other, but also by bubble growth, decomposition, or collapse. Bubble's shape transformation depends on the liquid content and can be used for quantitative content analysis of a liquid probe. CNN coupled with the transfer learning method allowed to classify samples by fluid content using a series of bubble images in dynamics (Fig. 9d).103 Pretrained CNNs with frozen layers can be successfully used for efficient model training without requiring a large dataset.
Fig. 10 (a) ANFIS structure scheme. Reprinted with permission from ref. 105. (b) Bubble size prediction based on a multi-layer artificial neural network. Reprinted with permission from ref. 110. (c) Workflow of applying ANN-Lee model for simulation of bubble condensation. Reprinted with permission from ref. 118. (d) Marginal and pairwise joint distribution of constitutive relation parameters. Reprinted with permission from ref. 119. (e) Using the YOLO net to predict the anomalous shape of cavitation bubbles. Reprinted with permission from ref. 120. |
The bubble size prediction discussed earlier can also be combined with physics, mathematics, and machine learning to get a simulation model. Jung et al. performed an efficient model that can accurately predict the size of cavitation bubbles on test data and in real systems.110 The approach was to create a pipeline with a multilayer perceptron (MLP) model capable of simulating the size of cavitation bubbles for various parameters (Fig. 10b). A low error of no more than 5% was obtained.
Similar pipelines were applied to identify the flow regime.111 Two-phase flow data was measured and extracted using Ultrasound Doppler Velocimetry (UDV). The obtained data was analyzed using three classification algorithms such as decision tree, K nearest neighbor, and support vector machine. All three algorithms showed classification accuracy above 90% in real time. For comparison, CNN and LSTM were used to identify the flow regime with an accuracy of about 94%. The effectiveness of the proposed approach was also showed by another research groups.112 Classic machine learning methods are also relevant for predicting the characteristics of the flow regime, the behavior of a single bubble in a flow, and its deformation.113–117
Integration of machine learning methods increases the efficiency of simulation models that are hardly performed for bubbles' transformation processes. Then, the bubble condensation simulation, which was previously performed using only the Lee model, was successfully implemented using artificial neural networks.118 The back propagation artificial neural network was used to obtain an empirical coefficient for the Lee model (Fig. 10c). The database consisted of four numerical inputs such as Prandtl and Reynolds numbers. The predicted empirical coefficient was in a good agreement with literature data, as well as modeling via the coupled ANN-Lee model.
A similar advanced approach with the integration of machine learning to clarify inputs and outputs was applied to flow simulation. Liu et al. used machine learning techniques to reduce the computational costs for multiphase computational fluid dynamics (MCFD) simulation of bubbly flow.119 Principal component analysis was performed to decrease the subspace of each MCFD simulation output, and then a feedforward neural network was used for surrogate modelling to predict MCFD results. A Gaussian process was used to obtain the model form uncertainty and parameters distribution (Fig. 10d).
Combining machine learning methods with only numerical methods is also an efficient approach, especially for real systems such as reactor with complex flows. Wang et al. discussed the applicability of CNNs combined with improved three-frame difference (ITFD) method for recognizing and tracking bubbles in a plate heat exchanger (PHE).120 The bubble size accuracy determination was 94% (Fig. 10e).
ML can also be applied for dynamic processes modelling without any support with classical simulation methods. This approach was performed to predict the migration of cavitation bubbles.121 The research demonstrated that instead of traditional experimental and numerical methods, a two-branch model of a deep neural network with the embedding of a Kelvin impulse can be used. The demonstrated method showed high efficiency.
The correct and appropriate use of ML techniques for soft and liquid molecular materials allows discovering their new properties at all scales, from molecular to macroscopic, based on the analysis of experimental or simulated homogeneous data sets, and demonstrate the promise of soft materials devices for complex technological applications. Our review provides many examples supporting this conclusion.
However, several remarks should be made to all the analyzed studies (Fig. 11). Generally, even recent studies do not consider multimodal data obtained by various methods. Also, most of the research papers do not offer open access to either the data or the source code of the used machine learning model. The importance of this practice has already been emphasized in chemistry125 and should be recognized by researchers in the field of soft materials. An open data policy will enable in-depth studies using various big data analysis algorithms when considering existing scientific problems in soft matter from different points of view. Consideration of complex data sets involves the introduction of universal descriptors that could link together the various properties of the analyzed system at several levels of the hierarchy. This will most likely require the creation of new, integrated approaches for multivariate analysis of the systems under study, that will be based on several ML methods. Validation of the created complex ML approaches, interpretation of the obtained ML data, their reliability and validity remain critical issues. The implementation of these steps in the development of machine learning technique will ensure the formation of a universal approach to the analysis and prediction of the behavior of systems and materials with a complex architecture. It will also pave the path to new interdisciplinary areas of research and bring scientific results that are difficult to imagine at the current level of applying ML and artificial intelligence methods for soft and liquid molecular materials.
Fig. 11 Envisioned development of a machine learning approach for further deep analysis of complex soft and liquid material systems. |
This journal is © The Royal Society of Chemistry 2023 |