Becoming a foodie in virtual environments: simulating and enhancing the eating experience with wearable electronics for the next-generation VR/AR

Abstract

Human–machine interfaces (HMIs) have received significant attention for their potential in augmented reality (AR) and virtual reality (VR). Perception of food is an important component of human sensations closely related to healthcare and overall quality of life, which, however, is an underrepresented area in current VR/AR technologies. This review summarizes recent progress in simulating chemical and physical sensations for enhancing eating experiences by utilizing emerging wearable electronics. We start with a brief overview of the key sensory components that shape eating-related perceptions, including the widely studied physical cues (auditory, visual, tactile) as well as the often-overlooked chemical senses (olfactory, gustatory). Then, we review prior work on eating experience-related HMIs, organizing them according to two main categories: sensors used for information capture and actuators used for the simulation of sensations. In the following section, we further discuss the integration of these wearable electronics with hardware and software to build Internet-of-Things and advanced HMIs for human-in-the-loop interactions. The final section summarizes remaining challenges and provides an outlook on the development of eating experience related VR/AR technologies for various applications, with the goal of providing references and guidelines for future research efforts in this underexplored yet thriving field.

---

Wider impact

The eating experience is a basic part of daily life, not only essential for supporting basic body functions but also deeply related to psychological well-being. Creating novel techniques for simulating the eating experience can contribute to improved life quality, enhanced health outcomes and immersive food-related experiences that promote social interaction and global collaboration. This review discusses recent advancements in wearable electronics for developing VR/AR technologies that simulate and enhance people's interactions with food in virtual environments. Given the complex nature of eating, we review the various sensory aspects involved in this process, including the well-established auditory, visual, and haptic cues, as well as the underrepresented olfactory and gustatory elements. With the aim of guiding the development of adaptive, human-in-the-loop systems for remote interaction and collaboration, this review covers recent advancements, including capturing and extracting characteristics from food sources, monitoring human actions, and providing stimuli to replicate sensory experiences. We also provide an overview of emerging multimodal operating systems that combine several sensory modalities to create realistic, immersive experiences. Together, this review aims to inspire future work in this underrepresented field of VR/AR, with the goal of developing a new model for people's interaction with food and social connections in the digital era.

1. Introduction

As a famous Chinese proverb says, “Food is the primary necessity of the people.” Food is more than just something we eat in order to maintain our body. Rather, it is a vital part of human culture, emotions, and social connections.^1,2 The sensory experience of eating involves a rich, intricate interplay of visual, auditory, tactile, olfactory, and gustatory cues, all of which shape our perception of food and influence our overall enjoyment.^3–5

Recent advancements in VR/AR technologies have revolutionized human–machine interactions, offering unprecedented opportunities to redefine how we perceive and interact with virtual environments.^6,7 With the integration of internet of things (IoT), these interactions can extend beyond individual devices, enabling interconnection of digital content across vast distances, facilitating communication between users and devices, as well as between users themselves, regardless of location^8–10 (Fig. 1). However, simulating and enhancing the eating experience in virtual environments remains a challenging, underexplored frontier, leaving a significant gap in the full sensory immersion VR/AR can offer.^11,12 This is associated with the complicated nature of the eating experience, technological constraints in simulating relevant sensations during food interaction, safety and hygiene concerns, and user acceptance. In an era where digital interactions are a pivotal phenomenon shaping nearly every aspect of our lives, it is essential that the eating experience is not left out.¹³ Just as we have embraced virtual socializing, remote working, and digital entertainment, integrating eating-related experiences into the digital realm can offer exciting new possibilities for both personal health/well-being and social connection.

Fig. 1 Brief overview of VR/AR technology-enabled simulation of eating experiences. (a) Development and evolution of wearable technologies to simulate/enhance the eating experience and bridge connections to distant realms by combining sensing, communication, and feedback systems. The figure illustrates an envisioned system where a robot performs distant tasting of food and transmits sensory data to a user for remote perception. (b) The five human sensory modalities and key components under each category related to food perception experiences. (c) Potential applications of simulating the eating experience through VR/AR technologies include, but are not limited to, entertainment, healthcare, education, exploration in extreme environments, and global collaboration. The images are used under a license from Adobe Stock.

In this context, the emergence of wearable electronics, which encompass various sensory modalities for both information capture and simulation, has created opportunities in this field. This review looks at how wearable electronics and modern VR/AR technologies are working together to improve the eating experience when combined with software systems and human-in-the-loop interactions. It includes studies on the widely studied physical cues (auditory, visual, tactile) as well as the often-overlooked chemical senses (olfactory, gustatory). We review prior studies on eating experience-related HMIs and categorize them into visual, audio, haptic, chemical, electrical and/or thermal stimulation approaches. Then, we further expand the scope by discussing the integration of wearable electronics with hardware and software to build the IoT and advanced HMIs through the involvement of human factors. The final section summarizes remaining challenges and provides an outlook on this underexplored yet rapidly thriving field. These developments not only enhance immersive VR/AR dining experiences for entertainment and social interactions but also provide practical applications in healthcare and personalized nutrition, such as creating diagnostic and rehabilitation tools for people with eating or swallowing difficulties. They might also help develop new ways of producing and consuming food in the digital era. Together, our goal is to provide general guidelines and inspire further research in this growing field.

2. Capturing information related to eating experiences for analysis and enhancement

To deliver meaningful eating experiences in virtual and augmented reality, it is crucial to first capture and interpret the complex multisensory signals that arise during real-world food interactions. This spans multiple dimensions, including the sensory characteristics of the food itself and the physiological responses elicited in humans during these interactions. This section outlines recent advancements in sensing technologies tailored to each of the five key sensory modalities—visual, auditory, tactile, olfactory, and gustatory. By leveraging various sensors in bio-integrated formats, and sometimes combining them with machine learning techniques, researchers have begun to decode how humans perceive food-related stimuli across different channels. These efforts serve as the foundation for developing actuation systems that can simulate the corresponding sensations, a topic that will be explored in subsequent sections.

2.1. Sensing of food information associated with vision and audition

Visual and auditory modalities are two well-established domains in current VR/AR technologies, where capturing information from users and their surrounding environments plays an important role. Visual sensing devices, such as complementary metal–oxide–semiconductor (CMOS) cameras, depth sensors, and spectral imaging systems, detect detailed features of food appearances, including color, shape, and surface texture. These sensors enable accurate representation of visual cues that influence taste perception. Auditory sensing technologies, including MEMS microphones and other acoustic sensors, capture food-related sounds like chewing, frying, or pouring, which enhance the perception of texture and freshness. These technologies are already well-developed and have been integrated into commercial devices at a relatively mature stage, with detailed information available in other sources. Consequently, this review will not provide an extensive overview of these areas. The current VR/AR devices, especially headsets, incorporate devices for both sensing and actuation of visual and auditory information/stimuli. More information will be provided in the later actuation sections.

2.2. Detection of food texture information associated with tactile sensation

Aside from the visual and auditory modalities that can be captured and/or simulated using commercially available headsets, eating experiences involving other senses have been much less explored using current VR/AR technologies. For example, while sounds and images are relatively easy to synthesize, there are very few studies that address human mechanosensitivity within the mouth during eating or the simulation of food textures. Integrating tactile sensations into VR/AR requires efforts across multiple dimensions and their combination, including understanding key textural parameters of food, capturing these parameters through sensors (on either robotic systems/users), evaluating human perception and feedback, recording and decoding responses, and ultimately, simulating these stimuli to create realistic sensations.^14–17 The mouth is the primary organ responsible for sensing the texture of food. Additionally, sensory input from other parts of the body, such as the hands or throat, also plays a role in shaping the eating experience. Collecting and deciphering signals from these organs can aid in analyzing human interactions with food, provide real-time feedback accordingly, and even enable interactive experiences for users in distant environments within future metaverse scenarios. Fig. 2 presents representative strategies for capturing food texture information, with a focus on their working principles, structural designs, and applications. Compared with conventional sensory evaluation (e.g., manual touch testing), the instrumental evaluation method can characterize many textural features of foods with improved accuracy, repeatability, and versatility. Fig. 2a illustrates a quantitative tester for characterizing the hardness of foods, such as fruits, designed to replace traditional subjective methods such as the “fingernail” or “finger press” test.¹⁸ The device determines hardness by measuring the air pressure required to insert a blunt-ended 5/32-inch diameter piston into the test material to a fixed depth (1/32 inch). The applied pressure is measured with a pressure gauge, and a circuit triggers an indicator light to signal when the predetermined insertion depth is reached. It has replaceable piston heads facilitating adjustable insertion depths, making it suitable for measuring different types of foods. The test material can remain essentially intact after the test with minimal damage. Fig. 2b illustrates the method of non-destructive measurement of textures of fruits and vegetables utilizing a multi-sensor data fusion technology: using forced impact (FOI) to predict the apparent elastic modulus and Magness-Taylor hardness of fruits and vegetables.¹⁹ The FOI system consists of a linear electromagnet driver, a control circuit, a load sensor, an aluminum plate, an adjustable height bracket, and a data acquisition unit. The electromagnet driver generates a controllable pulse, the steel ball hits the sample, and the load sensor records the impact signal and transmits it to the computer for analysis. The results can accurately predict the texture of food. Fig. 2c illustrates another example of a needle probe, a widely used geometry in texture analyzers, designed for quantitative characterization.²⁰ It measures the deformation and destructive behavior of food when subjected to an applied force. The needle penetrates the food sample at a controlled speed, while the force applied is either predefined or adjusted based on the test requirements. As penetration occurs, the sensor measures the resulting changes in force and displacement. The system then generates a force–displacement curve, which is used to analyze key textural parameters of the food, such as hardness, elasticity, and viscosity.

Fig. 2 Strategies employed by the research community for detecting and analysing food textures. (a) Measuring the gas pressure required to push the blunt end of a piston into the test material to analyse the texture of objects,¹⁸ Copyright 1949. (b) Using the FOI tester to measure the elasticity and firmness of fruit non-destructively by applying drop weight impact, forced impact, and acoustic impulse response fusion techniques,¹⁹ Copyright 2017, Elsevier. (c) Using the texture analyser with a needle probe to pierce samples and test their hardness, elasticity, and texture characteristics,²⁰ Copyright 2016, American Pharmacists Association. (d) Applying touch sensor array pads made of carbon nanotube and silicone composites for strain sensing,²¹ Copyright 2024, Springer Nature. (e) Using a flexible bacterial cellulose/Ti₃C₂T_x MXene bio-3D porous aerogel to measure bite force,²² Copyright 2021, American Chemical Society. (f) Integrating distributed fiber optic sensors into the mouthguard, enabling the measurement of bite force through mechanoluminescence,²³ Copyright 2022, Springer Nature. (g) Monitoring swallowing strain by applying filamentous serpentine thin epidermal patch electrodes,²⁶ Copyright 2020, The American Association for the Advancement of Science. (h) Using non-invasive skin-mounted sensors to acquire sEMG and strain signals for tracking swallowing and laryngeal movements,²⁷ Copyright 2019, The American Association for the Advancement of Science. (i) Employing TVAN and SPAN tactile sensors to sense texture, hardness, and temperature of objects,³¹ Copyright 2024, Wiley-VCH GmbH. (j) Using 3D architected electronic skin capable of sensing normal force, shear force, and strain to measure the texture of fruits of various shapes,³² Copyright 2024, The American Association for the Advancement of Science.

Nevertheless, conventional off-line food texture analyzers can only provide reading parameters and are not capable of communicating or delivering sensations. This limits their integration with VR/AR technologies that seek to provide users with immersive and realistic experiences. The recent development of bio-integrated electronics offers promising opportunities by digitally recording and communicating various signals at multiple locations of the body when interacting with food. The human body can perceive various mechanical stimuli, such as pressure, vibration, and movement through specialized mechanoreceptors. Such information can, in turn, be leveraged to analyze the food textural parameters and enhance the sensory aspects of the eating experience. Hard and soft tissues within the mouth, such as the teeth, gums, hard palate, and tongue, play important roles in texture perception. Among them, teeth have periodontal and intradental mechanoreceptors and can provide information about the hardness, coarseness, and brittleness of food. Bite force is a common parameter for evaluating masticatory function in dental research. Mouthguard sensors have been reported in recent years for detecting occlusal patterns through the mapping of biting force. Sensors in mouthguard design also serve as a class of bio-integrated electronics that can enable efficient data collection without disrupting daily activities. Fig. 2d introduces a teeth- and tongue-controlled oral pad (oPad) that employs a tactile sensor array composed of carbon nanotubes (CNTs) embedded in a silicone elastomer matrix.²¹ The CNTs provide exceptional sensitivity to pressure variations. To further enhance the sensitivity, the film surface incorporates spinous microstructures fabricated using a sandpaper stencil printing process, which concentrate local stress at contact points. The resulting sensor array can detect pressures of up to 700 kPa, making it responsive to both gentle tongue slides and strong teeth bites. Supporting the sensor array is a flexible printed circuit board (PCB) that houses critical electronic components. The oPad is encapsulated within a three-dimensional (3D)-printed thermoplastic polyurethane mouthguard that ensures both biocompatibility and mechanical flexibility. To protect the sensor from damage caused by saliva or excessive biting, the assembly is coated with polydimethylsiloxane (PDMS) elastomer layers. A compact lithium battery is embedded within the mouthguard, providing power for up to 20 hours of continuous operation. When the tongue slides or the teeth bite, the applied pressure causes the resistance of the composite to decrease. The oPad can convert oral movements into intuitive and precise control commands. This makes it an effective tool for a variety of applications, including gaming, virtual keyboard typing, and assistive technologies like wheelchair navigation, particularly benefiting individuals with mobility limitations. Fig. 2e showcases a flexible sensor engineered from a degradable bacterial cellulose/Ti₃C₂T_x MXene bioaerogel for detecting bite force.²² The sensor leverages bacterial cellulose as the matrix material, integrating its abundant functional groups with the superior mechanical and electrical properties of MXene to construct a 3D porous bioaerogel with high pressure sensitivity. During biting, the sensor undergoes deformation in its 3D porous structure upon application of pressure, which induces changes in resistance. The sensor quantitatively determines the magnitude of the applied pressure within a range of 0.1 N to 50 N. The lightweight bioaerogel is suitable for extended wear. It demonstrates excellent pressure-sensing capabilities, with high sensitivity (125.8 kPa⁻¹), a low detection limit of 10 Pa, and stable performance over 5000 loading and unloading cycles. The bioaerogel also shows remarkable biocompatibility, achieving a cell viability rate of 99.3% ± 1.5% in cytotoxicity tests, ensuring safety for direct contact with the human body. For example, it has been used to enhance real-time oral health monitoring by combining bite force measurement with gas sensing to accurately detect the intensity, position, and contact sequence of bite forces, providing important data support for dental occlusion analysis, orthodontic evaluation, and restoration fitting. Fig. 2f illustrates an interactive mouth guard based on mechanoluminescence-powered distributed-optical-fibre sensors (mp-DOFs).²³ Specifically, the design employs an optical fibre with various transition-metal-doped ZnS embedded at several predefined locations. Doping Cu²⁺/Mn²⁺, Cu²⁺ and Cu⁺ into the ZnS host enables orange, green and blue mechanoluminescent emissions. The sensor converts distributed compression externally applied to the fibre into mechanoresponsive phosphors emitting light at different wavelengths. Optical signals are transmitted through the fiber via total internal reflection and captured by red–green–blue (RGB) sensors, enabling the detection of various occlusal patterns during interactive bite-controlled operations. Combining machine learning algorithms with the device allows for the translation of complex patterns into specific data inputs with an accuracy of 98%. Representative demonstrations include operating computers, smartphones and wheelchairs.

The act of eating involves cooperation between muscles, nerves, and structures within the throat, making it a vital area for monitoring in food-related research and healthcare. The emergence of wearable sensing technologies has enabled non-invasive, real-time tracking of physiological signals associated with chewing, swallowing, and related muscle activity.^24,25 These results may provide insights into the biomechanics of eating, supporting applications in dietary assessment, dysphagia management, and the development of personalized nutrition and health strategies. Fig. 2g presents an electrically compensated tattoo electrode designed for electrophysiology at scale.²⁶ This work is characterized by the development of large-area, soft, breathable, substrate and encapsulation-free electrodes. These electrodes achieve high skin conformability, allowing for natural sweating and heat dissipation, while also covering extensive areas of the body, such as the chest, forearm, and neck, without imposing mechanical constraints. The absence of a substrate or encapsulation makes the electrodes lightweight, comfortable, and capable of delivering high-fidelity electrophysiological measurements. A Cartan curve–inspired transfer process ensures that the electrodes can be laminated onto complex, nondevelopable skin surfaces without introducing strain, wrinkles, or mechanical deformation. The concept is derived from the Cartan development principle, which governs how a curve or surface can be “rolled out” or transferred from one geometry to another while preserving its original length and shape. In this process, the electrodes are treated as filamentary curves, and the target surface is modeled as a curved, non-developable surface. The transfer process involves carefully “rolling” the electrodes onto the skin in a manner that eliminates tension or compression along the filaments, thereby maintaining their structural integrity and conformability. This method differs from traditional approaches, such as direct transfer printing, which often results in strain-induced defects like wrinkles or fractures. The Cartan transfer process not only enhances the mechanical robustness of the electrodes but also preserves their ability to measure biopotentials accurately across large, irregularly shaped surfaces. The sensors can detect signals related to the eating experience by monitoring neck muscle activity during tasks such as swallowing and chewing. For example, it detects spatial distributions of surface electromyography (sEMG) signals when the participants swallow different volumes of water. It also captures sEMG signals as the participant chews on either the left or right side of the mouth.

Building on the capabilities of flexible, skin-conformal electrodes, further advancements have led to designs that target more specific applications, such as monitoring muscle activity and laryngeal motion in the submental region during swallowing. Fig. 2h depicts a flexible submental sensor patch, non-invasively adhered to the mandibular area, capable of monitoring muscle activity and laryngeal motion during swallowing tasks and related movements.²⁷ The system consists of a thin, honeycomb-structured patch integrated with sEMG electrodes and a strain gauge. The recorded data provides high-resolution information about muscle coordination and mechanical strain during swallowing. The structure of the patch allows it to conform to the curvature of the skin, maintaining consistent contact during head or neck movement. The honeycomb design ensures breathability and comfort while providing elasticity and durability for repeated use. The sEMG electrodes are aligned parallel to the muscle fibers, and the strain gauge is positioned to detect thyroid notch movements. Data from the patch is transmitted wirelessly to a portable control unit clipped to the clothing. This unit processes and relays the data via Bluetooth to an external device for real-time monitoring or analysis. The custom-built MATLAB code ensures noise reduction and signal accuracy, distinguishing swallowing activity from surrounding muscle interference. When the flexible submental sensor patch operates, it adheres to the skin beneath the chin, capturing bio-signals crucial for managing oropharyngeal swallowing disorders.

The sensation perceived on the hand, although not directly involved in eating action, also serves as an important source of feedback about the texture, temperature, and overall interaction with the food. E-Skin technologies are emerging as transformative tools in food and eating applications. By replicating the sensory functions of human skin, these devices enable precise, real-time assessments of food quality, texture, and other physical properties. Recent advancements in E-Skin design have focused on creating biomimetic systems capable of detecting subtle variations in mechanical properties, such as elasticity and firmness, critical for evaluating food freshness and ripeness.^28–30 These flexible and durable sensors operate under complex conditions, including temperature changes and repetitive mechanical interactions, to provide reliable, multimodal feedback. Fig. 2i presents a zero-bias bionic fingertip E-Skin that leverages a combination of the Transient Voltage Artificial Neuron (TVAN) and Sustained Potential Artificial Neuron (SPAN) for tactile perception.³¹ The TVAN uses an Ecoflex thin film (200 μm thick) with a honeycomb surface pattern. This design ensures isotropic output along any sliding direction and enhances vibration sensing. During sliding contact with a surface, transient voltages are generated due to the triboelectric effect between the Ecoflex film and the object. The SPAN sensor is constructed using a double-network ionic hydrogel composed of polyvinyl alcohol and polyacrylamide, with lithium chloride as the ion source and glycerol as a flexibility regulator. Electrodes made of Cu and Al are attached to either side of the hydrogel, which operates in a self-powered, zero-biased mode. The system works based on the sustained electrochemical potential difference between the electrodes. Applied pressure increases the contact area and reduces contact resistance, enhancing the potential signal, while temperature changes increase the ion mobility within the hydrogel, decreasing intrinsic resistance and elevating the signal. The sensor can decouple pressure and temperature effects due to their distinct impacts on the output, which are further analyzed using machine learning to achieve accurate multimodal tactile sensing. Together, the E-Skin can successfully perceive detailed object characteristics, including texture, hardness, temperature, and material composition and can be used to evaluate the texture of foods such as fruits, vegetables, or baked goods. With just a light touch, the E-Skin can capture the unique characteristics of the target fruit and the average accuracy can reach 99.0%. It can also be used to detect foods of different temperatures and softness, such as warm and chilled steamed buns. While the bionic fingertip E-Skin focuses on achieving multimodal tactile perception through advanced material and sensor design, the three-dimensionally architected electronic skin (3DAE-Skin) expands these capabilities by emulating the structural complexity of human skin, enabling detailed and versatile applications including food quality evaluation. Fig. 2j illustrates a 3DAE-Skin that mimics human mechanical perception.³²

It features a multilayer architecture that replicates the tri-layer design of human skin, consisting of an epidermis for surface protection, a dermis that embeds the majority of the sensing components, and a hypodermis for support and cushioning. The sensor array in the 3DAE-Skin consists of 240 piezoresistive sensors integrated into a 5 × 5 grid. Each functional unit within the array comprises multiple layers, including two layers for force sensing, two for strain sensing, and five polyimide (PI) layers that encase the sensing elements. Force sensors are positioned closer to the surface on eight-armed cage mesostructures with a height of approximately 600 μm to detect normal and shear forces, while strain sensors are embedded deeper, on arch mesostructures with a height of about 250 μm, to sense tensile strain. The 3DAE-Skin decouples the sensing of normal force, shear force, and tensile strain. The system is both durable and flexible, with an elastic modulus of about 194 kPa, similar to that of human skin. The device enables tactile sensing for detailed spatiotemporal mapping of forces and strains, allowing assessments of object properties such as elasticity, curvature, and surface texture. In food quality assessments, it can measure the modulus of fruits and baked goods to determine their freshness and ripeness based on changes in rigidity.

2.3. Capturing olfactory data using sensors

Transitioning from mechanical interaction to chemical perception, olfactory cues represent a less tangible but equally powerful aspect of the eating process. Unlike tactile sensations, which rely on physical pressure or deformation, olfaction operates through airborne volatile compounds that stimulate specialized receptors in the nasal cavity. These smells often arrive before the food is even tasted, shaping expectations and evoking emotional or memory-based responses. Integrating olfactory data adds depth to sensory analysis, offering a more complete depiction of food perception beyond touch and feel. This section discusses recent progress in sensing technologies relevant to the development of advanced digital olfactory interfaces. Olfaction complements the basic tastes detected by the tongue. It integrates with other sensory inputs, such as texture and taste, to create the gestalt of the eating experience and is also strongly linked to emotions and memories through physiological and psychological influences. Thus, developing odor sensors capable of decoding complex odor species and integrating these inputs into next generation HMIs are essential for enhancing the chemical connectivity between virtual and real spaces. A long-lasting challenge in this field, however, is that gas molecules are typically small in size and low in molecular weight, and achieving selective detection using chemical sensors is challenging due to the lack of available highly specific receptors for individual gas molecules. Consequently, established electronic noses typically have low recognition capabilities in determining the exact gas species and concentrations in mixtures. The research community typically addresses this challenge by developing intelligent systems that integrate sensors with classification algorithms, with a focus on the miniaturization of physical systems, enhancing species recognition, and improving response time and accuracy.

Machine learning algorithms play an important role in enhancing the performance of artificial olfactory systems by enabling pattern recognition, feature extraction, and classification of multiple gas categories and complex gas mixtures.³³ Commonly used methods include K-Nearest neighbors (KNN),³⁴ support vector machines (SVM),³⁵ and principal component analysis (PCA),³⁶ which provide effective solutions for pattern recognition and data dimensionality reduction. Artificial neural networks (ANNs),³⁷ such as backpropagation neural networks (BPNNs),³⁸ convolutional neural networks (CNN),³⁹ and radial basis function neural networks (RBFNNs),⁴⁰ are also commonly employed to model complex relationships in sensor data. In addition, spiking neural networks (SNNs) have emerged as a promising approach, offering biologically inspired processing capabilities that align well with the dynamic nature of olfactory signals.⁴¹ Dimensionality reduction techniques, such as t-distributed stochastic neighbor embedding (t-SNE), are also applied to visualize high-dimensional sensor responses.⁴² Together, these algorithms form a versatile toolkit for interpreting sensor responses and supporting real-time odor recognition in electronic nose systems. Several representative studies that incorporate machine learning into electronic noses will be discussed in detail in the following section.

To facilitate clearer understanding of diverse sensing strategies in olfactory interface research, these systems can be categorized into two main types: engineered material interfaces and biological receptor-based interfaces. Engineered material interfaces utilize only synthetic or inorganic materials (such as conductometric metal oxides (MOXs), metal–organic frameworks (MOFs), solid polymer electrolytes (SPEs), and colorimetric hydrogels) that detect odorants through redox reactions, adsorption, or other physicochemical processes. In contrast, biological receptor-based interfaces employ olfactory receptor proteins or their synthetic analogs to enable highly specific molecular recognition, closely replicating the mechanisms of the native human olfactory system.

MOXs, which work based on surface reactions, are widely used and investigated materials in gas sensing due to their high sensitivity, low cost, and flexibility in mass production. Gas molecules can either donate electrons to or withdraw electrons from MOXs, resulting in changes to the electrical properties of MOXs that can be measured.⁴³ Based on well-established research in e-noses over the past few decades, this concept has been adapted for the detection of diverse odor species. Fig. 3a highlights an innovative design that incorporates MOXs into integrated chips for olfactory sensing.⁴⁴ The sensing chip comprises top electrodes, a multi-component interfacial (MCI) layer, a PdO/SnO₂ nanotube sensing layer in a porous alumina membrane (PAM) channel, bottom electrodes, an insulating layer and a Pt heater. The MCI layer, fabricated by suspended mask-assisted sputtering (SMAS), achieves a gradient distribution of four metal oxides (ZnO, NiO, In₂O₃, and WO₃) by varying the distance between the mask and the PAM substrate, creating distinct pixels. These pixels form a heterojunction with the PdO/SnO₂ sensing layer to regulate carrier transport, resulting in a sensor array with diverse sensors that mimic human olfactory receptors. Each pixel functions as an individual sensor with a unique resistance response to different gases, producing resistance heat maps specific to each gas type. The biomimetic olfactory chip (BOC) can detect eight gas species (acetone, carbon monoxide, ethanol, formaldehyde, nitrogen dioxide, toluene, hydrogen, and isobutylene) at concentrations ranging from 20 ppb to 4 ppm and is capable of recognizing 24 distinct odors, including mixtures. Advanced machine learning algorithms are employed, including a 4-layer CNN for single-gas classification, a 5-layer fully connected neural network (FNN) for gas mixture concentration prediction, and t-SNE for visualizing odor relationships. The compact chip measures just 8 mm² and integrates up to 10 000 individually addressable sensors with 10 × 10 μm² sensing units. It achieves a gas recognition accuracy of 99.04%, outperforming existing systems in sensitivity, diversity, and power efficiency. Key features include the innovative vertical structure, a Pt heater for optimal performance at 240 mW, excellent repeatability, and long-term stability. A robot dog integrated with the sensor chip can identify odors in blind boxes by integrating visual and olfactory sensory data. Using a pre-trained CNN network, the system efficiently recognizes gases and odors at distances of up to 4 m.

Fig. 3 Strategies employed by the research community for sensing odors: (a) Utilizing MOXs as sensing agents, combined with machine learning, to distinguish between pure gas chemicals and mixtures,⁴⁴ Copyright 2024, Springer Nature. (b) Employing MOFs as sensing agents to differentiate alcohols and aldehydes based on varying memory durations,⁵² Copyright 2024, Springer Nature. (c) Using colorimetric reagents to identify nonvolatile airborne microparticles,⁵⁴ Copyright 2020, Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim. (d) SPEs to detect the presence of H₂S,⁵⁷ Copyright 2023, Wiley-VCH GmbH. (e) Utilizing hORs with the aid of machine learning to discriminate four SCFAs and their mixtures,⁵⁹ Copyright 2024, The American Association for the Advancement of Science.

While progress has been made in identifying odor species, the response time of many systems remains at the scale of seconds, even though odors can occur within tens of milliseconds. A recent study utilizing MEMS-based MO_x sensors with ultrafast heater modulation cycles of 50 ms has achieved remarkable advancements.⁴⁵ These cycles are enabled by a combination of open-loop control, which uses a linear model calibrated to map heater resistance to temperature, and closed-loop control, which dynamically adjusts the temperature to compensate for errors and environmental fluctuations. This dual approach ensures precise and rapid cycling between 150 °C and 400 °C with high temporal accuracy. As a result, the system can classify single odor pulses as short as 10 ms and resolve odor switching patterns at frequencies up to 60 Hz, surpassing the performance of animal olfaction in specific tasks. Tested with synthetic food-related odorants (isoamyl acetate, ethyl butyrate, cineole, and 2-heptanone), it demonstrates response and recovery times of 87 ± 20 ms and 106 ± 24 ms, respectively, with 100% classification accuracy for odor pulses lasting 50 ms or longer. Leveraging advanced machine learning algorithms, such as KNN and SVM, along with phase-locked and frequency-domain features, the system ensures robust classification. Compact and energy-efficient (1.2–1.5 W), this electronic nose addresses critical challenges in mobile robotics, environmental monitoring, and security, enabling accurate odor tracking and source localization in dynamic, turbulent environments.

MOFs offer distinct advantages for gas sensing, including high surface area, tunable pore size, chemical versatility, and selective binding, making them a promising choice for advanced applications.^46–51 A novel design utilizing a conductive MOF, Ce-2,3,6,7,10,11-hexahydroxytriphenylene (Ce-HHTP), is showcased in Fig. 3b as the foundation for an artificial olfactory memory system.^52,53 This system integrates the MOF on an interdigital electrode connected to a series circuit to detect target gas molecules. It addresses key challenges in replicating human olfactory memory by integrating sensory input with distinct short- and long-term memories and adapting to dynamic chemical stimuli. The system exhibits short-term resistance responses to alcohols (methanol, ethanol, and propanol) and long-term resistance responses to aldehydes (formaldehyde, acetaldehyde, propionaldehyde, glutaraldehyde, and benzaldehyde). It also demonstrates unique short-term responses to ketones like acetone, where the response is only 17% of that for propanol, ensuring minimal interference with alcohol detection and no effect on aldehyde detection. With a detection limit of 3 ppm for propionaldehyde and stable operation over seven days, its applications include integration with unmanned aircraft for remote pollution monitoring, portable detection boxes, and wearable flexible patches for health management and aldehyde dosimetry, highlighting its significant potential for environmental and health-related innovations.

Colorimetric reagents are chemicals that react with specific analytes to produce distinct color changes, enabling the detection and identification of substances. These reactions typically involve chemical interactions such as oxidation–reduction, acid–base reactions, or complex formation, resulting in measurable color shifts.⁵⁰ Due to their high specificity and sensitivity, colorimetric reagents can be used for smell related target analyte detection. A novel artificial olfactory sensing system, showcased in Fig. 3c, utilizes a hydrogel matrix embedded with colorimetric reagents as its core component.⁵⁴ This system addresses critical challenges in detecting ultralow or nonvolatile analytes, such as airborne improvised explosive microparticulates, by mimicking natural olfactory mucosa and odorant binding proteins. By leveraging specific chemical reactions, it can detect substances including hypochlorite, chlorate, perchlorate, urea, and nitrate. The system demonstrates exceptional sensitivity and provides rapid response times ranging from 0.2 to 20 seconds depending on the analyte. Additionally, it features high specificity, excellent reusability over ten cycles, and robust performance against interference. These capabilities make it well-suited for detecting airborne explosive microparticulates, as well as broader applications in environmental monitoring and security. The integration of RGB-based analysis and hierarchical clustering ensures precise analyte discrimination.

SPEs are versatile materials consisting of a polymer matrix combined with ionic species, such as salts or ionic liquids, enabling ionic conduction in a solid-state medium. These ionic species interact with gas molecules, altering the electrical properties of the system, making SPEs highly suitable for gas sensing applications.^55,56 Fig. 3d demonstrates a study addressing the challenges of developing synaptic devices with high sensitivity, selectivity, and low operational voltage for olfactory systems by integrating sensing and memory functions to mimic cumulative gas exposure effects.⁵⁷ The biomimetic olfactory synapse, based on a polymer poly(3,4-ethylenedioxythiophene) doped with a poly(styrene sulfonate) (PEDOT:PSS) organic electrochemical transistor (OECT) and a porous SPE, demonstrates ppb-level sensitivity and exceptional selectivity for hydrogen sulfide (H₂S) over other gases. The porous SPE, fabricated via a breath-figure method, enhances ion mobility and surface interaction, enabling efficient low-voltage operation (≤1 V) and robust synaptic plasticity. Synaptic weight changes (ΔW), which represent the modulation of the postsynaptic current, allow the device to emulate short-term memory (STM) and long-term memory (LTM). STM is characterized by reversible changes in ΔW at low H₂S concentrations (e.g., 0.2–1 ppm), where the channel current increases during exposure and fully recovers after stimulus removal, resulting in ΔW = 0. For instance, at 0.2 ppm H₂S, the channel current transiently increases but returns to baseline after 300 seconds. In contrast, LTM is observed at higher H₂S concentrations (≥1 ppm) or prolonged exposure, where ΔW becomes persistent, indicating non-volatile behavior and cumulative damage. At 5 ppm H₂S, for example, the channel current remains suppressed even after stimulus removal, demonstrating LTM. The transition from STM to LTM, modulated by increasing H₂S levels or exposure durations, reflects the progressive impact of toxic gases on biological systems. This biomimetic synapse achieves real-time wireless monitoring of H₂S released by spoiled food, highlighting its significant potential for applications in food safety, healthcare, and intelligent artificial olfactory systems.

Although significant progress has been made in developing artificial odor sensors using chemical interfaces, these systems still lag behind the sophistication of mammalian olfactory systems, where olfactory receptor proteins specifically recognize and bind to odorant molecules. To bridge this gap, directly integrating human olfactory receptors (hORs) offers a promising avenue for advancing olfactory sensing systems.⁵⁸ A key challenge in this field is the accurate discrimination of chemically similar compounds, such as odorants with varying chain lengths. Addressing this challenge, Fig. 3e presents an artificial olfactory system (AOS) that combines hORs with artificial synaptic devices.⁵⁹ The system's uniqueness lies in its integration of hORs, which are overexpressed in Escherichia coli, purified, and assembled into nanodiscs with membrane scaffold proteins (MSPs). These nanodiscs are immobilized onto a graphene surface, forming an extended gate device capable of specific binding interactions with odorants. Upon exposure to short-chain fatty acids (SCFAs) like propionic acid, butyric acid, valeric acid, and hexanoic acid, as well as their mixtures, the AOS produces distinct conductance patterns. For mixture testing, the total concentration is fixed at 3 ppm, with each SCFA contributing equally. The conductance patterns from single SCFAs are combined to simulate mixture responses, which are processed by an ANN, enabling high-accuracy odorant recognition (>90% accuracy). The system demonstrates exceptional sensitivity, with detection limits ranging from 0.07 to 1.30 ppm depending on the SCFA and hOR used. By leveraging specific receptor-odorant interactions and neuromorphic algorithms, the AOS successfully mimics the pattern recognition capabilities of mammalian olfactory systems. This innovation opens the door to practical applications, such as diagnostics for identifying SCFAs as biomarkers for gastrointestinal diseases and halitosis, environmental monitoring, and next-generation sensory technologies for IoT applications.

2.4. Development of electronic tongues for taste information capture

While olfaction provides a preview of flavor through aromatic compounds, gustation completes the chemical dimension by identifying basic taste modalities such as sweetness, sourness, saltiness, bitterness, and umami. These two senses work synergistically to construct a unified flavor profile, yet each contributes unique information. Gustation involves direct interaction between soluble molecules and taste receptors on the tongue, creating immediate feedback on nutrient content, palatability, and safety. To simulate the full sensory richness of eating, it is crucial to capture taste information, accurately detect associated human responses, and understand their interaction with olfactory cues in shaping overall food appreciation.^60,61 To this end, the electronic tongue technology quantifies the chemical components of taste samples, enabling the objective identification and characterization of flavor attributes including both fundamental tastes and complex ones. This approach improves reproducibility and objectivity, minimizes variability in individual assessments, and delivers more accurate and quantifiable data compared to traditional sensory evaluation methods that rely on subjective human judgment. Some lipid membrane-based electrochemical sensors have been commercialized, such as the Alpha M.O.S and TS-5000Z electronic tongue systems demonstrated in Fig. 4a.^62,63 Specifically, when artificial lipid membranes are immersed in aqueous solutions, the acidic groups within the lipid molecules dissociate, resulting in a negatively charged membrane and the formation of a double electric layer at the interface. For example, in the presence of acidic substances, protons from the acidic compounds recombine with the acidic groups on the membrane, thereby preventing or reducing dissociation and decreasing the number of negatively charged sites. On the other hand, when salts such as NaCl are introduced, Na⁺ approach the negatively charged membrane surface, and their electrostatic interactions effectively confine the influence of the double electric layer to a smaller spatial region, a phenomenon commonly referred to as the shielding effect. Controlling the chemical composition of the lipid membrane, introducing functional modifications, and combining multiple lipid membranes enhance the resolution and accuracy of concentration measurements for different taste substances. However, most existing electronic tongues can only provide offline reading parameters to users and are not capable of delivering sensations in real-time for rapid feedback and decision making. The high cost, bulkiness, and lack of portability of commercial sensors emphasize the necessity of developing novel electronic tongue systems.

Fig. 4 Representative electronic tongue systems for taste signal measurement and analysis. (a) Representative models and sensing mechanisms of commercial lipid-membrane-based electronic tongue systems,^62,63 Copyright 2010, The Author(s). Copyright 2023, The Authors, License MDPI, Basel, Switzerland. (b) An integrated gravity-driven triboelectric electronic tongue for liquid classification of chemical, environmental, and food samples,⁶⁴ Copyright 2024, Springer Nature. (c) An intelligent multi-photoresponsive electronic tongue for umami, sour, and bitter taste sensing based on Tb³⁺-functionalized hydrogen-bonded organic frameworks,⁶⁵ Copyright 2024, Wiley-VCH GmbH. (d) An artificial taste neuron based on an extended-gate MOSFET biosensor capable of simultaneous perception and spike encoding,⁶⁶ Copyright 2022, American Chemical Society. (e) A flexible ion-conducting hydrogel-based artificial tongue for astringency detection,⁶⁷ Copyright 2020, The American Association for the Advancement of Science. (f) A bimetallic bioinspired taste sensor for detecting the synergistic effects of umami substances,⁶⁸ Copyright 2023, Elsevier. (g) A robotic chemical-flavor sensing glove for fingertip detection of key tastes, including sweetness, sourness, and spiciness, enabling a “touch and sense” experience,⁶⁹ Copyright 2018, American Chemical Society. (h) A gustatory interface with high-density, ultra-conformal tongue electrodes for taste stimulation-induced physiological signal capture and operative assessment in tongue cancer patients,⁷⁰ Copyright 2024, Springer Nature.

The difference in electron affinity between materials leads to natural charge transfer upon contact, generating measurable electrical signals. When solutions contain different taste substances, their conductivity and electron affinity vary accordingly. Based on this principle, triboelectric taste sensors have been developed with advantages such as simple structure, high sensitivity, and self-powered operation. Fig. 4b demonstrates an integrated triboelectric biomimetic electronic tongue system with a gravity-guided design, allowing a single droplet of the sample solution to sequentially contact different sensors under gravity, producing characteristic electrical signals.⁶⁴ Specifically, when the droplet touches the polymer layer at the top of the sensor, the difference in electron affinity causes the solution and polymer to acquire positive and negative charges, respectively. As the charged droplet approaches or moves away from the metal electrode, induced charges are generated on the electrode, leading to distinct electrical signals. By selecting materials with varying electron affinities (e.g., polytetrafluoroethylene (PTFE), perfluoroethylenepropylene (PEP), polyethylene (PE), and PDMS) as functional layers of the sensors and arranging them in sequence, the sequential interaction between the liquid and the sensors produces unique voltage-time response curves that encode the taste profile of the sample. By integrating support vector classifier (SVC) and random forest (RF) algorithms, the system achieves high-precision measurement and identification of chemical samples (e.g., deionized water, hydrochloric acid, sodium chloride, sodium hydroxide), environmental samples, and food samples (e.g., black tea, white tea, and oolong tea).

Photoresponsive taste sensors utilize optical signals to detect and distinguish taste compounds, offering an alternative approach characterized by high sensitivity, specificity, spatial resolution, energy-efficient operation, and ease of multifunctional integration. Fig. 4c illustrates a photoresponsive electronic tongue system based on a Tb³⁺-functionalized multimodal hydrogen-bonded organic framework (HOF).⁶⁵ This system integrates dual optical signals—fluorescence and phosphorescence—enabling the accurate and reproducible identification of umami, sour, and bitter tastes. The sensor generates characteristic dual-emission signals through the interaction between taste compounds, Tb³⁺ ions, and the organic framework. Specifically, the system relies on fluorescence at 544 nm and phosphorescence at 410 nm. Disodium 5′-inosinate (IMP) and disodium 5′-guanylate (GMP) (umami substances) significantly quench both fluorescence and phosphorescence signals, whereas citric acid and oxalic acid (sour substances) quench fluorescence without affecting phosphorescence. Bitter compounds, such as 2-furaldehyde (2-FA) and 5-hydroxymethylfurfural (5-HMF), cause distinct signal reductions based on their interactions with the Tb³⁺-HOF structure. This multimodal detection approach allows the sensor to create “fingerprint information” for each taste compound, enabling accurate identification and classification without requiring complex data processing.

Traditional electronic tongue designs often rely on separate sensing and computational units, which increase system complexity, energy consumption, and hardware requirements. In contrast, neuromorphic devices integrate sensing, computation, and signal processing into a single system. Fig. 4d illustrates a metal–oxide–semiconductor field-effect transistor (MOSFET)-based artificial gustatory neuron system that mimics the integrate-and-fire (IF) behavior of biological neurons.⁶⁶ Under a constant current source, charge accumulation in the drain parasitic capacitor causes the drain voltage to increase until it reaches the latch-up voltage, which triggers a rapid discharge to the source and generates a spike signal. The drain voltage then returns to its initial value, repeating the process to produce periodic spike signals. The extended gate, equipped with a functional sensing layer connected to the MOSFET gate, detects changes in ion concentration, which alter the gate potential and further modulate the latch-up voltage, resulting in spike signals with varying frequencies. This mechanism directly enables the conversion of ion concentration into frequency. The pH-sensitive and sodium-sensitive artificial gustatory neurons use Al₂O₃ and sodium ionophore X as functional sensing layers, respectively. Finally, a single-layer perceptron (SLP) neural network is constructed based on the artificial gustatory neurons and identically structured MOSFETs. This fully hardware-integrated electronic tongue successfully classifies vinegar and saline solutions.

In addition to the five basic tastes, astringency perception is a taste of interest for electronic tongue systems. Inspired by the ability of the human tongue to perceive astringency through interactions between salivary proteins and tannins, Fig. 4e demonstrates a flexible hydrogel-based artificial tongue that effectively mimics this unique mechanism.⁶⁷ The electronic tongue employs a thin saliva-like hydrogel material, which combines mucin proteins, lithium chloride (LiCl) electrolytes, and a 3D porous polyacrylamide (PAAm) polymer network. When exposed to tannic acid (TA), a representative polyphenol, the hydrogel undergoes a structural transformation. TA molecules interact with mucin through hydrogen bonding and hydrophobic interactions, forming hydrophobic aggregates that convert the microporous hydrogel into a hierarchical micro/nanoporous structure. This transformation enhances ionic conductivity, enabling the conversion of concentration signals into electrical currents, with a wide sensing range (0.0005–1 wt%) and high sensitivity (0.292 wt%⁻¹). The artificial tongue demonstrates wide applicability, such as the quantification of astringency in beverages and fruits. It effectively monitors tannin levels in wine and tracks astringency changes during fruit ripening using a simple wipe-and-detection method. A 3 × 3 sensor array provides real-time taste mapping, highlighting its potential for practical applications such as quality control and taste standardization.

The complex mechanisms of human taste perception require replicating taste signals that are consistent with human sensory responses. This involves not only measuring the concentrations of different substances but also incorporating functionalities similar to the human sensory system. One such functionality is the synergistic effect of umami, where combining two or more umami substances, such as amino acids and nucleotides, produces a taste intensity far greater than the sum of the individual substances. Fig. 4f illustrates an electronic tongue based on the Venus flytrap domain (VFT) of the human T1R1 receptor and bimetallic MoS₂-PtPd nanomaterials, which measures the synergistic effects of umami substances.⁶⁸ The T1R1-VFT domain assembles onto the surface of MoS₂-PtPd nanomaterials through Au–S bonds, forming a stable and high-affinity sensing interface. MoS₂-PtPd nanomaterials enhance electron transfer and catalytic activity, achieving an ultralow detection limit of 0.03 pM. The sensor shows excellent linearity (R² > 0.99) when detecting various umami substances, including amino acids, nucleotides, organic acids, and umami peptides. The sensor produces a greater signal response in the mixed solution than in the single solution, which demonstrates a significant synergistic effect between beefy meaty peptide and inosine 5′-monophosphate disodium. When glutamate binds to the variable conformation site near the hinge region of T1R1-VFT, the VFT movement becomes inhibited, and the closed state of T1R1 stabilizes. IMP binding to an adjacent site near the opening of the flytrap further strengthens the closed conformation, amplifying the intensity of umami taste.

Integrating electrochemical sensors with robotic arms offers the potential for automated detection of taste substances in liquids and solids. Fig. 4g illustrates a robotic hand equipped with a chemical sensing finger designed specifically for taste discrimination.⁶⁹ This system combines electrochemical sensors with flexible and stretchable materials, effectively mimicking human taste perception and enabling the automated detection of flavors such as sweetness, sourness, and spiciness in both liquid and solid food samples. The robotic sensing finger incorporates specific functional layers on screen-printed electrodes embedded in a soft nitrile glove. The index finger, designed for sourness detection, uses a polyaniline (PANI)-modified electrode that is highly sensitive to pH changes and organic acids such as ascorbic acid, with signals captured through square wave voltammetry (SWV). The middle finger measures sweetness using a glucose oxidase (GOx)-coated enzyme electrode, where GOx catalyzes the oxidation of glucose and generates electrochemical signals proportional to sweetness levels. The ring finger detects spiciness with a carbon electrode functionalized with β-cyclodextrin (β-CD), which forms host–guest complexes with capsaicin molecules, enabling selective and sensitive detection of spiciness. For solid samples, a conductive agarose gel facilitates sample collection by adhering the sample to the sensing electrode. The robotic hand effectively distinguishes between caffeinated and decaffeinated beverages as well as sugar-containing and sugar-free drinks, demonstrating its versatility in practical applications.

In addition to capturing taste information from the target food source, another critical aspect of gustation sensors involves monitoring human physiological responses to taste stimulation. Fig. 4h demonstrates a high-density ultra-conformal electrode system that objectively maps the spatial electrical activity of the tongue.⁷⁰ This system combines tongue electrical (TE) signals with electroencephalogram (EEG) recordings using a dual-modal fusion algorithm, enabling precise taste decoding. This approach maps taste-evoked electrophysiological responses to the gustatory cortex, achieving a taste classification accuracy of 97.8%. Despite the absence of taste buds, the reconstructed tongue generates distinct taste-dependent TE signals when combined with EEG data, demonstrating its capability to effectively decode taste perception. To provide an overview of recent developments, Table 1 summarizes representative sensors and electronic tongue systems discussed in this review, highlighting their sensing mechanisms, functional materials, detection targets, taste categories, and application scenarios.

Table 1 Representative sensors and electronic tongues for taste and liquid sensing

Sensing mechanism	Functional materials	Target of detection	Taste categories	Application scenarios	Ref.
a This work differs from other sensors listed in this table. It involves assessing participants’ subjective perceptions through electrophysiological signal recording, not through direct detection of chemical substances.
Electrochemical	Artificial lipid membranes (e.g., phospholipids, plasticizers)	Fundamental taste compounds (sweet, sour, salty, bitter, umami), astringency	Sweet, sour, salty, bitter, umami, astringency	Objective taste profiling and quality control in food, beverage, and pharmaceutical industries	62 and 63
Triboelectric	PTFE, PEP, PE, PDMS	Deionized water, hydrochloric acid, sodium chloride, sodium hydroxide, black tea, white tea, oolong tea	Salty, sour, bitter	Liquid classification for chemical, environmental, and food samples	64
Optical (fluorescence/phosphorescence)	Tb^3+-functionalized HOFs	IMP, GMP, citric acid, oxalic acid, 2-FA, and 5-HMF	Umami, sour, bitter	Molecular identification of taste compounds	65
MOSFET	Al2O3, sodium ionophore X	pH, sodium ion	Salty, sour	Neuromorphic ion sensing for taste classification in E-tongue	66
Resistive	Mucin-lithium salt-PAAm hydrogel composite	Tannic acid, gallic acid, catechin	Astringency	Astringency monitoring in beverages and fruit ripening assessment	67
Electrochemical	T1R1-VFT domain + MoS2-PtPd nanomaterials	Glutamate, IMP, beefy meaty peptide	Umami	Quantification of umami synergistic effects	68
Electrochemical	PANI, Prussian blue, GOx, β-CD, chitosan, Nafion	Ascorbic acid, glucose, capsaicin	Sour, sweet, spicy	Robotic food and beverage taste detection	69
Electrophysiological^a	High-density ultra-conformal tongue electrode array	Tongue electrical activity evoked by taste stimulation	Taste decoding from electrophysiological responses	Gustatory function assessment and reconstruction in tongue cancer patients	70

---

3. Simulating sensory modalities in VR/AR

While sensing technologies provide critical data on how users interact with food in physical space, simulating these sensations in virtual environments is key to creating immersive and engaging eating experiences. This section focuses on state-of-the-art techniques for delivering multisensory feedback to users through visual, auditory, tactile, olfactory, and gustatory interfaces. By integrating soft electronics, haptic actuators, and chemical stimulators, these simulation strategies enable users to not only see and hear food, but also to feel, smell, and taste it—transforming passive observation into active participation.

3.1. Simulating and enhancing the eating experience through visual and auditory cues

Visual and auditory senses are two primary ways humans perceive and interact with the world, as currently extensively established in existing HMI technologies. The appearance of food and the dining environment can significantly influence one's appetite and willingness to eat. Similarly, auditory input enhances the eating experience by reflecting food texture and stiffness through the sounds of biting and chewing, while ambient music or environmental noise can also affect one's mood and overall enjoyment of a meal. Devices for recording and playing video and audio have been commercially successful and widely used for decades, ranging from speakers and radios to cameras and monitors. Simulating the eating experience through these modalities in current VR/AR systems is well-established and does not differ significantly from their application in other domains. Therefore, this section aims to provide only a brief discussion of representative examples. In virtual environments (VEs), a head-mounted display (HMD) is a classic tool for creating an immersive experience by placing dual screens directly in front of the user's eyes, occupying the entire field of view. Modern HMDs include but are not limited to those shown in Fig. 5a.⁷¹ They integrate advanced visual and auditory technologies to enhance immersion in virtual environments. Devices such as the Apple Vision Pro, High Tech Computer Corporation (HTC) Vive XR Elite, Microsoft HoloLens 2, and Meta Quest 3 feature high-resolution displays, with per-eye resolutions up to 3680 × 3140 pixels, ensuring sharp and detailed visuals. Wide fields of view, spanning up to 110°, contribute to an expanded sense of space, while focal adjustment mechanisms improve depth perception and viewing comfort in mixed and virtual reality applications. For audio, spatial sound systems create 3D soundscapes, enhancing the realism of virtual dining environments. Beamforming microphones support precise voice capture, facilitating clear communication and voice interactions. These technologies collectively provide a seamless and immersive sensory experience, enriching virtual eating simulations by replicating visual presentation, environmental ambiance, and auditory food cues.^72,73

Fig. 5 Strategies for synthesizing images and sounds to simulate and enhance eating experience. (a) Four representative commercially available head-mounted displays, Copyright 2024,⁷¹ Springer Nature Switzerland AG. (b) A handheld augmented reality APP displaying food resources required for preparing a meal,⁷⁴ Copyright 2024, IEEE. (c) A mobile augmented reality solution enhancing the dining experience of tourists enjoying a blueberry pie,⁷⁵ Copyright 2020, The Author(s). (d) A projection-based augmented reality system modifying the perceived color of real food.⁷⁶ (e) The Microsoft HoloLens providing spatial audio guidance to facilitate cooking in low-light or dark environments,⁷⁷ Copyright 2021, IFIP International Federation for Information Processing.

Fig. 5b–d present several illustrative examples of simulating eating scenarios, where commercially available techniques are used to develop custom-designed systems. Fig. 5b illustrates a new prototype that employs AR technologies in a handheld device to enhance the home cooking experience while promoting environmental sustainability.⁷⁴ In this prototype, the researchers developed the application using Unity. The process begins with users uploading the ingredients they plan to use in their meal, followed by scanning their dining room and kitchen. The application then displays visual representations of the ingredients, such as water, plants, and livestock, on the screen within the context of the dining room or kitchen. This provides users with a more intuitive understanding of how their meal choices could impact the environment. This visual AR application aims to encourage a more environmentally friendly approach for cooking in the future.

Fig. 5c illustrates another application utilizing mobile visual AR technology, designed for tourists traveling in Finland who wish to experience local cuisine.⁷⁵ Users scan a food item with their mobile device, and the app displays information directly over the real-world image of the food, including the name of the dish, a list of its main ingredients, the distance each ingredient traveled to the location, cultural stories about the food, and ratings and comments from other users. This approach enhances the dining experience for food enthusiasts by providing an immersive travel experience, helping tourists make informed decisions when selecting dishes that are more authentically local. Color significantly influences subjective food perception by affecting how individuals judge taste, freshness, and overall quality. Even before tasting, visual cues like hue and saturation can shape expectations—such as red enhancing perceived sweetness or brown indicating richness. These effects occur independently of the food's actual composition, making color a powerful tool in shaping consumer experience and behavior. Fig. 5d presents a projection-based AR system that changes the perceived color of real food in real time using a projector-camera setup.⁷⁶ By altering chroma and hue, it enhances the perceived sweetness in cake and changes flavor impressions of chips, demonstrating how visual cues alone can influence taste perception. Another study similarly highlights the power of visual cues by introducing a system that digitally shares lemonade through the detection of color and pH in real-world samples, then simulating the experience using RGB LEDs and electrical tongue stimulation. An experiment comparing the visual (pre-taste) and taste perceptions of real and virtual lemonade demonstrates the effectiveness of the method.^61,78 Fig. 5e illustrates an application utilizing audio mixed reality (MR) technology to enable users to simulate cooking in a completely dark environment.⁷⁷ In this experience, the user is blindfolded and wears an HMD, specifically the HoloLens, equipped with spatial audio capabilities to simulate binaural sound perception. This feature helps the user navigate by directing them toward desired locations through auditory cues. Passive haptics are incorporated by placing real objects in the user's vicinity, allowing them to physically touch and interact with their environment. The researchers also developed a digital voice assistant named Axela, which provides guidance on recipes and cooking procedures. This study highlights how audio MR can support cooking in challenging or unique environments. Additional examples of comprehensive eating experience simulations, incorporating visual and auditory cues alongside other modalities, are presented in Fig. 9 and 10.

3.2. Haptic interfaces for creating eating-related tactile sensations

Although auditory feedback enhances situational awareness and emotional engagement, it lacks the physical presence that makes eating tangible. Tactile simulation bridges this gap by allowing users to feel food-related textures and forces in real time. Whether through vibrations, pressure application, or mechanostimulation inside the oral cavity, these tactile interfaces can replicate subtle differences in food texture—like the smoothness of chocolate or the resistance of a chewy candy. Consequently, advancing tactile simulation is essential for achieving true realism in VR/AR-based dining scenarios, building upon the audio-visual framework with direct somatosensory engagement. To date, the VR/AR domains lack standardized digital methodologies for accurately simulating food textures. As tissues within the mouth (primary) and hands (secondary) serve as the main interfaces for interacting with food, pioneering studies have explored the custom design of haptic systems and strategies for simulating food textures tailored to these specific locations, utilizing handheld servo motor,⁷⁹ vibratory pin arrays,⁸⁰ electrical stimulation to the masseter muscle,⁸¹ and VR headset ultrasound phased arrays.⁸² The following section provides details. However, despite these advancements, limited progress has been achieved over the past one or two decades. Additionally, these existing technologies are still limited to obtrusive designs/implementations. Finally, the force and/or pressure varies at different locations within the mouth during the dynamic interaction with food, but providing texture perception with spatial resolution remains challenging.^83,84 Ergonomic, bio-integrated design is highly desirable to accurately capture/deliver texture information while remaining comfortable for users.^15,16

In addition to mechanical and ergonomic challenges, the surface structure and characteristics of food, such as texture, roughness, softness, and stickiness, also play a crucial role in shaping the user's haptic perception. Variations in these surface properties necessitate adjustments to the tactile feedback modalities—such as frequency, amplitude, and duration—delivered through haptic interfaces to enhance the perceived realism and immersion of virtual eating experiences. For example, foods with rough or irregular surfaces may require higher-fidelity haptic rendering to replicate the fine tactile sensations, while soft or adhesive foods may demand more sophisticated feedback mechanisms to simulate their compliance and adhesion.⁸⁵ Recent studies have demonstrated the use of electrical muscle stimulation (EMS) on facial regions, particularly the masseter muscle, to simulate chewing forces during eating.⁸⁶ These systems typically synchronize EMS delivery with the user's biting action, detected through sensors such as photoreflectors, and adjust stimulation parameters—such as frequency, pulse duration, and intensity, based on pre-recorded food texture profiles. For example, higher stimulation frequencies are used to replicate harder textures like crackers, while longer pulse durations convey the elasticity of softer foods such as cheese. This approach enables the dynamic reproduction of food texture perception in virtual dining scenarios. Despite its importance, the simulation of food surface structures remains an underexplored area because of the complexity of accurately reproducing diverse textures and the lack of standardized methods for characterizing and rendering such tactile details. Researchers may advance this field by adapting insights from haptic designs targeting skin surfaces, where parameters like actuation frequency and amplitude have been successfully mapped to fine roughness and macro roughness.⁸⁷ Such cross-domain approaches could offer a foundation for encoding food-specific surface properties into haptic systems for virtual eating experiences.

Fig. 6a describes a mechanically driven food texture simulator that applies forces to mimic the physical properties of food in the user's mouth.⁷⁹ The device integrates a force sensor to measure biting force and a servo motor to actuate the system. The working principle involves capturing biting force profiles from real foods using a sensor and replicating these profiles through controlled force application. When working, it begins with the measurement of real biting force using a thin, film-like sensor,

Fig. 6 Strategies for simulating feeding sensation and food texture perception. (a) Using the tactile interface food simulator with four connecting rods placed in the mouth to simulate the texture of real food,⁷⁹ Copyright 2004, IEEE. (b) Using the TactTongue electrotactile stimulator on the tongue to enhance its tactile perception,⁸⁸ Copyright 2023, ACM. (c) Applying EMS to the masseter muscle to simulate food texture including hardness and elasticity,⁸⁹ Copyright 2016, The Author(s). (d) Using the beam-wave ultrasonic transducer array to deliver pulses to the oral cavity and achieve the tactile effect,⁸² Copyright 2022, The Author(s).

FlexiForce. This sensor is placed between the food and the user's teeth to record force profiles during biting. For instance, when biting a cracker, the sensor detects two distinct force peaks: the first peak represents the breaking of the hard surface, while the second peak corresponds to the collapse of the internal structure of the food. For softer foods like cheese, the sensor records a gradual increase in force, indicating its elastic deformation. Next, the recorded force data is analyzed to create a time-force profile specific to each type of food. This profile is used as a template to simulate the corresponding biting experience. The device employs a 1-degree-of-freedom (DOF) mechanical system powered by a direct current (DC) servo motor, which drives a set of linkages designed to fit inside the user's mouth. These linkages apply force to the user's teeth via an end effector, replicating the recorded force profile in real-time. Then the simulator uses a two-stage force control mechanism to mimic the dynamics of biting. In the first stage, the device applies a consistent force to simulate the resistance of the food's surface. Once the force exceeds the first peak, the system transitions to the second stage, which uses an open-loop control method to replicate the breakdown of the food's internal structure. For soft foods like cheese, the system calculates the spring constant from the recorded data to simulate elastic deformation during the first stage. To ensure accurate replication, the system continuously monitors the user's applied biting force using its sensors, adjusting the simulation in real-time. Operating at an update rate of 1700 Hz, the simulator provides precise control over the force dynamics.

Besides replicating bite forces on the teeth, another avenue of research has explored simulating tactile sensations directly on soft tissues, such as the tongue. Electrical stimulation of soft tissues can replicate textures, pressures, and other sensations associated with the act of eating. Fig. 6b illustrates an electrotactile stimulation device, “TactTongue”.⁸⁸ At its core is a flexible electrode array constructed using a flexible PCB. Each electrode in the array has a diameter of 2 mm, with a 4 mm spacing between them, ensuring high spatial resolution for delivering tactile feedback. The platform integrates with Arduino Uno or Arduino Nano microcontrollers, leveraging their pulse-width modulation capabilities to generate stimulation waveforms. The stimulation system relies on biphasic pulses grouped into bursts, which are specifically designed to activate tactile receptors on the tongue without causing discomfort or overstimulation. For user safety, a resistor-capacitor network is incorporated into the design to ensure minimal net charge flow through the tongue. The tongue has a high density of touch receptors that are very sensitive to electrical stimulation. Due to the high conductivity and moist environment of the tongue tissue, electrical stimulation can produce a noticeable sense of touch at a relatively low current intensity.

While devices like TactTongue focus on precise and localized tactile stimulation, others are also exploring ways to simulate the properties of food textures at a larger scale. These efforts are critical for applications like virtual dining, where the sensation of biting and chewing must feel authentic. By combining electrical stimulation with databases of food properties, such systems can reproduce tactile sensations at various levels that align with user expectations. Fig. 6c shows a system for simulating the tactile properties (hardness and elasticity) of food and realizing a virtual dining experience through electrical muscle stimulation (EMS).⁸⁹ The bite detection module is responsible for capturing the user's biting action. It utilizes a photoreflector or similar sensor to identify when the user's jaw closes, which ensures that the electrical stimulation is precisely synchronized with the chewing motion. A database stores various food texture profiles, which are used as reference data to adjust the EMS settings. The electrical stimulation unit delivers EMS signals to the masseter muscle, which controls chewing movements. This unit includes a medical-grade electrical stimulator, such as the Digitimer D185, that generates controlled square wave signals. These electrodes are calibrated individually for each user to ensure precise targeting and effective stimulation. The EMS signals are modulated in terms of frequency, duration, and strength; higher frequencies simulate harder textures, while longer durations create a perception of greater elasticity. Finally, an Arduino-based control interface integrates the bite detection module, food texture database, and electrical stimulation unit to enable the synchronized and precise delivery corresponding to the user's chewing actions. The system is designed to adhere to safety standards, with voltage levels kept below 20 V and current below 10 mA, ensuring both effectiveness and user comfort. Non-contact solutions, such as ultrasonic phased arrays, offer an alternative to wearable haptic interfaces, providing improved comfort, usability, biocompatibility, and safety in VR/AR settings by eliminating the need for direct deployment on the human body, as shown in Fig. 6d.⁸² The device generates high-frequency sound pressure, focusing on the lips, teeth, and tongue. The device consists of an array of 64 ultrasonic transducers and control circuits, installed at the bottom of a VR headset. In this design, users do not need to wear additional equipment or change the existing structure of the VR headset. When working, it controls the emission pattern of sound waves to locate tactile feedback points in 3D space, generating a variety of tactile effects such as point pulses, sliding and continuous vibration, simulating drinking water and other oral-related tactile scenes. Beyond conventional actuation strategies, integrating photothermal mechanisms into the system enables active feedback by leveraging light-induced thermal effects, thereby enriching the bidirectional tactile interface. Photothermal actuators can deliver dynamic tactile feedback directly to the skin. By converting light energy (typically from near-infrared (NIR) or visible light) into localized heat, these actuators can induce mechanical deformations in responsive materials, thereby simulating tactile sensations such as temperature or pressure.^90,91

3.3. Creating olfactory sensations through chemical or electrical stimulation

Following the successful delivery of mechanical feedback through tactile actuators, olfactory simulation introduces another crucial sensory layer by engaging the sense of smell. Scent has the unique ability to trigger strong emotional and memory responses, often anchoring users in a specific time or place. In virtual dining systems, reproducing food-related aromas via controlled chemical or thermal release enhances the perceived authenticity of the experience. By layering olfactory stimulation on top of touch, sight, and sound, the virtual meal transforms from a basic visualization into a truly immersive multisensory journey. Research in odor detection has opened promising opportunities to expand the chemical dimension of VR/AR technologies. However, olfactory feedback and generation are still in their early stages so far. Current technologies usually rely on large instruments for odor release from bulky and obtrusive equipment or VR headsets. Other drawbacks include feature wired connections and slow response times. Consequently, their potential applications are limited. This section briefly summarizes the two predominant categories of strategies: chemical and electrical stimulation. Heating is a common method used to transform liquids into gases and as a result, serves as a viable solution to building bio-integrated olfactory interfaces. Among various options, wax is a widely used medium due to its biocompatibility and the melting temperature at approximately 50 °C. Fig. 7a depicts an integrated system that heats wax mixed with liquid perfume to generate scents.⁹² At the core of this system are odor generators (OGs), which include a perfume-infused paraffin wax layer, a heating system consisting of gold (Au) traces on a PI substrate with a thermistor for temperature monitoring, and a cantilever made of a Cu coil and a permanent magnet. This cantilever system dynamically controls the temperature by lifting to reduce heat dissipation and increase temperature or lowering to facilitate cooling. The system is housed within a soft silicone frame, and a two OG-design can be comfortably mounted on a user's upper lip. A mask structure design achieving a high channel count supports up to nine OGs within a single device measuring just 18 × 16 × 3 mm, with rapid response time (1.44 s), low power consumption (0.25 W), and light weight (26.3 g). The bio-integrated design has allowed for applications in various scenarios, such as 4D movie watching and smell message delivery, medical treatment, human emotion control and VR/AR based online teaching. In subsequent work by the same group, the olfactory interface model is refined to feature a simplified device design by eliminating the need for a separate thermistor. Instead, the heating electrode functions as the thermistor by monitoring resistance variations. This improvement reduces the size and weight of the OGs, enhances the cooling efficiency and significantly shortens the response time, making it specifically optimized for delivering messages to individuals who are deaf and blind.⁹³ In another advancement from the same group, an enhanced design achieves zero latency through the integration of artificial intelligence (AI) algorithms.⁹⁴ In the MR system, a person wearing VR glasses walks around various odor sources equipped with OGs. Data from the positioning system, wind speed sensor, and temperature sensor at the odor sources, along with the individual's walking velocity, facing direction, and 3D location captured by the VR glasses, are analyzed. This analysis enables the development of an algorithm that preemptively activates the olfactory interface to compensate for the time discrepancy between the person's movement and the odor release from the sources. Fig. 7b illustrates a Peltier-based olfactory display that employs a similar wax heating method.⁹⁵ This display features an aroma-dispensing unit, with each unit comprising several key components. At the top is an aluminum cap designed to hold paraffin wax with essential oil. A Cu plate is positioned beneath the cap to ensure uniform heat distribution, and it houses a commercial thin-film temperature sensor for precise monitoring. The Peltier device, acting as the heat source, is situated between the copper plate and a heat sink, facilitating efficient thermal management. The completed aroma generator weighs 1.7 kg and measures 13.5 × 28 × 14.5 cm. This configuration has been utilized in sensory training to test the detection and recognition times of aromas associated with wines.

Fig. 7 Strategies employed by the research community for generating odors. (a) Using a PI-supported Au trace to heat paraffin mixed with liquid perfume, enabling the generation of various odors,⁹² Copyright 2023, Springer Nature. (b) Utilizing a Peltier device to heat paraffin wax infused with essential oils to produce wine-related odors,⁹⁵ Copyright 2019, IEEE. (c) Employing an air pump and odor filter to release specific scents,⁹⁶ Copyright 2006, IEEE. (d) Using an ultrasonic atomizer, controlled by a Raspberry Pi, to evaporate liquid fragrances,⁹⁷ Copyright 2022, Springer Nature. (e) Leveraging an UVE-TENG to create an EFAE on a BFM, transforming perfume solution into odors,⁹⁸ Copyright 2022, The Authors. (f) Using SAW technology to evaporate liquid fragrances,⁹⁹ Copyright 2018, IEEE. (g) Electrically stimulating the nasal septum to induce trigeminal sensations, which are typically associated with the perception of smells,¹⁰⁰ Copyright 2021, ACM. (h) Applying long, thin electrodes to electrically stimulate the nasal conchae for olfactory activation,¹⁰¹ Copyright 2016, ACM.

An alternative strategy for odor generation is through pumping to create airflow. Fig. 7c illustrates such an olfactory display system.⁹⁶ It has an odor control module, managed via a laptop, which oversees the overall functionality of the system. An odor-generating unit utilizes a commercial DC motor air pump capable of matching the human respiratory rate of 6 L min⁻¹. Air is directed through an odor filter, a tube containing cotton saturated with perfumed water. As air passes through the filter, it becomes infused with the perfume, creating fragrant air. This unit incorporates four air pumps: three channel air through the filter, and one bypasses it. The proportion of airflow between these pumps is adjusted to precisely regulate the intensity of the scent. An odor-presenting unit delivers the scented air directly to the user's nose via a head-mounted tube. Integrated with a positioning system, the display can release odors based on the user's movements.

Ultrasonic atomization is another method for scent release, which features a piezo ceramic component capable of vibrating to vaporize liquid fragrances upon contact. Fig. 7d illustrates an example that utilizes an ultrasonic atomizer.⁹⁷ The vibration is managed by a single-board computer, which controls the on/off timing to regulate the intensity of the emitted odor. This device is featured in a VR scene as a demonstration of its real-world application. Developed in Unity, the game communicates with the olfactory interface device through the representational state transfer (REST) application program interface (API). In the game scene, the device automatically releases a floral fragrance when the user crosses a path lined with flowers.

Electrostatic field accelerated evaporation (EFAE) can improve the evaporation efficiency of liquids by reducing the free energy required for phase transition through the application of a high-voltage electrostatic field. This method enables precise evaporation control and is demonstrated in Fig. 7e, showcasing its use in virtual olfactory generation. The process begins with an electrospray (ES) stage, where an inner cylinder holds perfume.⁹⁸ The perfume is sprayed onto a bionic fibrous membrane (BFM) receiver under the influence of a high-voltage electrostatic field created by an ultrafast voltage-elevation triboelectric nanogenerator (UVE-TENG). The perfume droplets are rapidly absorbed by the BFM receiver, which drives the liquid to the opposite side due to differences in surface energy. In the EFAE stage, the perfume solution evaporates quickly from one side of the BFM onto the skin surface, guided by a spherical electrode. The BFM design effectively divides the gas into two chambers, blocking the natural emission of odor from the inner cylinder. This system achieves an evaporation rate of 0.12 mg s⁻¹, enabling the user to detect the scent within 3 seconds. This design can be leveraged for use in virtual olfactory interfaces and assisted breathing and nasal delivery.

The surface acoustic wave (SAW) steaming phenomenon enables the atomization of odorant liquids by utilizing high-frequency oscillations to disrupt the surface tension of liquids. The SAW technology has also been widely used in odor generation applications. Fig. 7f illustrates an integrated olfactory display incorporating a SAW device.⁹⁹ A solenoid valve is used to dispense liquid droplets in precise, stable amounts onto the SAW device, which operates at a frequency of 9.6 MHz and features an effective atomization area of 8.9 × 16 mm. A DC fan positioned behind the SAW device directs the scent toward the user's nose. This design ensures the accurate and consistent production of scents, and sensory tests confirm its ability to successfully blend different fragrances showing its potential in contributing to scent creation and reproduction using odor components.

Unlike chemical stimulation, electrical stimulation in olfactory research remains less developed with limited success so far due to the risks associated with field testing involving human subjects and the incomplete understanding of how electrical signals influence human perception of smell. Fig. 7g presents an olfactory device that stimulates the user's nasal septum to access the trigeminal nerve, creating a smell-like sensation.¹⁰⁰ The device consists of two PCBs connected by wires, with magnets to secure it across the nasal septum and electrodes on the back of the PCBs to stimulate the trigeminal nerve. This device is highly compact, weighing only 3.4 g and measuring 10 × 23 × 5 mm and 10 × 23 × 7 mm for each nostril. It offers adjustable pulse widths for the output signals. User studies have identified key parameters of the electrical waveform related to smell perception: the absolute electric charge correlates with odor intensity, while the phase order and net charge determine the perceived direction of the smell. Compared to chemical stimulation devices, this electrical design is more compact, as it eliminates the need for reservoirs, pumps, and similar components. Its reliance on electrical signals allows for microsecond-level control, making it highly precise. Additionally, electrically stimulating only the nasal nerves and avoiding direct, invasive electrical stimulation of the olfactory bulb enhances the safety and user acceptance. This design can serve as a substitute for olfactory interfaces to create smell sensations. Fig. 7h showcases an alternative design for electrical stimulation, targeting the nasal conchae, an area with a high density of olfactory receptors.¹⁰¹ The device features two silver electrodes, each 100 mm in length, with bulb-shaped tips measuring 0.5 mm in diameter. These electrodes are designed to deliver DC electrical pulses to the walls of the nasal conchae. A current controller circuit, equipped with a microcontroller, connects to a computer to enable programmed stimulation. However, user experiments have not yet been reported.

3.4. Gustatory interfaces for taste generation and stimulation

As olfactory stimulation evokes expectations about flavor, gustatory simulation fulfills these expectations by delivering artificial taste sensations. This transition from smelling to tasting is both natural and essential in real-world eating and should be mirrored in virtual environments. Simulating taste requires precise control of electrical, thermal, or chemical stimuli, which must be safely and effectively administered to the tongue or oral cavity. These systems allow users to perceive sweetness, saltiness, or even umami, completing the virtual reproduction of food. When combined with olfactory feedback, gustatory simulation not only enhances immersion but also supports applications in health monitoring, flavor training, and sensory rehabilitation. Beyond detecting taste information, recent advancements in the field have focused on developing gustatory interfaces through thermal, electrical, and chemical stimulation. The most direct method of taste restoration is to mix taste substances in specific proportions to match the target flavor. The high-fidelity flavor simulation system shown in Fig. 8a adopts a controllable cartridge structure, containing 18 cartridges: 6 for basic tastes (sweet, sour, salty, umami, and two types of bitterness), 3 for mouthfeel (oiliness, astringency, capsaicin), 6 for aroma simulation, and 3 for color adjustment.¹⁰² It also allows for temperature regulation based on the reconstructed flavor information. By analyzing real flavors, the system uses distilled water as a base and precisely mixes different taste components to simulate target flavors, creating various formulations of virtual orange juice and virtual rooibos tea. Experimental results indicate that virtual flavors closely resemble real flavors in most cases, with low distinguishability. Additionally, the system supports taste adjustment, enabling users to optimize formulations based on personal preferences, with 83.3% of participants preferring the adjusted version. However, the system is bulky, does not allow for continuous taste adjustments, and is challenging to integrate into existing VR/AR devices. Fig. 8b presents a taste simulation system based on ion electrophoresis technology, achieving system integration and dynamic taste control.¹⁰³ The system structure consists of five different gels, each containing NaCl (salty), glycine (sweet), MgCl₂ (bitter), citric acid (sour), and monosodium glutamate (umami). The power supply, gel, the user's hand, and tongue form an electrical circuit. When no voltage is applied, all five tastes can be perceived; when voltage is applied to specific channels, ions move away from the tongue under the influence of the electric field, reducing the perception of certain tastes, achieving “subtractive synthesis”. Experimental results indicate that this device effectively adjusts taste intensity and allows dynamic modification of taste perception. For example, participants could enhance the sourness and saltiness of sushi through electric field adjustments, making the taste more realistic. Additionally, the system maintained stable taste signal output even after 30 minutes of continuous use, demonstrating good durability. However, since this system involves the human body as part of the electrical circuit, the current flows directly through the user, posing potential safety risks. Moreover, the current depends on the electrical resistance of the body, which varies significantly among individuals and even within the same individual under different conditions, resulting in low reproducibility.

Fig. 8 Various existing technologies for taste stimulation. (a) A high-fidelity flavor simulation system with controllable cartridges for virtual taste reconstruction,¹⁰² Copyright 2023, IEEE. (b) A hand-held taste simulation system based on ion electrophoresis for dynamic taste modulation through subtractive synthesis,¹⁰³ Copyright 2020, The Author(s). (c) A portable taste generation system using iontophoresis for precise multi-level taste control,¹⁰⁴ Copyright 2024, The Author(s). (d) Utensils capable of electrical stimulation for sourness and saltiness enhancement,¹⁰⁵ Copyrights 2018, Elsevier. (e) A digital lollipop working based on electrical stimulation for sourness, saltiness, and bitterness perception,¹⁰⁶ Copyright 2016, ACM. (f) A self-powered electronic gum utilizing piezoelectric materials for taste generation through electrical stimulation,¹⁰⁷ Copyright 2018, The Author(s). (g) A temperature-controlled taste interface based on the Peltier effect,¹⁰⁸ Copyright 2018, IEEE. (h) A virtual taste interface combining electrical and thermal stimulation for taste enhancement,¹⁰⁹ Copyright, Springer Nature.

Fig. 8c presents a taste simulation system based on iontophoresis technology, designed to overcome the limitations of ion electrophoresis.¹⁰⁴ In ion electrophoresis, the tongue functions as part of the electrical circuit, and charged taste substances migrate away from the tongue under the influence of an electric field, thereby reducing taste perception. In contrast, iontophoresis applies a voltage across the agar gel, utilizing both electromigration and electroosmosis to enhance taste delivery. Charged taste molecules migrate toward the tongue-contact region via electromigration, while neutral taste molecules move with the solvent as hydrated ions migrate directionally under the electric field, dragging water molecules and dissolved taste substances along. Compared to ion electrophoresis, this approach eliminates the need to incorporate the body into the circuit, making it safer while also minimizing variations caused by individual physiological differences, thereby enabling more precise and more universally applicable taste delivery. The system consists of multiple miniature taste modules, each containing specific taste compounds, including sugar, sodium chloride, citric acid, cherry, passion fruit, green tea, milk, durian, and grapefruit.

By monitoring the current and operation time in each TG channel, the taste interface can transmit data to a paired computer, utilizing a custom-developed data-driven mathematical model to accurately predict the generated taste mass and concentration. Additionally, this system integrates odor generators, creating a combined olfactory/gustatory interface that enables users to simultaneously perceive specific tastes and odors in a virtual environment, thereby enhancing immersion.

The taste substance delivery system provides an intuitive solution for taste restoration; however, it still relies on the transport of chemical substances, making it difficult to precisely control concentration and resulting in a relatively long response time. In contrast, the electrical stimulation method does not require the delivery of chemical substances but instead directly acts on taste receptors on the tongue through electric field modulation. Fig. 8d presents two types of utensils with electrically enhanced sourness and saltiness functions: a pair of chopsticks and a soup bowl.¹⁰⁵ Despite their different forms, both utilize silver electrodes to apply specific electrical pulses to the tongue, thereby generating electric taste perception. Specifically, sourness enhancement is achieved with a current magnitude of 180 μA and a pulse width modulators (PWM) duty cycle of 70%, while saltiness enhancement is achieved with a current magnitude of 40 μA and a PWM duty cycle of 20%. Experimental results indicate that this system significantly enhances the perception of saltiness and sourness in food. More specifically, the electrically stimulated chopsticks increase the perception of saltiness and sourness in unsalted mashed potatoes, while the soup bowl effectively enhances the perception of sourness in diluted miso soup samples.

In addition to integrating electric taste stimulation devices into utensils, alternative implementations include the digital lollipop (Fig. 8e)¹⁰⁶ and electronic gum (Fig. 8f).¹⁰⁷ The digital lollipop consists of a control system and a tongue interface. The control system regulates the current intensity (20 μA–200 μA), frequency (50 Hz–1200 Hz), and polarity (positive or negative current). The tongue interface comprises a spherical electrode (positioned above the tongue) and a flat electrode (contacting the tongue's underside). Experimental results indicate that most participants could perceive sourness, saltiness, and bitterness, while a minority reported a sweet taste when reverse polarity was applied. Furthermore, by adjusting the current intensity, three different levels of sourness were successfully simulated, closely matching real sour tastes.

The electronic gum, in contrast, eliminates the need for an external power source or wiring by utilizing piezoelectric materials. These materials convert the pressure exerted during chewing into electrical energy, enabling continuous taste stimulation. Experimental results demonstrate that most participants could perceive taste sensations, with saltiness and bitterness being the most prominent. Additionally, parallel integration of two piezoelectric elements further enhanced taste intensity. However, generating different tastes is usually challenging using electrical stimulation as it is simply driving a current flow – this limits its capability to a few sensation categories as discussed above.

Temperature, alongside chemical taste stimuli and electrical stimulation, is a key factor influencing taste perception and is often combined with these methods to enhance sensory experiences. Fig. 8g presents a temperature-controlled taste enhancement device, which utilizes a Peltier module to rapidly heat or cool a silver electrode in contact with the tongue by adjusting the current direction.¹⁰⁸ A temperature sensor enables real-time monitoring, ensuring precise closed-loop control. Experimental results indicate that warming induces sensations of sweetness, fattiness, and electric taste, while also significantly enhancing the perception of sweetness and spiciness. Conversely, cooling generates a minty sensation and enhances sourness and saltiness perception.

Fig. 8h illustrates a virtual taste interface designed to simulate taste perception through electrical, thermal, and hybrid stimulation.¹⁰⁹ Similar to previous observations, electrical stimulation alone enables some participants to perceive sourness, saltiness, and bitterness, while temperature changes elicit sweetness and mint sensations. When both stimuli are combined, the perception of sourness and saltiness is further enhanced, demonstrating the potential of hybrid stimulation for taste modulation.

4. Integrated cross-modal systems

4.1. Cross-modal, integrated systems for simulation and augmentation of eating experiences

As discussed in the preceding sections, recent advancements in sensor technologies, together with the growing demand for multisensory experiences, have resulted in developing HMIs that mimic eating experiences in multiple dimensions encompassing the five human sensations. These modalities present unique opportunities for creating immersive and interactive user experiences by delivering rich, intuitive feedback. This section reviews efforts to develop cross-modal systems aimed at simulating, augmenting, or transforming eating experiences and integrating these innovations into next-generation VR/AR technologies.

Recent advancements have demonstrated integrating olfactory and visual stimuli into interactive systems. This approach not only augments the sensory perception of food but also provides a platform for exploring the psychological and physiological impacts of multisensory stimuli on eating behavior. The system described in Fig. 9a integrates a custom olfactory display with a VR environment to achieve a rich eating experience.¹¹⁰ The olfactory display is designed to generate and deliver scents in a controlled manner. The scent generator module includes porous materials soaked with fragrances, which are placed in containers. Scented air is generated using an airflow method where air is pumped through these materials, ensuring precise control over the quantity and concentration of the scent. The scent delivery module uses flexible silicone tubes to transport the scented air from the containers to the subject's nostrils. To ensure appropriate odor localization, separate airflow tubes are used for each nostril, and non-scented air is simultaneously delivered to the other nostril. A breath synchronization mechanism, utilizing a KY-013 temperature sensor, detects inhalation by identifying the temperature drop associated with breathing. The system activates the air pumps precisely at the moment of inhalation to ensure efficient odor delivery. The visual stimuli presentation integrates with an HMD (the Oculus Rift DK2), to present images that complement or contrast with the odors, depending on the experiment's objectives. To maintain the separation of odor streams and prevent mixing, a divider wall made of hard rubber is positioned between the nostrils. The system features a controller and integration module that employs an Arduino platform, such as the Kuman UNO, to coordinate the system. The Arduino board manages input signals from the temperature sensor, activates the pumps, and synchronizes the visual and olfactory stimuli. Finally, the support and housing of the system are designed for ease of use and flexibility. Fragrance containers are secured with small clothespins, allowing for easy replacement and positioning near the nostrils. These containers are often attached to the elastic band of the HMD to minimize delivery delays and ensure consistent scent perception. This designed structural experimental setup can provide controlled and directed olfactory stimulation synchronized with visual input.

Fig. 9 Integrated multimodal system for the simulation, enhancement, and augmentation of eating experiences in various application scenarios. (a) Using the olfactory display to integrate vision and smell in virtual reality environments,¹¹⁰ Copyright 2019, ASME. (b) Using the olfactory feedback device “smell-o-spoon” and VR glasses to achieve visual and olfactory stimulation, creating a sensory experience similar to eating,¹¹¹ Copyright 2023, IEEE. (c) Using the directional scent controller and the acoustic levitation system LeviSense to deliver food to the user's tongue, combining olfactory and gustatory stimulation,¹¹² Copyright 2024, Springer Nature. (d) Using electrical stimulation of taste buds and odor manipulation with “Vocktail” to enhance gustation and olfaction in VR,¹¹³ Copyright 2019, Informa UK Limited. (e) Using head-mounted visual and olfactory displays to achieve cross-modal integration of vision and olfaction, altering the perceived taste of food,¹¹⁴ Copyright 2011, IEEE. (f) Using a system comprising VR glasses, micro air pumps, and check valves to simulate vision, smell, and taste for enhancing flavor,¹¹⁵ Copyright 2021, Springer Nature.

The smell-O-spoon shown in Fig. 9b advances multimodal operation by integrating olfactory delivery directly into a utensil.¹¹¹ This innovation not only enhances the immersion of the VR environment but also ensures a seamless interplay between the physical act of eating and the multisensory virtual experience. The base of the smell-O-spoon is a standard household spoon. It features a mini fan and disperses the odors toward the participant's nose. The fan's speed is controlled through a USB power supply with an on/off switch and a potentiometer set at 500 Ω, which ensures consistent airflow during use. A small metal spring is integrated into the smell-O-spoon to hold the odor samples prepared as smell pads, consisting of tissues infused with some drops of the chosen odor (e.g., banana, cucumber, carrot, or tomato). The smell pads are wrapped in tape to prevent liquid leakage and contamination of the spoon, and they can be easily inserted and removed using tweezers. The smell-O-spoon also incorporates an OptiTrack motion tracking system to monitor the real-world position of the spoon within the VR space. Tracking markers, attached to the spoon using Velcro, ensure the VR system accurately synchronizes the corresponding visual stimuli with the physical movements of the spoon. When working, the user lifts the spoon to eat neutral zucchini mash, and the smell-O-spoon distributes the selected odor. The fan blows the scent towards the participant's nose, simulating the smell of the food they see in VR.

While the integration of visual and olfactory cues provides an engaging starting point, the interplay between smell and taste directly shapes flavor perception, offering opportunities to redefine traditional gustatory experiences and strengthen sensory memory. By precisely controlling these sensory modalities, such systems not only simulate complex taste perceptions but also cater to individual preferences. This section introduces two technologies, LeviSense and Vocktail, that demonstrate how the fusion of olfactory and gustatory modalities can redefine the boundaries of food and beverage experiences. Fig. 9c illustrates the LeviSense, a sophisticated multisensory platform designed to deliver food morsels to users via acoustic levitation while simultaneously integrating synchronized olfactory stimuli.¹¹² Structurally, the core levitation unit can control the independent movement of food morsels in 3D space. The distance sensor releases smells retronasally or orthonasally based on proximity to the food morsels. Acoustic waves transport food particles through levitation, which involves utilizing high-frequency sound waves to create standing wave patterns in the air. The process begins with acoustic transducers emitting sound waves that reflect off a surface, forming standing waves. Food particles are then positioned at the nodes of these standing waves, where the pressure difference counters the force of gravity. By carefully adjusting the frequency and intensity of the sound waves, the position of the food particles can be controlled, guiding them along specific paths. This process ensures the transportation of food particles hygienically and precisely, without physical contact or traditional utensils. For smell delivery, LeviSense employs a custom-built setup consisting of electro-valves, carbon filters, and an ultra-low-noise compressor. This system directs filtered air through small glass bottles containing essential oils or other scent solutions, releasing aromas orthonasally (via the nose) or retronasally (via the mouth). The flow rate, duration, and direction of the airflow can be adjusted for controlled olfactory stimulation. Its contactless and hygienic design makes it suitable for clinical and hygienic applications, such as testing in sterile environments or catering to individuals with specific dietary or sensory needs. Fig. 9d illustrates the Vocktail for multisensory drinking experiences.¹¹³ At its core is a specially designed cocktail glass, which serves as the vessel for the liquid, typically water or another neutral base. The rim of the glass features two silver electrodes that deliver controlled electrical stimulation to the tongue, simulating salty, sour, or bitter tastes based on the frequency of the electrical pulses. When the user places their tongue against the electrodes, the electrical currents stimulate the taste buds responsible for detecting these basic tastes. Different taste receptors respond uniquely to varying pulse frequencies, allowing the electrodes to trigger corresponding taste perceptions. For instance, lower-frequency pulses may simulate salty tastes, medium frequencies might evoke sourness, and higher frequencies can induce the perception of bitterness. The glass also contains three micro air-pumps connected to scent cartridges filled with liquid scents, such as lime, vanilla, squid ink, or chocolate. These pumps release the scents as vapor onto the surface of the liquid to enhance the olfactory aspect of the experience. While LeviSense focuses on delivering a multisensory food experience through the integration of acoustic levitation and olfactory stimulation, Vocktail takes a similar approach to beverages, combining taste and smell to create an equally immersive and customizable drinking experience.

Fig. 9e (ref. 114) and Fig. 9f (ref. 115) illustrate two systems that induce flavor perception through the combination of olfactory and visual cues, focusing on solid and liquid foods, respectively. Fig. 9e shows “Meta Cookie”, a cross-modal gustatory display that can change the perceived taste of real cookies by overlaying visual and olfactory information. The system consists of four parts: a pattern printed plain cookie, a cookie detection unit, an overlaying visual information unit and an olfactory display. During operation, the user wears a head-mounted visual and olfactory display. The cookie detection unit identifies the pattern on the cookie and determines its 6DOF position and the distance between the cookie and the user's nose. After calculating the position, an image of a flavored cookie will be overlaid onto the real one in the headset. Concurrently, the olfactory display emits the scent of a flavored cookie, with the intensity of the scent being calibrated according to the computed distance to the user's nose. This function is accomplished by the olfactory display's multiple air pumps (six in total, with one for fresh air and five for scented air), controllers, and scent filters. There are six air pumps in total, one for fresh air and five for scented air. The intensity of these scents can be adjusted to 127 different levels. By mixing fresh air and scented air, the olfactory display can generate an odor in an arbitrary level of scent at the same air volume. Users can select a preferred cookie from a variety of options, and the chosen cookie's appearance and scent are then projected onto it. Results from the user study show that this system could change the perceived taste, with over 70% of participants associating various flavors with common cookies. The “Meta Cookie” system suggests the feasibility of letting humans perceive various tastes without changing chemical substances. The system structure shown in Fig. 9f is similar to “Meta Cookie”, using an AR display and olfactory display to overlay images on real objects and generate olfactory sensations. It uses OvrVision stereo cameras fixed on the head mounted display to capture the surrounding environment. When a target is detected (such as a beverage cup), the AR scene will be rendered, and the olfactory display will start to generate odors. The olfactory display is made up of two DC micro air pumps, airline tubing for guiding and delivering air, and check valves that serve as small reservoirs using cotton balls soaked in liquid fragrance. When in operation, the DC micro air pumps spray the liquid fragrance from these reservoirs. For example, when consuming the creamer, its appearance is digitally transformed into that of coffee, and the user both sees and smells coffee. These pseudo-gustatory simulations allow users to experience a variety of flavors by altering visual and olfactory stimuli.

4.2. Integrated multimodal system for interactive eating experience

Advancements in interactive technologies have revolutionized the way we approach dining experiences, blending physical, digital, and virtual elements to create innovative solutions for modern challenges. For individuals who face barriers to traditional social dining, whether due to physical distance, mobility limitations, or solitary living, the new systems have emerged to reimagine mealtime as a shared, engaging, and multisensory event. One example, CoDine, appears in Fig. 10a, which enhances remote dining through tactile and sensory interfaces.¹¹⁶ The CoDine system consists of the following four modules: The interaction screen, hosting table, ambient tablecloth, and food teleportation. As the interaction screen uses a Kinect sensor, integrated RGB camera and depth camera, it can realize video conferencing functions and display the other party and their table scene. The operator can select functions, such as delivering food, through gestures. This part is completed by using Open Natural Interaction (OpenNI) API and PrimeSense's Natural Interaction Technology for End-users (NITE) middleware to communicate with other modules through Bluetooth. The hosting table uses electromagnetic adsorption and two-dimensional linear motion control to remotely move bowls and plates. Permanent magnets are installed at the bottom of the bowls and plates, and the electromagnetic components under the table are controlled by stepper motors to move on the x–y axis. When the user selects the menu icon, the electromagnetic components are positioned to the corresponding position and move the bowls and plates over. The ambient tablecloth uses thermochromic ink and Peltier semiconductor modules. The Peltier module reverses the voltage polarity to achieve heating and cooling, so that the thermochromic ink changes color to change the color and pattern of the tablecloth, which is used to display patterns such as heart shapes and expressions, and to convey emotions or blessings. Finally, there is a food teleportation part. When the user selects the food delivery icon through the interactive screen, the remote printing mechanism will be triggered. The three-axis robotic arm controls the position of the print head, and the food extrusion component heats the syringe to liquefy the ingredients for printing, such as printing “LOVE”. Through the cooperation of these four modules, a remote interactive dining experience is achieved. The innovation of CoDine is to go beyond traditional digital communications such as video and audio and enhance intimacy and achieve natural interaction through physical interaction and multi-sensory experience (vision, taste, touch).

Fig. 10 Strategies and applications of remote interactive eating experiences. (a) Using the interactive multi-sensory system “CoDine” to enable gesture-based screen interaction, co-serving, and edible information delivery,¹¹⁶ Copyright 2011, ACM. (b) Using a mixed reality eating device developed with Oculus CV1 HMD and Unity3D to allow users to enjoy meals in a virtual environment,¹¹⁷ Copyright 2018, Elsevier. (c) Using the VR glasses overlay system to present real food within a virtual scene, enabling the experience of eating in a virtual world,¹¹⁹ Copyright 2022, ACM. (d) Using the Oculus Rift CV1 HMD and Intel Realsense SR300 depth sensor to make the physical buffet and virtual living room coexist and interact in real time,¹²¹ Copyright 2019, ACM. (e) System architecture of an interactive metaverse platform based on an ATH-Ring, providing users with cross-spatial perception,¹²² Copyright 2022, Springer Nature.

Another promising direction for enhancing the dining experience is building fully immersive VEs. Here, the integration of advanced HMD devices and depth sensors allows for the creation of compelling mixed-reality dining scenarios. This shift from tangible interfaces to immersive virtual simulations opens up new possibilities for addressing challenges such as the sense of isolation during solitary meals. Fig. 10b illustrates a prototype system that combines an HMD device with a depth sensor to achieve dining in a VE.¹¹⁷ It uses an Oculus CV1 HMD equipped with an Intel Realsense SR300 depth sensor, which is fixed to the device through a helmet-type mounting bracket, and the HMD is connected to a computer. When the user wears the HMD, the Unity3D software engine will create a VE, such as a modern kitchen (including simple furniture, traffic background sounds) and an autumn park (with birdsong, trees and flowers) to achieve dynamic mesh rendering, superimpose real-world elements (such as the user's hands and food) into the VE, and generate a mixed view of the real world and the virtual world. By creating a virtual dining scene, the experience of dining alone can be enhanced, leading to an increase in the user's food intake. This approach also opens possibilities for customization, such as integrating social features. For example, personalized modules can be added, allowing users to engage in shared dining experiences through virtual avatars, simulating the feeling of eating together.

Enhancing eating experiences through VEs offers potential benefits in improving the nutritional intake and overall health, particularly for individuals susceptible to social isolation. Simulating different dining environments or enabling shared virtual meals can help mitigate the psychological effects of eating alone. HMDs are commonly used for such simulations. However, one limitation is that it obstructs the user's view of their real environment (RE) making it impossible to see the food placed in front of them. To address this, a technique known as video see-through (VST) can be implemented.¹¹⁸ VST dynamically adjusts the transparency of the display using the forward-facing camera on the HMD, revealing the RE when the user tilts their head downward to see their food on the table. While this allows the user to see their food, it reduces immersion and the sense of presence in the VE, as the user is intermittently exposed to the RE every time they tilt their head downward. A system that solves the issue without compromising on the immersion, called Ukemochi, is introduced in Fig. 10c.¹¹⁹ This system allows users to experience eating at different locations or environments, while being able to see only their hands and the food that is in front of them in the RE. This technique exposes only the necessary regions and hence does not compromise the immersion. The figure shows the user eating their food in a park with only the food being overlaid into the VE. This level of immersion is achieved by using a smart food recognition system that identifies, segments and tracks the food, and projects that onto the VE using the overlay function in the Open VR API. Similar to the operation of semantic segmentation used for object classification in images, a video object segmentation called SiamMask allows for differentiating the food from the surroundings and tracking in real time with high accuracy.¹²⁰ The Ukemochi system operates using a client-server architecture to achieve this seamless integration of food in the VE. The client module captures the image of the food using the front-facing camera of the HTC VIVE HMD and transmits it to the server module for processing. The server module then performs food region segmentation, employing object detection and tracking algorithms trained on the UECFood100 and VOT-2018 datasets to accurately isolate the food from its surroundings. The processed segmentation data is then relayed back to the client, which computes the corresponding Unity coordinates and overlays the food image into the VE which achieves the desired effect.

In support of these efforts, a team from Denmark has studied the effectiveness of augmented virtuality – a mixture of the real and the virtual world – to simulate people eating together. This was achieved using the Oculus Rift CV1 HMD with an Intel RealSense SR300 depth sensor mounted on top of the HMD. The HMD allows the users to see and hear others and the depth sensor helps reconstruct the food as a textured geometry in the VE. In this experiment, all users were placed in different isolated eating stations that were equipped with this setup. Then, these people were projected into a virtual living room represented by a genderless avatar. These individuals were able to see and communicate with the other participants which simulated eating with others. The left part of Fig. 10d shows a user seated in the isolated eating station and the right shows projected avatars, where the other individuals were also projected to eat together.¹²¹ The findings of the study suggest that a more technologically advanced system capable of rendering true to life avatars, environments and sensations will improve the acceptance and effectiveness of this kind of system. The size and design of the HMD also play a crucial role in the application as it affects the user mobility and eating convenience.

In addition to projecting images and sounds, cross space perception can help tackle some of the limitation of virtual eating by introducing true to life somatic sensations. This gives users the ability to interact with each other's environment in the VE. This effectively reduces the perceived distance between these isolated individuals. Fig. 10e demonstrates a system that can sense cutaneous sensations like temperature, texture, shape and vibrations, and process and transfer this vibro-and thermo-haptic information of an object between two users in the VE.¹²² This system uses minimalistic rings called augmented tactile perception and haptic-feedback rings (ATH-Rings). These rings can be worn on fingers and are equipped with TENG sensors for continuous monitoring of bending gestures, flexible pyroelectric sensors for temperature sensing, eccentric rotating mass (ERM) vibrators for vibro-haptic feedback, and nichrome (NiCr) metal wires for thermo-haptic feedback. The miniaturized size of the rings and the separately positioned IOT module, conveniently placed on the back of the palm for wireless data transmission, enhance the portability and functionality of the system. Pyroelectric and thermoelectric materials used in these sensors enable self-powering and low-voltage feedback elements which reduce the need for a heavy battery pack, which further adds to the portability. The TENG tactile sensors are placed on the inner portion of the ring which senses the swelling of the muscles when the fingers are bent. The temperature sensors are placed on the bottom of the outer portion of the ring which meets the surface of the object that is being held by the user. This information is then processed by the IOT module which consists of a signal processing unit, a wireless transmission unit and a micro-controller unit with an analog to digital converter and PWM. The signal collected by the IOT module is wirelessly transmitted to an AI-enabled cloud server that performs object recognition and finger tracking and projects this information into the metaverse. This projected object can be touched and felt by the users in the VE. The PWM pins in the IOT module control the ERM vibrator mounted on the top of the ring to provide haptic feedback for the whole finger. Similarly, these pins also control the NiCr heating wires placed along the inner part of the ring to stimulate the temperature of the object that is being held. Therefore, with a synergetic working of the sensors, the processing unit and the actuators, a seamless cross space perception has been achieved.

5. Conclusions

This review highlights how recent advancements in wearable technologies can revolutionize eating experiences in next-generation VR/AR applications. While eating-related auditory and visual cues are relatively well-established, with implementations that do not significantly differ from other scenarios, the other components involved in creating eating-related experiences demand further attention. Human perception is highly complex, and replicating these sensations individually or jointly remains a significant challenge. For example, simulating food texture requires capturing a wide range of characteristics, such as hardness, stickiness, viscosity, fracturability, springiness, and chewiness. While pioneering research has introduced initial efforts that simulate some of these cues by varying the amplitude, frequency, and duration of signals, many questions remain unanswered regarding how to achieve a realistic sensation that accurately reflects real-world perception.

Current challenges in this field are discussed as follows: compared to physical sensations, the simulation and generation of chemical signals—primarily olfaction and gustation—remain significantly underexplored, particularly in the context of digital and wearable interfaces. While current methods, such as electrical and thermal stimulation, offer preliminary means of evoking basic sensory perceptions by applying currents or altering temperatures, these techniques remain limited to a narrow spectrum of sensations and often lack specificity, subtlety, and repeatability. For example, electrical stimulation of the tongue can generate tingling or metallic tastes, but cannot yet replicate the nuanced profiles of natural flavors. In this context, delivering chemical components in a digitally controlled manner—such as through microfluidic delivery, aerosol generation, or thermally triggered release—offers greater versatility and has the potential to replicate a broader palette of sensory cues.

However, due to the intricate structure and dynamic responsiveness of the human chemosensory system, particularly the olfactory epithelium and taste buds, it remains impractical to design universal systems capable of replicating all possible chemical sensations. A more feasible pathway is to develop application-specific gustatory and olfactory interfaces, such as those targeting flavor enhancement in food tasting, appetite modulation, or immersive VR experiences—albeit with the trade-off of a limited range of reproducible sensations.

Moreover, the oral cavity and nasal pathways pose unique challenges in terms of bio-compatibility, hygiene, and stability. Given that these environments are moist, sensitive, and highly regulated physiologically, it is critical to establish clear safety standards covering multiple domains—chemical toxicity, thermal thresholds, electrical leakage, and magnetic interference. The development of mouth-integrated or intraoral electronics thus requires rigorous attention to materials selection, sealing strategies, and user acceptability. Currently, this area remains underrepresented in both technological development and regulatory discussion. Safety validation must go beyond theoretical risk analysis; it demands empirical data from user studies and pilot clinical trials. To ensure reusability and address hygiene concerns, reliable sterilization protocols—potentially using UV, autoclaving, or replaceable components—must be standardized for both internal and external surfaces of such devices.

A growing trend in this field is the development of miniaturized, skin- or mouth-integrated HMIs that can be seamlessly and comfortably worn by users for extended periods. Reducing form factors, improving ergonomics, and optimizing user-friendliness will be key to making such systems viable for widespread deployment in VR/AR platforms, especially where interactions involving food, speech, or emotion are concerned. Beyond traditional entertainment or online retail experiences, these systems have transformative potential for biomedical and behavioral research, particularly in understanding, enhancing, or correcting eating behaviors related to disorders such as obesity, dysphagia, or anorexia.

Unlike conventional strategies based on bulky benchtop equipment, wearable and bio-integrated HMIs can capture and deliver sensory feedback in real-time under dynamic, real-life conditions—such as chewing, swallowing, speaking, or multitasking. These interfaces may be integrated with flexible electronics, soft robotics, and biosignal sensors (e.g., EMG, EEG, and ECG), allowing continuous and non-invasive monitoring during complex behavioral tasks.

Looking ahead, a critical direction is to build multimodal HMI systems that simultaneously deliver physical (e.g., texture, pressure, temperature) and chemical (e.g., aroma, taste molecules) feedback. Such systems will offer richer, more immersive representations of food experiences in virtual environments. As illustrated in Fig. 1, integrating both sensing and actuation functions within a unified platform can create bidirectional interfaces that not only provide feedback to the user but also collect physiological data for real-time monitoring, diagnosis, and personalization. Importantly, these systems can enable remote social eating, virtual dining, or telemedicine applications, fostering human connection across distances by synchronizing multisensory inputs. Finally, coupling these interfaces with AI-powered digital twins of the eating process may allow researchers to simulate and predict user preferences, track dietary compliance, or optimize nutritional interventions.

Data availability

No primary research results, software or code have been included and no new data were generated or analysed as part of this review.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

This work was supported by the National Science Foundation ECCS-2223387; the National Institute Of Biomedical Imaging And Bioengineering of the National Institutes of Health R21EB035153; the Defense Advanced Research Projects Agency HR00112410004; the Chronic Brain Injury Pilot Award Program at The Ohio State University; and the Sloan Research Fellowship.

References

1. L. Canetti, E. Bachar and E. M. Berry, Behav. Processes, 2002, 60, 157–164.
2. F. Parasecoli, Discourse, Cult. Organ., 2019, 129–153.
3. A. Krishna and R. S. Elder, Consum. Psychol. Rev., 2021, 4, 121–134.
4. E. T. Rolls, Handbook of Olfaction and Gustation, 2015, 1027–1048.
5. J. V. Verhagen and L. Engelen, Neurosci. Biobehav. Rev., 2006, 30, 613–650.
6. J. S. G. A. Balushi, M. I. A. A. Jabri, S. Palarimath, P. Maran, K. Thenmozhi and C. Balakumar, 2024 3rd International Conference on Applied Artificial Intelligence and Computing (ICAAIC), 2024, pp. 760–765.
7. E. C. Crofton, C. Botinestean, M. Fenelon and E. Gallagher, Innovative Food Sci. Emerging Technol., 2019, 56, 102178.
8. G. Marques and R. Pitarma, New Knowl. Inf. Syst. Technol., 2019, 34–44.
9. G. Marques and R. Pitarma, Appl. Sci., 2019, 9, 438.
10. M. Mehra, S. Saxena, S. Sankaranarayanan, R. J. Tom and M. Veeramanikandan, Comput. Electron. Agric., 2018, 155, 473–486.
11. E. Crofton, N. Murray and C. Botinestean, Foods, 2021, 10, 1154.
12. R. C. Erem, O. Oyekoya and M. Horlyck-Romanovsky, Companion Proceedings of the 2023 Conference on Interactive Surfaces and Spaces, 2023, pp. 24–26.
13. I. Shklovski, L. Barkhuus, N. Bornoe and J. J. Kaye, Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 2015, pp. 1477–1487.
14. G. Du, J. Zhao, Y. Shao, T. Liu, B. Luo, S. Zhang, M. Chi, C. Cai, Z. Liu, S. Wang and S. Nie, eScience, 2025, 5, 100324.
15. X. Hu, Y. Dou, J. Li and Z. Liu, Small, 2019, 15, 1804805.
16. J. Li, J. Yin, M. G. V. Wee, A. Chinnappan and S. Ramakrishna, Adv. Fiber Mater., 2023, 5, 1417–1430.
17. R. Wang, Z. Liu, G. Wan, T. Jia, C. Zhang, X. Wang, M. Zhang, D. Qian, M. J. de Andrade, N. Jiang, S. Yin, R. Zhang, D. Feng, W. Wang, H. Zhang, H. Chen, Y. Wang, R. Ovalle-Robles, K. Inoue, H. Lu, S. Fang, R. H. Baughman and Z. Liu, ACS Appl. Mater. Interfaces, 2019, 11, 10862–10873.
18. E. Ross, Science, 1949, 109, 204.
19. B. Pourkhak, S. A. Mireei, M. Sadeghi and A. Hemmat, Measurement, 2017, 101, 157–165.
20. Y. X. Liu, M. J. Cao and G. M. Liu, Eval. Technol. Food Qual., 2019, 441–463.
21. B. Hou, D. Yang, X. Ren, L. Yi and X. Liu, Nat. Electron., 2024, 7, 777–787.
22. X. Jin, L. Li, S. Zhao, X. Li, K. Jiang, L. Wang and G. Shen, ACS Nano, 2021, 15, 18385–18393.
23. B. Hou, L. Yi, C. Li, H. Zhao, R. Zhang, B. Zhou and X. Liu, Nat. Electron., 2022, 5, 682–693.
24. B. Nicholls, C. S. Ang, E. Kanjo, P. Siriaraya, S. M. Bafti, W. H. Yeo and A. Tsanas, Comput. Biol. Med., 2022, 149, 106068.
25. E. Sazonov, S. Schuckers, P. L. Meyer, O. Makeyev, N. Sazonova, E. L. Melanson and M. Neuman, Physiol. Meas., 2008, 29, 525.
26. Y. Wang, L. Yin, Y. Bai, S. Liu, L. Wang, Y. Zhou, C. Hou, Z. Yang, H. Wu, J. Ma, Y. Shen, P. Deng, S. Zhang, T. Duan, Z. Li, J. Ren, L. Xiao, Z. Yin, N. Lu and Y. Huang, Sci. Adv., 2020, 6, eabd0996.
27. M. K. Kim, C. Kantarcigil, B. Kim, R. K. Baruah, S. Maity, Y. Park, K. Kim, S. Lee, J. B. Malandraki, S. Avlani, A. Smith, S. Sen, M. A. Alam, G. Malandraki and C. H. Lee, Sci. Adv., 2019, 5, eaay3210.
28. C. M. Boutry, M. Negre, M. Jorda, O. Vardoulis, A. Chortos, O. Khatib and Z. Bao, Sci. Robot., 2018, 3, eaau6914.
29. M. A. Mousa, M. Soliman, M. A. Saleh and A. G. Radwan, Sci. Rep., 2021, 11, 3054.
30. X. Wei, J. Tian, C. Wang, S. Cheng, X. Fei, F. Yin, L. Xu and Y. Li, J. Colloid Interface Sci., 2025, 680, 66–76.
31. X. Guo, Z. Sun, Y. Zhu and C. Lee, Adv. Mater., 2024, 36, 2406778.
32. Z. Liu, X. Hu, R. Bo, Y. Yang, X. Cheng, W. Pang, Q. Liu, Y. Wang, S. Wang, S. Xu, Z. Shen and Y. Zhang, Science, 2024, 384, 987–994.
33. A. D. Wilson and M. Baietto, Sensors, 2009, 9, 5099–5148.
34. H. R. Estakhroueiyeh and E. Rashedi, 2015 5th Int. In Conf. Comput. Knowl. Eng. ICCKE, 2015.
35. M. Pardo and G. Sberveglieri, Sens. Actuators, B, 2005, 107, 730–737.
36. S. Capone, M. Epifani, F. Quaranta, P. Siciliano, A. Taurino and L. Vasanelli, Sens. Actuators, B, 2001, 78, 174–179.
37. H. Yu, J. Wang, C. Yao, H. Zhang and Y. Yu, LWT – Food Sci. Technol., 2008, 41, 1268–1273.
38. Z.-b Shi, T. Yu, Q. Zhao, Y. Li and Y.-B. Lan, J. Bionic Eng., 2008, 5, 253–257.
39. X. Ren, Y. Wang, Y. Huang, M. Mustafa, D. Sun, F. Xue, D. Chen, L. Xu and F. Wu, IEEE Sens. J., 2023, 23, 6027–6038.
40. X. Jiang, P. Jia, R. Luo, B. Deng, S. Duan and J. Yan, Sens. Actuators, B, 2017, 249, 533–541.
41. H. Chen, D. Huo and J. Zhang, IEEE Trans. Biomed. Circuits Syst., 2022, 16, 169–184.
42. M. Piłat-Rożek, E. Łazuka, D. Majerek, B. Szeląg, S. Duda-Saternus and G. Łagód, Sensors, 2023, 23, 487.
43. A. Kononov, B. Korotetsky, I. Jahatspanian, A. Gubal, A. Vasiliev, A. Arsenjev, A. Nefedov, A. Barchuk, I. Gorbunov, K. Kozyrev, A. Rassadina, E. Iakovleva, M. Sillanpää, Z. Safaei, N. Ivanenko, N. Stolyarova, V. Chuchina and A. Ganeev, J. Breath Res., 2020, 14, 016004.
44. C. Wang, Z. Chen, C. L. J. Chan, Z. A. Wan, W. Ye, W. Tang, Z. Ma, B. Ren, D. Zhang, Z. Song, Y. Ding, Z. Long, S. Poddar, W. Zhang, Z. Wan, F. Xue, S. Ma, Q. Zhou, G. Lu, K. Liu and Z. Fan, Nat. Electron., 2024, 7, 157–167.
45. N. Dennler, D. Drix, T. P. A. Warner, S. Rastogi, C. D. Casa, T. Ackels, A. T. Schaefer, A. van Schaik and M. Schmuker, Sci. Adv., 2024, 10, eadp1764.
46. H. L. Lee, T. Lee, Z. X. Liu, M. H. Tsai, Y. C. Tsai, T. Y. Lin, S. L. Cheng, S. W. Lee, J. C. Hu and L. T. Chen, Adv. Mater. Res., 2013, 699, 392–397.
47. H. Y. Li, S. N. Zhao, S. Q. Zang and J. Li, Chem. Soc. Rev., 2020, 49, 6364–6401.
48. S. Okur, P. Qin, A. Chandresh, C. Li, Z. Zhang, U. Lemmer and L. Heinke, Angew. Chem., Int. Ed., 2021, 60, 3566–3571.
49. S. Okur, Z. Zhang, M. Sarheed, P. Nick, U. Lemmer and L. Heinke, Sens. Actuators, B, 2020, 306, 127502.
50. P. Qin, B. A. Day, S. Okur, C. Li, A. Chandresh, C. E. Wilmer and L. Heinke, ACS Sens., 2022, 7, 1666–1675.
51. H. Yuan, N. Li, W. Fan, H. Cai and D. Zhao, Adv. Sci., 2022, 9, 2104374.
52. X. Yin, H. Zhang, X. Qiao, X. Zhou, Z. Xue, X. Chen, H. Ye, C. Li, Z. Tang, K. Zhang and T. Wang, Nat. Commun., 2024, 15, 8409.
53. L. Li, S. Xie, F. Zhu, J. Ning, Q. Chen and Z. Zhang, Int. J. Food Prop., 2017, 20, 1762–1773.
54. G. Wang, Y. Li, Z. Cai and X. Dou, Adv. Mater., 2020, 32, 1907043.
55. S. N. Kareem, M. H. Suhail and O. G. Abdullah, Trends Sci., 2023, 20, 5890.
56. Y. Wang, H. Yan and E. F. Wang, Sens. Actuators, B, 2002, 87, 115–121.
57. Y. Deng, M. Zhao, Y. Ma, S. Liu, M. Liu, B. Shen, R. Li, H. Ding, H. Cheng, X. Sheng, W. Fu, Z. Li, M. Zhang and L. Yin, Adv. Funct. Mater., 2023, 33, 2214139.
58. M. Lee, H. Yang, D. Kim, M. Yang, T. H. Park and S. Hong, Sci. Rep., 2018, 8, 13945.
59. G. Wang, Y. Li, Z. Cai and X. Dou, Adv. Mater., 2020, 32, 1907043.
60. Y. Yu, S. Jiang, Z. Cui, N. Zhang, M. Li, J. Liu, H. Meng, S. Wang, Y. Zhang, J. Han, X. Sun, W. Zhao and Y. Liu, Biosens. Bioelectron., 2023, 234, 115357.
61. N. Ranasinghe, P. Jain, S. Karwita and E. Y.-L. Do, Eleventh International Conference on Tangible, Embedded, and Embodied Interaction, 2017, 183–190.
62. Y. Kobayashi, M. Habara, H. Ikezazki, R. Chen, Y. Naito and K. Toko, Sensors, 2010, 10, 3411–3443.
63. Y. Li, N. Langley and J. Zhang, Biosensors, 2023, 13, 414.
64. J. Liu, J. Qian, M. Adil, Y. Bi, H. Wu, X. Hu, Z. Wang and W. Zhang, Microsyst. Nanoeng., 2024, 10, 57.
65. Z. Zhang and B. Yan, Adv. Funct. Mater., 2024, 34, 2316195.
66. J. K. Han, S. C. Park, J. M. Yu, J. H. Ahn and Y. K. Choi, Nano Lett., 2022, 22, 5244–5251.
67. J. Yeom, A. Choe, S. Lim, Y. Lee, S. Na and H. Ko, Sci. Adv., 2020, 6, eaba5785.
68. Y. Yu, S. Jiang, Z. Cui, N. Zhang, M. Li, J. Liu, H. Meng, S. Wang, Y. Zhang, J. Han, X. Sun, W. Zhao and Y. Liu, Biosens. Bioelectron., 2023, 234, 115357.
69. B. Ciui, A. Martin, R. K. Mishra, T. Nakagawa, T. J. Dawkins, M. Lyu, C. Cristea, R. Sandulescu and J. Wang, ACS Sens., 2018, 3, 2375–2384.
70. X. Wang, G. Bai, J. Liang, Q. Xie, Z. Chen, E. Zhou, M. Li, X. Wei, L. Sun, Z. Zhang, C. Yang, T. H. Tao and Z. Zhou, Nat. Commun., 2024, 15, 8967.
71. M. Y. F. Aladin, A. W. Ismail, C. S. Yusof and N. S. Safiee, Encyclopedia of Computer Graphics and Games, 2024, pp. 1267–1275.
72. C. Ebner, S. Mori, P. Mohr, Y. Peng, D. Schmalstieg, G. Wetzstein and D. Kalkofen, IEEE Trans. Visualization Comput. Graphics, 2022, 28, 2256–2266.
73. J. Wang, R. Shi, X. Li, Y. Wei and H. N. Liang, IEEE Trans. Visualization Comput. Graphics, 2024, 2033–2043.
74. N. Rosa, In 2024 IEEE VIS Workshop on Visualization for Climate Action and Sustainability, 2024, pp. 10–16.
75. J. Lee, K. Kaipainen and K. Väänänen, The 23rd International Conference on Academic Mindtrek, 2020.
76. M. Nishizawa, W. Jiang and K. Okajima, The 2016 workshop on Multimodal Virtual and Augmented Reality, 2016.
77. R. Guarese, F. Bastidas, J. Becker, M. Giambastiani, Y. Iquiapaza, L. Macedo, L. Nedel, A. Maciel, F. Zambetta and R. van Schyndel, Cham, 2021, 318–322.
78. M. Nishizawa, W. Jiang and K. Okajima, The 2016 workshop on Multimodal Virtual and Augmented Reality, 2016, pp. 1–6.
79. H. Iwata, H. Yano, T. Uemura and T. Moriya, In IEEE virtual reality 2004, 2004, pp. 51–57.
80. Y. Ikei and M. Shiratori, In Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002, pp. 327–334.
81. A. Niijima and T. Ogawa, The 29th Annual ACM Symposium on User Interface Software and Technology, 2016.
82. V. Shen, C. Shultz and C. Harrison, The 2022 CHI Conference on Human Factors in Computing Systems, 2022.
83. G. Du, J. Zhao, Y. Shao, T. Liu, B. Luo, S. Zhang, M. Chi, C. Cai, Z. Liu and S. Wang, eScience, 2025, 5, 100324.
84. R. Wang, Z. Liu, G. Wan, T. Jia, C. Zhang, X. Wang, M. Zhang, D. Qian, M. J. de Andrade and N. Jiang, ACS Appl. Mater. Interfaces, 2019, 11, 10862–10873.
85. T. Aktar, J. Chen, R. Ettelaie, M. Holmes and B. Henson, J. Texture Stud., 2017, 48, 181–192.
86. A. Niijima and T. Ogawa, The 2016 ACM International Symposium on Wearable Computers, 2016.
87. Y. Huang, J. Zhou, P. Ke, X. Guo, C. K. Yiu, K. Yao, S. Cai, D. Li, Y. Zhou and J. Li, Nat. Electron., 2023, 6, 1020–1031.
88. D. Mukashev, N. Ranasinghe and A. S. Nittala, The 36th Annual ACM Symposium on User Interface Software and Technology, 2023.
89. A. Niijima and T. Ogawa, The 29th Annual ACM Symposium on User Interface Software and Technology, 2016.
90. J. Li, R. Zhang, L. Mou, M. Jung de Andrade, X. Hu, K. Yu, J. Sun, T. Jia, Y. Dou and H. Chen, Adv. Funct. Mater., 2019, 29, 1808995.
91. T. Wang, D. Torres, F. E. Fernández, C. Wang and N. Sepúlveda, Sci. Adv., 2017, 3, e1602697.
92. Y. Liu, C. K. Yiu, Z. Zhao, W. Park, R. Shi, X. Huang, Y. Zeng, K. Wang, T. H. Wong, S. Jia, J. Zhou, Z. Gao, L. Zhao, K. Yao, J. Li, C. Sha, Y. Gao, G. Zhao, Y. Huang, D. Li, Q. Guo, Y. Li and X. Yu, Nat. Commun., 2023, 14, 2297.
93. W. Park, Y. Liu, Y. Jiao, R. Shi, J. Nan, C. K. Yiu, X. Huang, Y. Chen, W. Li, Y. Gao, Q. Zhang, D. Li, S. Jia, Z. Gao, W. Song, M. M. H. Lam, Z. Dai, Z. Zhao, Y. Li and X. Yu, ACS Nano, 2023, 17, 21947–21961.
94. Y. Liu, S. Jia, C. K. Yiu, W. Park, Z. Chen, J. Nan, X. Huang, H. Chen, W. Li, Y. Gao, W. Song, T. Yokota, T. Someya, Z. Zhao, Y. Li and X. Yu, Nat. Commun., 2024, 15, 4474.
95. A. Tiele, S. Menon and J. A. Covington, IEEE Sens. J., 2020, 20, 631–636.
96. T. Yamada, S. Yokoyama, T. Tanikawa, K. Hirota and M. Hirose, IEEE Virtual Reality Conference (VR 2006), 2006, pp. 199–206.
97. M. de Paiva Guimarães, J. M. Martins, D. R. C. Dias, R. d F. R. Guimarães and B. B. Gnecco, Multimedia Syst., 2022, 28, 1573–1583.
98. P. Yang, Y. Shi, X. Tao, Z. Liu, S. Li, X. Chen and Z. L. Wang, EcoMat, 2023, 5, e12298.
99. T. Nakamoto, S. Ito, S. Kato and G. P. Qi, IEEE Sens. J., 2018, 18, 5213–5218.
100. J. Brooks, S. Y. Teng, J. Wen, R. Nith, J. Nishida and P. Lopes, Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021, pp. 1–13.
101. S. Hariri, N. A. Mustafa, K. Karunanayaka and A. D. Cheok, Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality, 2016, pp. 1–4.
102. A. Chalmers, D. Zholzhanova, T. Arun and A. Asadipour, IEEE Comput. Graph. Appl., 2023, 43, 23–31.
103. H. Miyashita, Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–6.
104. Y. Liu, W. Park, C. K. Yiu, X. Huang, S. Jia, Y. Chen, H. Zhang, H. Chen, P. Wu, M. Wu, Z. Liu, Y. Gao, K. Zhu, Z. Zhao, Y. Li, T. Yokota, T. Someya and X. Yu, Proc. Natl. Acad. Sci. U. S. A., 2024, 121, e2412116121.
105. N. Ranasinghe, D. Tolley, T. N. T. Nguyen, L. Yan, B. Chew and E. Y. L. Do, Food Res. Int., 2019, 117, 60–68.
106. N. Ranasinghe and E. Y. L. Do, ACM Trans. Multimedia Comput., Commun., Appl., 2016, 13, 1–22.
107. N. Ooba, K. Aoyama, H. Nakamura and H. Miyashita, Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, 2018, pp. 157–159.
108. K. Karunanayaka, N. Johari, S. Hariri, H. Camelia, K. S. Bielawski and A. D. Cheok, IEEE Trans. Vis. Comput. Graph., 2018, 24, 1496–1505.
109. N. Ranasinghe, Multimed. Tools Appl., 2024, 83, 56517–56548.
110. L. Micaroni, M. Carulli, F. Ferrise, A. Gallace and M. Bordegoni, J. Comput. Inf. Sci., Eng., 2019, 19, 031015.
111. F. Weidner, J. E. Maier and W. Broll, IEEE Trans. Vis. Comput. Graph., 2023, 29, 2423–2433.
112. L. D. Stafford, Smell, Taste, Eat: The Role of the Chemical Senses in Eating Behaviour, 2024.
113. E. Kerruish, Senses Soc., 2019, 14, 31–45.
114. O. Halabi and M. Saleh, Multimed. Tools Appl., 2021, 80, 36423–36441.
115. T. Narumi, T. Kajinami, S. Nishizaka, T. Tanikawa and M. Hirose, Virtual Mixed Reality – New Trends. VMR, 2011, 6773, 260–269.
116. J. Wei, X. Wang, R. L. Peiris, Y. Choi, X. R. Martinez, R. Tache, J. T. K. V. Koh, V. Halupka and A. D. Cheok, Proceedings of the 13th international conference on Ubiquitous computing, 2011, pp. 21–30.
117. D. Korsgaard, T. Bjøner and N. C. Nilsson, Food Res. Int., 2019, 117, 30–39.
118. D. Korsgaard, N. C. Nilsson and T. Bjørner, 2017 IEEE 3rd Workshop on Everyday Virtual Reality (WEVR), 2017, pp. 1–4.
119. K. Nakano, D. Horita, N. Isoyama, H. Uchiyama and K. Kiyokawa, Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–8.
120. Q. Wang, L. Zhang, L. Bertinetto, W. Hu and P. H. Torr, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 1328–1338.
121. D. Korsgaard, T. Bjørner, P. K. Sørensen and J. R. B. Pedersen, Proceedings of the 31st European Conference on Cognitive Ergonomics, 2019, pp. 168–176.
122. Z. Sun, M. Zhu, X. Shan and C. Lee, Nat. Commun., 2022, 13, 5224.

---

Footnote

† These authors contributed equally.

---