Retina-inspired flexible photosensitive arrays based on selective photothermal conversion

Abstract

Vision is a vital system for human perception of the outside world, and more than 80% of the external information received by humans originates from vision. The curved retina and the visual cells with intrinsic color perception are the essential factors to obtain colorful, stereoscopic, and undistorted images. Recent imaging arrays are mainly based on photoelectric conversion, and their color sensitivity relies on extra spectroscopic systems or filters. Here, we propose an intrinsic color perception based on selective photothermal conversion. Different photosensitive liquids have been designed and served as cone cells or rod cells on the retina, and the flexibility of liquids offers the possibility of achieving the curved structure of the retina. As a proof of concept, we succeed in making a flexible photosensitive array with bionic photopic and scotopic vision.

---

Introduction

Visual perception is one of the most important systems for human beings since it receives over 80% of the information from the outside world,^1,2 such as distance, morphologies, and colors. The curved retina, which contains photoreceptor cells with variable color and brightness sensitivities (Scheme 1a), contributes greatly to the acquisition of stereoscopic and colorful visual information.³ Concretely, there are two types of photoreceptor cells in the retina: cone cells and rod cells.⁴ Cone cells can be subdivided into long-wave cones (LW cones, with the maximum absorption peak located in the red light region), middle-wave cones (MW cones, with the maximum absorption peak located in the green light region) and short-wave cones (SW cones, with the maximum absorption peak located in the blue light region). Their selective absorptions of light with different peaks (Scheme 1b) are the significant basis for the color perception in photopic vision.³ Besides, rod cells can respond to the brightness, and they dominate the scotopic vision. Moreover, these photoreceptor cells are distributed on a curved surface, which strictly matches the curved image field of the crystalline lens and avoids the generation of optical aberrations.⁵

Scheme 1 (a) Schematic illustration of the human eye and four types of photoreceptor cells in the retina. (b) Normalized absorption spectra of photoreceptor cells.³ (c) Schematic diagram of the retina-like photosensitive array and three kinds of selective photothermal nanoparticles (SPNs) and broad-spectrum photothermal nanoparticles (BPNs). (d) Normalized absorption spectra of the SPNs and BPNs.

Compared to the retina, current artificial imaging arrays, which are mainly based on photoelectric conversion and could sense the photons with energy higher than the band gap, lack the ability of selective absorption toward light and only respond to brightness like rod cells.⁶ As a result, the color distinction in such arrays requires extra spectroscopic systems or filters.^7,8 Besides, traditional imaging arrays are flat, which would inevitably result in some imaging defects with the existence of the lens, such as field curvature aberration and vignetting.⁹ Using a flexible photosensitive array is a feasible strategy to avoid these optical aberrations.^10–14 However, flexible imaging arrays that can identify light with specific wavelength bands and have selective photosensitivity are rare.^15–18

When light is absorbed, it is unavoidable that part of the energy will be converted into heat: photothermal conversion is also an important but more common method for light utilization of different wavelengths, especially for the light of long wavelength, which could be hard to utilize through photoelectric conversion.^19–21 Current applications of photothermal conversion mainly focus on photothermal therapy^22–24 and water cleaning.^25,26 If the heat from light transfers to thermosensitive materials, the light signal will finally be detected through an electrical signal. Our previous works have proved the feasibility of this method. Flexible and self-healing photodetectors can be fabricated with the combination of photothermal materials and thermosensitive fluidic ionic conductors.^27–29 Based on the photo-thermal-electric method, a photodetector that is sensitive to broad-spectrum light could be prepared and act as rod cells. More significantly, materials with photothermal ability could be modulated and rationally designed to differentially respond to light of different colors, namely selective photothermal conversion (SPC). Intrinsically, cone-like photodetectors with different absorption peaks are realizable with SPC and could be used for color perception.

Herein, artificial vision based on a retina-like photosensitive array with three kinds of selective photothermal nanoparticles (SPNs) and a kind of broad-spectrum photothermal nanoparticle (BPN) has been investigated (Scheme 1c). By modulating the type and size of the metal nanoparticles, three SPNs with different absorption peaks are designed and used to identify colors (Scheme 1d). Carbon nanoparticles with broad-spectrum absorption have been combined with the thermosensitive liquid ionic conductor to mimic rod cells and respond to light intensity. Due to the intrinsic flexibility, retinal-like photodetector arrays could be prepared and all photosensitive pixel dots arrays could distribute on a curved surface.

Results and discussion

Noble metal nanoparticles feature plasmon-based photothermal conversions, in which the plasmon wavelength could be readily controlled by the element, size, and shape of the particles.^30–33 Based on this principle, three kinds of nanoparticles, gold nanorods (Au NR), gold nanospheres (Au NS), and silver nanospheres (Ag NS), have been prepared with absorption peaks falling to the ranges of red light, green light and blue light, respectively, and their maximum absorption peaks are correspondingly 627 nm, 538 nm, and 396 nm (Scheme 1d). The nanoparticles are stabilized by surfactants and ions in the solutions, and the ionic solutions are thermosensitive. The average sizes and the images of their solutions are shown in Fig. 1a–c. Cyan, magenta, and yellow are the optical complementary colors of red, green, and blue, respectively, which are in accordance with the absorption. The solutions of the three particles with selective absorption could act as cones on the retina, and be used for the color perception of SPN-based photosensitive detectors. Besides, carbon nanoparticles (C NPs) have also been prepared with broad-spectrum absorption (Fig. 1d), and 99.9% of visual light could be absorbed by the carbon nanoparticles, which could serve as rod cells and respond to the brightness. The SEM images and corresponding size distributions are provided in Fig. 1e–l and Fig. S1–S3 (ESI†), and the average width and length of the Au NRs are 28.7 nm and 53.0 nm, while the average diameters of the Au NPs, Ag NPs and C NPs in this work are, respectively, 50.1 nm, 20.1 nm, and 50.0 nm. The sizes and shapes of the noble metal nanoparticles agree with their absorptions.^30,33

Fig. 1 (a–d) Schematic diagrams and optical images of the Au NR (a), Au NS (b), Ag NS (c), and C NP (d) solutions. (e–h) SEM images of the Au NRs (e), Au NSs (f), Ag NSs (g), and C NPs (h). (i–l) Size distributions of the Au NRs (i), Au NSs (j), Ag NSs (k), and C NPs (l).

After light absorption, the nanoparticles would convert the energies into heat and result in the increase of the temperature, which would then improve the mobilities of ions in the solution and decrease the resistance. As illustrated in Fig. 2a, three independent factors play important roles in the electric response of red, green and blue light, including light absorbance (α), photothermal conversion efficiency (η) and thermosensitivity (δ). If x, y, and z are the photosensitive coefficients of red, green and blue light, respectively, then they could be expressed as follows:

x ∝ α_r·η_r·δ (1)

y ∝ α_g·η_g·δ (2)

z ∝ α_b·η_b·δ (3)

where the α and η with the subscripts of r, g, and b represent the absorbance and photothermal conversion efficiency of the nanoparticles to red, green and blue light, respectively.

Fig. 2 (a) Schematic diagram of the photo-thermal-electric process. (b–e) Temperature changes under light of different N value in the RGB model: Au NR (b), Au NS (c), Ag NS (d), and C NP (e) solutions are irradiated by red, green, blue, and white light for 5 min, respectively. (f–h) Temperature increasing curves of the Au NR (f), Au NS (g), and Ag NS (h) solutions under irradiation of red (N_r = 255), green (N_g = 255) and blue (N_b = 255) lights within 5 min. (i) Temperature increasing curves of the C NP solutions under irradiation of red (N_r = 100), green (N_g = 100) and blue (N_b = 100) lights within 5 min.

Detailed investigations have been conducted on the photothermal performances of these solutions. A projector is used as the light source, and its color and intensity could be controlled by the RGB values entered, involving N_r, N_g and N_b, which are numbers between 0 and 255 and indicate the relative intensity of red, green and blue light, respectively. As shown in Fig. 2b–d, when Au NR, Au NS and Ag NS solutions are irradiated by red, green and blue light correspondingly, their temperature changes (ΔT) raise with the increase of the RGB values. The thermal images confirm the satisfactory photothermal performance of these nanoparticles (Fig. S4, ESI†). As for the BPN solution (Fig. 2e and Fig. S4, ESI†), white light with the same N_r, N_g and N_b has been used and a similar increase in temperature has been observed when raising the light intensity. It should be noted that the relationships between the temperature changes and RGB values are non-linear, which results from the disproportional increase of light intensity with the rise of the RGB values.

The temperature increases of the solutions under light with different colors are compared in Fig. 2f–h. Two conclusions could be drawn from the results: firstly, different temperature increases under the same light are observed for the solutions of Au NR, Au NS and Ag NS; secondly, the heat production rates of the same nanoparticle solutions under light with different colors are distinct. The differences in both the absorptions and photothermal efficiencies of these nanoparticles give rise to the diverse temperature changes, which are a significant precondition for artificial color perception. Specifically, the heat production rate of Ag NSs is mainly influenced by their absorption, of which the photothermal efficiencies of different lights are close and the temperature changes are generally consistent with their absorption. In terms of Au NRs and Au NSs, the photothermal efficiencies of green light are obviously higher than those of red or blue light (Table S1, ESI†), which coincidentally complies with the fact that human eyes are also more sensitive to green light. And the difference in photothermal efficiencies is similar to that reported in the literature,³⁴ which affect the resulting temperature changes and lead to the disagreement between the absorption and photothermal performances of Au NRs. Additionally, it should be noted that the light intensities and peak widths of the red, green, and blue light sources also play a role in the temperature-rising curves (Fig. S5, ESI†), in which green light has the highest power while red light has the lowest.

In regards to the solution of carbon nanoparticles, lights with different colors and lower intensities (N = 100) are used to respectively irradiate the solution. It is found that the temperature rises more rapidly under green light than under blue light, while it rises more slowly under red light (Fig. 2i). Considering that the photothermal conversion efficiencies (Table S1, ESI†) are investigated for red (N_r = 100), green (N_g = 100), and blue light (N_b = 100), the result also confirms the higher power of green light and the lower power of red light. In general, the carbon nanoparticles are less sensitive to the color but more sensitive to the light intensity, which is similar to the rod cells.

The indirect photoelectric response of nanoparticles mediated by thermal energy has been studied in Fig. 3. Fig. 3a–d show the relationship between the temperature changes and electrical responses of photosensitive chips (Fig. S6, ESI†) irradiated by red (N_r = 255), green (N_g = 255) and blue (N_b = 255) light. It should be noted that all points of the Au NR solution are almost distributed on the same line, of which the slope is its thermosensitivity (δ), namely . Similar results are observed in the Au NS, Ag NS and C NP systems, which confirms that the thermosensitivities are independent of the light color and power.

Fig. 3 (a–d) Resistance responses of Au NR (a), Au NS (b), Ag NS (c) and C NP (d) chips over temperature changes. (e–h) Resistance responses of chips irradiated by light of different N value: Au NR (e), Au NS (f), Ag NS (g) and C NP (h) chips are irradiated by red, green, blue, and white light for 5 min, respectively. (i–l) On–off cycles of the electrical responses for Au NR (i), Au NS (j), Ag NS (k) and C NP (l) chips under the irradiation of red (N_r = 255), green (N_g = 255) and blue (N_b = 255) lights.

As shown in Fig. 3e–l and Fig. S7 and S8 (ESI†), distinguishable and repeatable resistance responses to lights of different power and color have been observed for all the sensors based on SPN. Greater electrical responses are detected for light of higher RGB value due to the production of more heat (Fig. 3e–h). Besides, the selective photothermal conversion of nanoparticles enables the chips to discriminate different colors (Fig. 3i–k), and the carbon nanoparticles with broad-spectrum photothermal conversion could identify the light intensities (Fig. 3l). All the nanoparticle chips exhibit good stability, as shown in Fig. 3i–l and Fig. S7 (ESI†).

Physiologically, after the retina converts light into electrical pulses of nerves through rod cells and cones, the brain would finally integrate these electrical signals into the sensation of light color and intensity. Similarly, a matrix-based algorithm has been developed in this work to convert the electrical responses from SPN-based photosensitive detectors and BPN-based photosensitive detectors into information of color and brightness. Mathematically, the RGB value (N) has firstly been transformed to N′ by the function f, and the resultant N′ is proportional to the actual light power and resistance response (Fig. 4a–c). The linear relationship between N′ and light power simplifies the subsequent calculations. Considering the function f and eqn (1)–(3), it could be concluded that the relationship between the electrical response to monochromic light and its N′ is linear.

Fig. 4 (a–c) Mathematical relationships between N′ and N (blue line) for red (a), green (b), and blue (c) lights; red points are resistance responses under corresponding light of different N′. From top to bottom, the sensors are Au NR (a), Au NS (b) and Ag NS (c) chips, respectively. (d–f) Resistance response of Au NR (d), Au NS (e), Ag NS (f) chips irradiated by red (N_r = 240), green (N_g = 170), blue (N_b = 100) light and polychromic light (N_r = 240, N_g = 170, N_b = 100).

The additivity of the electrical responses is another important requirement for the calculation, and it could ensure that the electrical responses to polychromic light could be decomposed into the superposition of the responses to monochromatic light. As shown in Fig. 4d–f and Fig. S9 (ESI†), the sum of the resistance responses of monochromatic light is equal to that of polychromic light for all SPN-based detectors, which conforms to the expected additivity.

Based on eqn (1–3) and the above results, the electrical responses of SPN-based photosensitive detectors under polychromic light could be expressed as follows:

in which the electrical response matrix is the product of the photosensitive coefficient matrix (P) and N′ matrix. As a result, the light signals based on N′ could be calculated by inversing matrix P as eqn (5), and the difference in the absorption of nanoparticles actually ensures the existence of an inverse matrix. The specific values of the matrices P and P⁻¹ are provided in eqn (S1) and (S2) (ESI†).Based on the mechanism introduced above, a flexible photosensitive array with color and brightness sensitivities was prepared, as shown in Fig. 5a and b. Four kinds of nanoparticle solutions were sequentially filled into the uniform holes of a flexible PDMS substrate, which acted as sub-pixel dots. Four sub-pixel dots of 2 × 2 constitute a pixel, and it can recognize the color and light intensity. PET strips coated with ITO are used as flexible wires on the top and bottom to detect the changes in electrical signals. Transparent double-sided tapes were used to bond different layers. The photosensitive array is flexible and could be bent into a curved surface (Fig. 5c), which is similar to the retina. And the bending of the photosensitive array hardly affects its photosensitive performance (Fig. S10, ESI†). The whole experimental setup is shown in Fig. S11 (ESI†). Under the illumination of light with different colors, the pixel dot can recognize the color like cones and successfully achieve photopic vision (Fig. 5d). In dim light, the pixel dot can also detect the difference in light intensity, and it can perceive the outline of an object, like rods, which successfully achieves scotopic vision (Fig. 5e). As a proof of concept, only 16 sub-pixel dots are fabricated in this demonstration, which could constitute at least 4 pixels. However, it is believed that this method could be used to take images of more pixels, and simplify the lens with good flexibility.

Fig. 5 (a) Schematic illustration of the layered structure of the photosensitive array. (b) Circuit layout and the optical image of the array. (c) Diagram and optical image of the curved photosensitive array. (d and e) Photosensitive array simulating photopic vision under well-lit conditions (d) and scotopic vision in low-light levels (e). Results after mathematical calculations are provided on the right.

Conclusions

Inspired by the structure and function of the retina, three kinds of metal nanoparticles with selective photothermal conversion and carbon nanoparticles with broad photothermal conversion in the visible band have been designed in this work to respectively detect color and brightness. The SPNs serve as cone cells, which have different absorption peaks and are responsible for the color perception in photopic vision. Meanwhile, the BPN acts as the rod cells, and it is sensitive to the brightness in scotopic vision. The photosensitive solutions could be fabricated into flexible photosensitive array, which achieves the simulation of visual cells and the retina. The ultimate device could be used in artificial vision: its flexibility would facilitate the simplification of the lens, and the electrical signals could be mapped to the output color and light intensity mathematically.

Selective photothermal conversion has been proposed and utilized for the first time, and plays a key role in this photo-thermal-electrical method of artificial vision. Encouragingly, the SPC-based photodetector can not only provide intrinsic color recognition but also endow the photosensitive materials with flexibility. We believe that this work provides a unique idea for flexible photoreceptor devices, offering a potential approach for flexible full-color image sensors and bio-inspired vision applications.

Author contributions

Y. H. conceived the idea. X. Z. designed and performed the experiments with the help of Z. C., X. L., X. W. and Y. H. who instructed the operation of the instruments including the SEM, plasma cleaner and magnetron sputtering instruments. X. T. drew the diagram of the photoreceptor and the experimental setup. X. Z. analyzed the results and prepared the first draft of the manuscript. Z. C. and Y. H. revised the manuscript. Y. W. and Y. H. supervised the project and finished the editing of the manuscript.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

This work was financially supported by the National Natural Science Foundation of China (22005336 and 21825503) and the Fundamental Research Funds for the Central Universities, and the Research Funds of Renmin University of China (20XNLG15). Dr. Shimin Zhang and Yingchao Ma are acknowledged for their help with experimental data acquisition.

Notes and references

1. G. Wang, R. Wang, W. Kong and J. Zhang, Cognit. Neurodyn., 2018, 12, 615–624.
2. X. Han, Z. Xu, W. Wu, X. Liu, P. Yan and C. Pan, Small Struct., 2020, 1, 2000029.
3. J. K. Bowmaker and H. J. Dartnall, J. Physiol., 1980, 298, 501–511.
4. C. S. Cowan, M. Renner, M. De Gennaro, B. Gross-Scherf, D. Goldblum, Y. Hou, M. Munz, T. M. Rodrigues, J. Krol, T. Szikra, R. Cuttat, A. Waldt, P. Papasaikas, R. Diggelmann, C. P. Patino-Alvarez, P. Galliker, S. E. Spirig, D. Pavlinic, N. Gerber-Hollbach, S. Schuierer, A. Srdanovic, M. Balogh, R. Panero, A. Kusnyerik, A. Szabo, M. B. Stadler, S. Orgül, S. Picelli, P. W. Hasler, A. Hierlemann, H. P. N. Scholl, G. Roma, F. Nigsch and B. Roska, Cell, 2020, 182, 1623–1640.e1634.
5. Y. L. Lay, R. S. Chang, P. W. Chen and M. C. Tu, Opt. Laser Technol., 1997, 29, 339–344.
6. F. Teng, K. Hu, W. Ouyang and X. Fang, Adv. Mater., 2018, 30, 1706262.
7. L. Gu, S. Poddar, Y. Lin, Z. Long, D. Zhang, Q. Zhang, L. Shu, X. Qiu, M. Kam, A. Javey and Z. Fan, Nature, 2020, 581, 278–282.
8. S. Mariani, V. Robbiano, R. Iglio, A. A. La Mattina, P. Nadimi, J. Wang, B. Kim, T. Kumeria, M. J. Sailor and G. Barillaro, Adv. Funct. Mater., 2020, 30, 1906836.
9. H. T. Zhao, Y. Yang, L. K. Chin, H. F. Chen, W. M. Zhu, J. B. Zhang, P. H. Yap, B. Liedberg, K. Wang, G. Wang, W. Ser and A. Q. Liu, Lab Chip, 2016, 16, 1617–1624.
10. H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C.-J. Yu, J. B. Geddes Iii, J. Xiao, S. Wang, Y. Huang and J. A. Rogers, Nature, 2008, 454, 748–753.
11. I. Jung, J. Xiao, V. Malyarchuk, C. Lu, M. Li, Z. Liu, J. Yoon, Y. Huang and J. A. Rogers, Proc. Natl. Acad. Sci. U. S. A., 2011, 108, 1788.
12. G. J. Lee, C. Choi, D.-H. Kim and Y. M. Song, Adv. Funct. Mater., 2018, 28, 1705202.
13. C. Xie and F. Yan, Small, 2017, 13, 1701822.
14. Z. Rao, Y. Lu, Z. Li, K. Sim, Z. Ma, J. Xiao and C. Yu, Nat. Electron., 2021, 4, 513–521.
15. C. Jo, J. Kim, J. Y. Kwak, S. M. Kwon, J. B. Park, J. Kim, G.-S. Park, M.-G. Kim, Y.-H. Kim and S. K. Park, Adv. Mater., 2022, 32, 2108979.
16. Z. Lan and F. Zhu, ACS Nano, 2021, 15, 13674–13682.
17. Y. Pei, L. Yan, Z. Wu, J. Lu, J. Zhao, J. Chen, Q. Liu and X. Yan, ACS Nano, 2021, 15, 17319–17326.
18. Q. Wang, G. Zhang, H. Zhang, Y. Duan, Z. Yin and Y. Huang, Adv. Funct. Mater., 2021, 31, 2100857.
19. H. Lin, X. G. Wang, L. D. Yu, Y. Chen and J. L. Shi, Nano Lett., 2017, 17, 384–391.
20. J. Wang, Y. Y. Li, L. Deng, N. N. Wei, Y. K. Weng, S. Dong, D. P. Qi, J. Qiu, X. D. Chen and T. Wu, Adv. Mater., 2017, 29, 1603730.
21. Y. Q. Qi, T. Jin, K. Yuan, J. Y. You, C. Shen and K. Y. Xie, J. Mater. Sci. Technol., 2022, 127, 144–152.
22. L. Zhao, Y. Liu, R. Chang, R. Xing and X. Yan, Adv. Funct. Mater., 2019, 29, 1806877.
23. T. Sun, H. Liu, N. Jiang, Q. Wu, C. Li, R. Xia, B. Gao and Z. Xie, Adv. Funct. Mater., 2021, 31, 2103714.
24. C. Ji, W. Cheng, Y. Hu, Y. Liu, F. Liu and M. Yin, Nano Today, 2021, 36, 101020.
25. D. Weng, F. Xu, X. Li, Y. Li and J. Sun, J. Mater. Chem. A, 2018, 6, 24441–24451.
26. X. Chen, N. Yang, Y. Wang, H. He, J. Wang, J. Wan, H. Jiang, B. Xu, L. Wang, R. Yu, L. Tong, L. Gu, Q. Xiong, C. Chen, S. Zhang and D. Wang, Adv. Mater., 2022, 34, 2107400.
27. Y. He, X.-Q. Xu, S. Lv, H. Liao and Y. Wang, Langmuir, 2019, 35, 1192–1198.
28. H. Jia, X. Tao and Y. Wang, Adv. Electron. Mater., 2016, 2, 1600136.
29. Y. He, S. Liao, H. Jia, Y. Cao, Z. Wang and Y. Wang, Adv. Mater., 2015, 27, 4622–4627.
30. H. Chen, L. Shao, T. Ming, Z. Sun, C. Zhao, B. Yang and J. Wang, Small, 2010, 6, 2272–2280.
31. X. Ye, C. Zheng, J. Chen, Y. Gao and C. B. Murray, Nano Lett., 2013, 13, 765–771.
32. Y. Zheng, X. Zhong, Z. Li and Y. Xia, Part. Part. Syst. Charact., 2014, 31, 266–273.
33. Y. Wan, Z. Guo, X. Jiang, K. Fang, X. Lu, Y. Zhang and N. Gu, Colloid Interface Sci., 2013, 394, 263–268.
34. H. Zhang, H. Yang, H. J. Chen, X. Du, D. Wen and H. Wu, Energy, 2017, 141, 32–39.

---

Footnote

† Electronic supplementary information (ESI) available: Details of synthesis and measurements, SEM of nanoparticles, thermal images of chips, spectrum of projector light, photothermal conversion efficiency ratio of nanoparticles, resistance response of nanoparticle chips in different light and schematic diagram of the experimental setup. See DOI: https://doi.org/10.1039/d2tc04010g

---