Owing to their unique properties, nanomaterials have been widely used in biomedicine since they have obvious inherent advantages over traditional ones. However, nanomaterials may also cause dysfunction in proteins, genes and cells, resulting in cytotoxic and genotoxic responses. Recently, more and more attention has been paid to these potential toxicities of nanomaterials, especially to the risks of nanomaterials to human health and safety. Therefore, when using nanomaterials for biomedical applications, it is of great importance to keep the balance between therapeutic efficiency and cytotoxicity (i.e., increase the therapeutic efficiency as well as decrease the potential toxicity). This requires a deeper understanding of the interactions between various types of nanomaterials and biological systems at the nano/bio interface. In this review, from the point of view of theoretical researchers, we will present the current status regarding the physical mechanism of cytotoxicity caused by nanomaterials, mainly based on recent simulation results. In addition, the strategies for minimizing the nanotoxicity naturally and artificially will also be discussed in detail. Furthermore, we should notice that toxicity is not always bad for clinical use since causing the death of specific cells is the main way of treating disease. Enhancing the targeting ability of nanomaterials to diseased cells and minimizing their side effects on normal cells will always be hugely challenging issues in nanomedicine. By combining the latest computational studies with some experimental verifications, we will provide special insights into recent advances regarding these problems, especially for the design of novel environment-responsive nanomaterials.