Jonathan P.
McMullen
*a and
Jon A.
Jurica
b
aProcess Research and Development, Merck & Co., Inc., P.O. Box 2000, Rahway, NJ 07065, USA. E-mail: jonathan.mcmullen@merck.com
bAnalytical Research and Development, Merck & Co., Inc., P.O. Box 2000, Rahway, NJ 07065, USA
First published on 22nd May 2024
The field of reaction engineering is in a constant state of evolution, adapting to new technologies and the changing demands of process development on accelerated timelines. Recent advancements in laboratory automation, data-rich experimentation, and machine learning have revolutionized chemical synthesis research, bringing significant enhancements to reaction engineering. To showcase these advantages, this study introduces a machine-assisted process development workflow that uses data-rich experimentation to optimize reaction conditions for drug substance manufacturing. The workflow adopts a scientist-in-the-loop approach, ensuring valuable contributions and informed decision-making throughout the entire procedure. Two case studies are presented: a copper-catalyzed methoxylation of an aryl bromide and the global bromination of primary alcohols in gamma-cyclodextrin. In addition to identifying the optimal reaction conditions, the workflow emphasizes the importance of process knowledge. Data-driven reaction models are constructed for both case studies, showcasing how early-stage reaction data can inform late-stage process characterization and control strategies. The speed and efficiency offered by the machine-assisted approach enabled complete reaction optimization and reaction modeling in one week, approximately. This reaction data, along with other process knowledge obtained throughout development, highlight the future prospects for reaction engineering in drug substance development. As the field continues to embrace innovative technologies and methodologies, there is vast potential for further advancements in reaction engineering practices, leading to more streamlined and efficient process development and accelerating the discovery and optimization of chemical manufacturing processes.
Automation and data-rich experimentation (DRE) have revolutionized process development by offering enhanced capabilities at the lab benchtop.1 The integration of process analytical tools, such as on-line HPLC2,3 and in situ spectroscopy,4–7 increases the data density in each experiment, providing information on both the reaction kinetics and the overall reaction performance. Enhancements to throughput can be achieved through the utilization of parallel reactor technology, integrated with automated sampling strategies, allowing for collection of comprehensive reaction profiles.8 This enables a deeper understanding of the correlation between reaction inputs and process dynamics.9 Moreover, advancements in analytical equipment along with data analytics enable a higher throughput of off-line samples with a more straightforward visualization of results.
In recent years, the application of feedback optimization algorithms to guide experiments towards desired conditions has proven to be an effective and efficient approach to process development.10–14 This is particularly evident in scenarios with complex syntheses with multiple reaction variables. Initially implemented in flow reactor systems, where sequential experiments can be easily modified by adjusting flow rates, early examples identified local solutions to simple optimization problems.15–22 However, more recent demonstrations have showcased the use of sophisticated operations, algorithms, and multifaceted objective functions.12,23–33 To further expand this methodology in chemical synthesis research, similar approaches are required for reactions where batch operations are preferred.
When selecting a batch reaction optimization platform, careful consideration of various features is essential to ensure that the experimental hardware and chosen algorithm align with the goals of drug substance development. The choice of reactor technology should be made based on factors such as desired throughput, the physical properties of the reaction, and the required data density. Plate-based reactors may be appropriate when there is limited starting material and the aim is to optimize reaction performance across multiple solvents, bases, and catalysts.34,35 Larger reactors with overhead stirring may be more appropriate when the goal is to study the reaction with scale-up considerations. When it comes to the optimization approach, selecting an algorithm that is reputable for its experimental efficiency is obviously important.36,37 However, equally important is choosing an algorithm that aligns with the process uncertainties and the experimental workflow. Consideration should be given to size of the reaction design space, the algorithm's ability to effectively handle experimental noise, its ability to update the objective function and experimental constraints, and the suggested number of experiments with each iteration (e.g., single sequential or multiple parallel experiments). These features can be especially valuable during early process development, when flexibility and overall experimental speed are crucial.
This work presents one methodology for machine-assisted batch reaction optimization with data-rich experimentation, emphasizing important considerations in the process. To illustrate the approach, two case studies are provided: the first involves the copper-catalyzed methoxylation of an aryl bromide38 (Scheme 1), while the second focuses on the global bromination of primary alcohols in gamma cyclodextrin (Scheme 2).39 In both cases, a “scientist-in-the-loop” approach was employed to contribute valuable insights and filter the data effectively. The experimental design and algorithmic operations were carefully selected to ensure that iterations of experimentation, data acquisition, and analysis could be completed within one week, aligning with common drug substance development timelines. Furthermore, post-run data-driven reaction modeling was conducted, highlighting how the optimization process can be further utilized to extract kinetic information and provide design space details for formal process characterization.
Scheme 2 Bromination of γ-cyclodextrin via Bromo–Vilsmeier reagent (4) to produce drug substance intermediate Broomdex (5). |
The temperature range offered by the AmigoChem workstation was considered suitable for conducting optimization investigations, and the Teflon coated stir bar ensured adequate mixing for the methoxylation's thin slurry nature and the homogeneous bromination. These cursory suitability checks instilled confidence that the optimization results would translate well to other equipment and scales. These features also made the AmigoChem the ideal system for these specific optimization investigations due the low starting material consumption per experiment, the parallel reactor technology enabling the desired experimental throughput, and the data-rich reaction sampling to enable essential kinetic and stability information to support process modeling.
During the reaction, 40 μL samples were collected by the AmigoChem and diluted with 960 μL of a quench solution (4:1 v/v acetonitrile: acetic acid solution) in a 2 mL UPLC vial. Post experiment, serial dilution (5×) of samples was performed by combining 200 μL sample with 800 μL quench. Analysis of reaction results was performed on Agilent 1200 series UPLC, using an Acquity UPLC BEH C18 column (1.7 μm × 2.1 mm × 100 mm, P/N: 186002352) with detection at 210 nm. Reaction results were reported as liquid chromatography area percentage (LCAP). See the ESI† for additional analytical information.
During the reaction, 250 μL samples were collected by the Amigochem and added to 2 mL HPLC vials containing 50 μL of water. Upon reaction completion, the samples were removed, 15 μL of 48% HBr (Thermo Scientific) added, stir bars added and placed on a tumble stirrer heated to 40 °C for 5.5 h. The samples were then cooled to room temperature, 1.5 mL of 50% DMSO (Sigma Aldrich)/Acetonitrile (Sigma Aldrich) was added and then analyzed by UPLC. See the ESI† for additional analytical information.
The stable noisy optimization by Branch and Fit (SNOBFIT) algorithm developed by Huyer and Neumaier40 was implemented in this machine-assisted reaction optimization demonstration, although numerous alternative algorithms could have been considered. Because it is a black-box approach, mechanistic information is not required. The algorithm accounts for noisy measurement responses, which is a common occurrence in experimental settings, and allows the user to specify the resolution in optimization variables. Each call to the algorithm is considered an iteration, and the number of experiments reported per iteration can be tailored to suit the specific capabilities of the experimental technology employed. For instance, in the present work, 10 experiments were executed per call to the algorithm, coinciding with the parallelizability of the system, enabling 10 reactions to be conducted simultaneously per iteration. The experimental throughput played an important role in selecting the SNOBFIT algorithm as it can parallelize search experiments. In contrast, traditional Bayesian optimization approaches are typically sequential in nature, although ongoing research is addressing this limitation.41,42 Specific SNOBFIT algorithm details for both case studies, including parameter settings and implementation specifics, can be found in the ESI.†
The optimization of the reaction conditions involved adjusting several key variables, including the reaction temperature, copper bromide loading, DMF equivalents, sodium methoxide solution equivalents, and reaction time. The lower and upper bounds for each variable can be found in Table 1. In the first call to the SNOBFIT algorithm, 10 space-filling points were generated to provide essential information for subsequent search experiments. During these initial experiments, the objective function was not formally defined yet, as the reaction outcomes under the diverse range of conditions would determine which reaction properties to exploit and which to avoid. While the SNOBFIT algorithm requested a single time measurement for each experiment, multiple measurements were taken throughout the course of each reaction. This time-series information was used to assess whether kinetic phenomena, such as reaction stalling or product degradation, should be incorporated into the objective function. Moreover, a subset of these reaction results that were of interest to the scientist were added to the optimization routine.
The reaction results of the initial call to the algorithm are provided in the ESI,† and revealed several noteworthy trends that influenced the subsequent optimization process. In general, all experiments resulted in incomplete conversion and only several experiments resulted in modest product yield. Considering this observation, the upper bounds for both temperature and reaction time were extended to explore conditions with faster kinetics and complete conversion (see Table 1 footnote). Concerning the reaction performance, the profiles did not unveil any significant issues regarding impurity generation or product degradation. Therefore, the reaction optimization procedure presented an opportunity to maximize yield and process efficiency. The ability to analyze preliminary reaction data, formulate informed objectives, and update the feasible design space are key advantages of the scientist-in-the-loop optimization method. This increased flexibility leads to more significant improvements in subsequent rounds of reaction optimization.
For this methoxylation, the optimizer aimed to maximize a two-term objective function, as outlined in Table 1. The first term represented product yield, while the second term served as an incentive to minimize catalyst usage, aligning with a common goal in reaction development. The round 1 reaction results were converted to objective function values, and are provided in Fig. 1a.
Fig. 1 Optimization results for methoxylation investigation with objective function and experimental conditions denoted for a) the first round, b) the second round, c) the third round, and d) the fourth round, along with e) a strip chart correlating experimental coded conditions (see ESI†) with end of reaction (EOR) 2 LCAP for all runs, with grey background denoting experiments selected for space-filling design or targeted in unexplored regions. |
Following the initial round of experiments, subsequent iterations using the SNOBFIT algorithm actively searched for the optimal reaction conditions. Each iteration involved conducting 10 experiments with reaction temperatures, charge amounts, and reaction times specified by the algorithm. Within each iteration, seven of the 10 experiments were dedicated to searching for the local optimum, while the remaining three searched unexplored regions to better ensure that the best conditions corresponded to a global optimum. In each experiment, the requested SNOBFIT sample time was either collected or substituted with a similar time point already present in the reaction profile. Once the round of experimentation was completed, the entire reaction profile data was collected, analyzed, and utilized to calculate the objective function values. However, only the objective function value for the SNOBFIT time point and other user-selected points were inputted into the algorithm. This approach capitalized on the nearest-neighbor algorithm utilized by SNOBFIT to construct surrogate models and to perform iterative searches for optimal conditions, allowing for a more comprehensive exploration of experimental conditions and preventing bias of the sample time. Utilizing more data in the algorithm may be possible in future works through appropriate scaling of reaction variables.
The reaction results from rounds 2 to 4 of the optimization process are depicted in Fig. 1b–d, respectively. To effectively summarize the reaction performance (y-axis) over time (x-axis), the data markers are encoded with various properties, such as type, size, interior color, and outline color, to represent the multidimensional inputs of the reaction. Compared to the initial search experiments, shown in Fig. 1a, the algorithm rapidly identified an ensemble of reaction conditions during the round 2 search (Fig. 1b) that collectively exhibited faster reaction rates and higher conversions. This trend of performance improvement continued in rounds 3 and 4 (Fig. 1c and d), except for experiments intentionally targeting unexplored regions. Reaction conditions for SNOBFIT points with the maximum objective function and those yielding the highest product yield are detailed in Table 2. Furthermore, all reaction conditions and their corresponding product profiles from each experiment can be found in the (ESI†).
Experiment | Maximum | Temperature (deg. C) | CuBr (mol%) | NaOMe (eq.) | DMF (eq.) | Time (h) | Obj. Fun. | LCAP 2 |
---|---|---|---|---|---|---|---|---|
14 | Objective Function | 83 | 6.0 | 5.0 | 2.88 | 25.2 | 97.2 | 92.8 |
34 | LCAP 2 | 76 | 8.0 | 5.0 | 2.88 | 28 | 96.9 | 95.0 |
The end-of-reaction product LCAP and the corresponding conditions requested by the SNOBFIT algorithm are displayed in Fig. 1e. The clear background data points represent experiments directed by the algorithm, leading toward the optimal reaction conditions. In contrast, the shaded regions denote experiments selected by the algorithm for the initial space-filling design or in unexplored areas of the design space, providing comprehensive search coverage. This plot explains how the algorithm converged towards conditions at the upper ranges of temperature, DMF equivalents, and NaOMe equivalents, while identifying an appropriate CuBr loading to achieve higher conversion under these conditions. The alignment between expected trends and the performance of the algorithm in penalizing experiments with elevated CuBr levels showcases the logical nature of the operating conditions. This congruence between expectations and algorithm output is an important step in validating machine-assisted process workflows into drug substance development.
While the optimization results are directly impactful to process development, leveraging the data to build process knowledge that transcends development life cycle is just as important. One such approach is presented below by interrogating the data through machine learning practices and data-driven modeling, though alternative approaches to extract similar knowledge exist.43–48
To gain a better understanding of the reaction performance, a clustering algorithm was used to identify experimental data in the vicinity of the highest yielding run, experiment 34. From the SNOBFIT reaction dataset, 15 experiments were identified as being reasonably close to the optimal conditions using the Mahalanobis distance as a metric (see ESI† for approach and corresponding experiments). The reaction results from the set of local experiments were then modelled using functional principal component analysis (FPCA).49–53 FPCA is a branch of functional data analysis (FDA), an applied statistics discipline that involves the analysis and regression of data objects, curves, or functions rather than individual points.49 In this work, the FPCA methodology utilized the principal analysis by conditional estimation (PACE) algorithm, allowing its application to datasets with sparse or dense sampling strategies.50–52 For the sake of brevity, the discussion below is streamlined to provide a basic understanding of the modeling framework applied to the reaction results. A more detailed description of the methodology is provided in the ESI† and supported by the referenced literature.
Similar to principal component analysis (PCA), FPCA effectively reduces the dimensionality of the data by identifying functional principal components that account for longitudinal (time) variations, along with the corresponding FPC scores that elucidate the variation across experiments. Mathematically, this is provided by eqn (1), where ŷ(t) is the FPCA prediction for the reaction response time profile, μ(t) is the functional mean, ϕk(t) is a set of k eigenfunctions, also referred to as the functional principal components (FPC), and ξik is the corresponding FPC score in experiment i. To move from a descriptive model to a predictive model, the FPC scores can be treated as a separate response and regressed against the experimental input factors, as in eqn (2), where β terms are regressed coefficients, xp and xq represent indexed input factors, and NF is the number of factors.54 See ESI† for more information.
(1) |
(2) |
Using the manner described above, a FPCA model with regressed FPC scores was performed and the results are provided in Fig. 2. Statistical analysis associated with the FPC score regression is provided in the ESI.† As the results illustrate, there is excellent agreement between the experimental data, and the FPCA model using both the numerically derived FPC scores and the regressed scores (eqn (2)) for many of the experiments. While some discrepancies exist at early reaction times (e.g., experiments 16, 17, 32), the agreement improves toward the end of the reaction where conversion predictions matter most.
Fig. 2 Functional principal component analysis (FPCA) model for a set of methoxylation results in the local neighborhood around the optimal yield conditions (experiment 34) with experimental data (), FPCA model (eqn (1), ), and FPCA model with regressed coefficients (eqn (2), ) overlaid. |
Notably, as the objective is to gather this reaction optimization data in the early stages of development, these models can be utilized to identify robust operating spaces as the process matures and development progresses towards manufacturing. To demonstrate this potential, the developed FPCA model was used to estimate the product response as a function of temperature and copper bromide within the design space used for model development. As the contour profiles in Fig. 3 suggest, a wide operating space that achieves target reaction performance can be achieved by tuning the temperature, CuBr loading, and reaction time.
Fig. 3 Methoxylation reaction profiles estimated through FPCA model as function of temperature and CuBr using coded values (see ESI†), keeping DMF and NaOMe equivalents at optimized values (see Table 2). |
The assay yield from the initial 10 space filling experiments performed by the SNOBFIT algorithm are presented in the ESI.† Notably, these experiments demonstrated the diverse range of reaction behaviors achievable within the defined design space. This encompassed conditions with low conversions, those exhibiting fast kinetics and high conversion, as well as instances leading to rapid product degradation. Such product instability poses significant challenges for control strategies that rely on in-process quality control samples with long turnaround times. Consequently, the objective was to identify conditions that would result in high product yield while also ensuring product stability. To achieve this goal, the objective function to maximize, as outlined in Table 3 incorporated a penalty function to discourage experimental conditions in which notable product degradation was observed. Furthermore, it is noteworthy to mention the artificially low assay yield in several data points due to an imperfect sampling protocol. These data points were removed from consideration for the purposes of optimization and post-run analysis (see ESI† for specific samples). This observation emphasizes one of the critical needs for a scientist-in-the-loop machine assisted development. Objective function values for this first set of experiments are provided in Fig. 4a.
Results from optimization rounds 2–4 for the bromination are illustrated in Fig. 4b–d, respectively. In each round, the algorithm allocated seven experiments to local optimization searches and three experiments in unexplored regions to ensure the identification of the global optimum. The reaction profiles convincingly demonstrate the algorithm's effectiveness in rapidly identifying experimental conditions that result in fast kinetics, high conversion, and good stability as evident in the first set of search conditions of round 2 (Fig. 4b). Similar outcomes continued throughout the subsequent optimization rounds. Conditions corresponding to the optimal objective function are provided in Table 4. Because the penalty function was not active in this run, the optimal objective function corresponds to maximum yield in this result. See the ESI† for complete bromination optimization data.
Experiment | Temperature (deg. C) | Volume (mL g−1) | 4 (eq.) | Time (h) | Obj. Fun. |
---|---|---|---|---|---|
40 | 80 | 17.6 | 22.6 | 20 | 97.8% |
In contrast to the previous methoxylation case study, the optimization investigation in this scenario yielded limited dynamic data. Nevertheless, this data can still serve as a valuable source for generating crucial process knowledge in subsequent development stages. To demonstrate this, experimental data within a local neighborhood surrounding the optimal conditions for the assay yield at 8 hours were utilized to construct a response surface model. Through stepwise linear regression analysis of 20 experiments, a statistically significant quadratic model was derived (see ESI†). The resulting contour plot, depicted in Fig. 5, illustrates the relationship between temperature and equivalents of 4 in relation to the assay yield. This plot emphasizes the sensitive nature of the reaction and the need to strike a balance between temperature and Bromo–Vilsmeier reagent equivalents to achieve the desired conditions. Notably, at higher reaction temperatures, elevated levels of 4 were found to enhance product stability. These interesting findings were further validated through detailed characterization experiments and mechanistic studies conducted in separate research work.39
Fig. 5 Assay yield response surface model over range of temperature and equivalents of 4 around the optimal reaction yield at 8 hours with DMF set at 15 mL g−1. |
Applying similar workflows in early drug substance programs have the promise to greatly accelerate overall process development timelines when the optimization data is leveraged throughout the research lifecycle. In this work, that data bridge was exemplified by using data-driven modeling methods. Functional principal component analysis was demonstrated as a powerful process modeling tool for the methoxylation kinetics; whereas, in the bromination case study, a more standard empirical model was established using a nearest neighbor algorithm and stepwise regression. Though beyond the scope of this individual work, the greatest gains for this approach are met when the optimization results are for long-term route development in early development are retained and leveraged in late-stage process characterization to streamline quality risk assessment and control strategy selection.
The outlook for machine-assisted process development and similar methodologies for reaction engineering are highly promising. Continued advancements in data throughput, data-rich experimentation, data engineering and analytics, as well as optimization methods, will play pivotal roles in driving significant improvements in these reaction optimization approaches. Beyond technical growth, successful implementation and sustainability of machine-assisted process development approaches require careful consideration of distribution and adoption of the technology. One barrier on this front is the initial uncertainty associated with the resource commitment, including time and material, as well as the expected gains from the optimization investigation compared to the current state. To facilitate widespread adoption, the development of numerical methods that can confidently establish tight upper and lower bounds around the expected benefits and so-called costs using minimal experimental data points will be invaluable.
Overall, with continued advancements and efforts in these areas, machine-assisted process development will continue to revolutionize reaction engineering, enabling more efficient and effective optimization strategies and driving progress in chemical manufacturing process.
Footnote |
† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d4re00141a |
This journal is © The Royal Society of Chemistry 2024 |