University of Southern California Dissertations and Theses

The implementation of data-driven techniques for the synthesis and optimization of colloidal inorganic nanocrystals

(USC Thesis Other)

The implementation of data-driven techniques for the synthesis and optimization of colloidal inorganic nanocrystals

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content
THE IMPLEMENTATION OF DATA-DRIVEN TECHNIQUES FOR THE
SYNTHESIS AND OPTIMIZATION OF COLLOIDAL INORGANIC
NANOCRYSTALS

By

Emily Mae Williamson

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(CHEMISTRY)

August 2023

Copyright 2023 Emily M. Williamson
ii
Acknowledgements

Five years ago, I made the smartest decision I could have ever made for myself by choosing
to pursue my Ph.D. at USC. Having the privilege to complete the work presented in this
dissertation was nothing short of an honor. For that I would like to thank the chemistry department
and my advisor, Professor Richard Brutchey, who provided me with both the opportunity to pursue
an avenue of research that truly excites me, and the freedom to grow and expand my knowledge
and skills in a way I never knew was possible. I would also like to thank the rest of my committee
members, Professor Mark Thompson, Professor Mike Inkpen, Professor Matthew Pratt, and
Professor Sarah Feakins. You have all been so generous with your time and have made the process
of achieving this degree such a smooth and pleasant one. My respect for you all is immeasurable
and it has been the best experience presenting my work to you and discussing science.
Richard – Thank you. You were the best advisor I could have ever asked for and more. In
terms of how I learn and how I am as a person, you understood me, motivated me, and pushed me
to achieve things I would have never achieved elsewhere or otherwise. I can’t express enough my
eternal gratitude for your guidance, wisdom, and compassion throughout the last five years. You
truly inspire me to be the best version of myself as a scientist and as a person. Thank you from the
bottom of my heart for giving me the tools to succeed and believing in me to do so, even when my
success was uncertain. Your support gave me the confidence to believe that I could do anything I
set my mind to. I will never be able to repay you for everything that you’ve done for me, but I will
consider you a key person in my existence and development for the rest of my life. Aside from
what you have done for me professionally, I cherish all of the laughs and fun times we’ve shared,
and I’ll miss our chats and conversations. Having you as an advisor made work enjoyable.
To the rest of the Brutchey group, past and present, I can’t thank you enough for your
support and companionship. I can’t imagine going through this experience with anyone else, and
I know I have made lifelong friends. I’d like to start with the pre-covid fab 5; you all made my
first three years an unbelievable joy. First and foremost, Lanja – When we met during our visitation
weekend as newly accepted students, I somehow immediately knew you were going to play a huge
part in my life. My commitment to USC was heavily influenced by knowing that you had also
committed to come here, and it made moving to Los Angeles alone without knowing a single soul
feel so much less terrifying, because I knew I would at least have one friend. Since then, I can truly
say that I was right about the feeling I had when we first met, as you have become one of my best
friends. Some of the best times in my life have been with you by my side, and you have remained
by my side in such an immense way throughout some of the worst things that have ever happened
to me. You have truly been my rock throughout this journey, and I don’t want to know where I
would be without you. I can no longer imagine my day-to-day life without our gym sessions, our
walks to get lunch, our dance parties in lab, our deep talks, our weekend adventures, our
uncontrollable laughter, and the overall general chaos that inevitably ensues when we’re together.
After spending essentially every waking moment together for the last five years, we’ve grown to
a point where I view our relationship in some ways as that of siblings, because no matter what
happens, or where we both end up, I know we’ll always have each other at the end of the day. I
love you so much. To Sara – Your impact throughout my first 3 years is something I will never
forget. You showed me the ropes and made me feel accepted as a member of this group so quickly,
for which I am so thankful. As my officemate, you made coming into lab everyday fun, and I truly
looked forward to our banter each day. As my friend, the fun we had during your time here and
after will hopefully continue for years to come. To Bryce – Your kindness, wit, and positivity were
iii
a bright light in this lab every single day. Your love for science was infectious and your unwavering
positive presence made this group feel whole. You, your work as a scientist, and how you carry
yourself through this life are an inspiration to me. Thank you for your help in my research and
thank you for being yourself. It was an honor to work with you and publish together. To Kris – As
my desk mate for four years, my Covid shift buddy, and essentially one of my roommates
throughout Covid, I remember one particular morning, you were driving me to campus on our
covid shift, and as we were mindlessly singing to a song in the car together, I had a realization that
you were my best friend at the time. In a time when I felt overwhelmingly isolated and alone, I
want you to know how much your company and friendship meant to me. Thank you for the years
of entertainment, insane conversation topics, and support as a co-worker.
Additionally, I would like to thank all the Brutchey group members that make up our
current group, as you have made the transition after covid so much easier. To Kyle – Your presence
in lab has been such a joy, and I’m so glad you joined the group. I know I can always count on you
for a laugh, and you have become a great friend. I will miss our long-winded science rants, and I
know you will achieve great things after me and Lanja are gone. To Marissa – it has been such a
pleasure to watch you build confidence and grow as a scientist. Your constant kindness has made
the tough days easier. To Allison – Your determination and vigor for science have been so fun to
watch, and your positive attitude has been an asset to this group. I am excited to see what your
future holds and everything you will achieve throughout the rest of your time here. To Zhaohong
– I couldn’t think of a better person to continue on the research path that I have been on for the
last few years. It has been an absolute pleasure to mentor you and show you the ropes as a scientist.
You are brilliant and I know you will go above and beyond in the coming years.
Furthermore, I would like to thank my friends and family for everything they have done
for me throughout this experience. To my dad, Derrick, I would be lost without you. You have
made me feel so loved, strong, and supported no matter what. To my mom, Barb, you have
sacrificed so much to get me to where I am today, and I owe so much to you. Thank you forever.
To my stepmom, Mary, my stepdad, Tracy, my sisters, Kait, Mia, Emma, and Dara, my brothers,
Dan and Beck, my bonus siblings, Olivia and Hunter, and my aunt, Karen, you have all been my
biggest cheerleaders these last few years, and I love you endlessly. I am beyond lucky to have such
an enormous support system who make me feel loved no matter where we all are on the globe. To
my friends I’ve made in California; my roommates over the years, Kathe, Allie, and Maddy, my
friends in the department, Shelby, JP, Joe, Michael, Adam, and so many more, and my friends
from outside of USC, Izzy, Katie, Austin, Caroline, Cole, Jacob, Maddie, Olivia, and everyone
else, you have all made California feel like home. I’m so thankful for the life I’ve built here with
all of you. To my long distance friends whom I still talk to everyday without fail, thank you for
everything. My best friends from college; Laura, Lily, Claudia, Emily, Jere, Holtz, and Ella, your
support has meant more to me than you know. You are some of the greatest friends in the world.
My best friends from home; Em Hoke, Wurster, Shanna, Peighton, Gabby, Jeslyn, Amanda, Lisa,
Maddy, Nelson, Derek, Brenden and all the rest, you are truly lifelong friends, and you have given
me strength beyond belief these last five years through thick and thin. I love you all forever.
Lastly, I’d like to dedicate this dissertation to the late Tyler Updegraff; my first love and
my partner for more than 7 years of my life, who remained a dear family friend until the end. Ty
– So much of me is because of you. I know everything I have achieved would not have been
possible without the confidence and assurance in myself that you instilled in me early on, or the
optimistic mindsets you taught me to have about life as I take on each day. I’ll never be able to
properly thank you, and I wish you were here to witness this milestone. We miss you.
iv
Table of Contents

Acknowledgements......................................................................................................................... ii
List of Tables................................................................................................................................ vii
List of Figures............................................................................................................................... xii
List of Schemes......................................................................................................................... xxiii
Abstract...................................................................................................................................... xxiv
Chapter 1. Design of Experiments for Nanocrystal Syntheses: A How-To Guide for
Proper Implementation
1.1. Abstract................................................................................................................... 1
1.2. Introduction............................................................................................................. 1
1.3. General Workflow of DoE...................................................................................... 5
1.4. Defining the Optimization Problem........................................................................ 7
1.5. Setting the Experimental Bounds............................................................................ 9
1.6. Screening Designs................................................................................................. 11
1.6.1. Factorial Design Anatomy............................................................. 12
1.6.2. Design Analysis and Interpretation............................................... 14
1.6.3. Design Improvement..................................................................... 18
1.7. Optimization Designs............................................................................................ 21
1.7.1. Design Characteristics................................................................... 21
1.7.2. Choosing an Optimization Design................................................. 23
1.7.3. Design Analysis and Interpretation............................................... 24
1.7.4. Multi-Response Optimizations...................................................... 27
1.7.5. Validating the Model..................................................................... 29
1.8. Conclusions........................................................................................................... 30
1.9. References............................................................................................................. 31

Chapter 2. Statistical Multiobjective Optimization of Thiospinel CoNi2S4
Nanocrystal Synthesis via Design of Experiments
2.1. Abstract................................................................................................................. 37
2.2. Introduction........................................................................................................... 38
2.3. Results and Discussion.......................................................................................... 41
2.3.1. General Synthesis and Defining the Reaction Parameter
Space............................................................................................. 41
2.3.2. Design of Experiments (DoE)........................................................ 45
2.3.3. Optimization via Response Surface Methodology (RSM)............. 54
2.4. Methods................................................................................................................. 71
2.4.1. Materials and General Considerations........................................... 71
2.4.2. Nanocrystal Synthesis................................................................... 71
2.4.3. Characterization............................................................................ 72
2.4.4. Design of Experiments (DoE)/Response Surface
v
Methodology (RSM)..................................................................... 73
2.5. Conclusions........................................................................................................... 78
2.6. References............................................................................................................. 80

Chapter 3. Throughput Optimization of Molybdenum Carbide Nanoparticle Catalysts
in a Continuous Flow Reactor Using Design of Experiments
3.1. Abstract................................................................................................................. 85
3.2. Introduction........................................................................................................... 86
3.3. Results and Discussion.......................................................................................... 89
3.3.1. Continuous Flow Millifluidic Synthesis and Reactor Setup.......... 89
3.3.2. Design of Experiments.................................................................. 93
3.3.3. Optimization via RSM: Maximizing Throughput........................ 100
3.4. Experimental Procedures/Methods...................................................................... 109
3.4.1. Continuous Flow Synthesis of MoC1–x Nanoparticles................. 109
3.4.2. Product Characterization............................................................. 110
3.4.3. Reactor Temperature Control...................................................... 111
3.4.4. Experimental Screening Design and Response Surface
Methodology............................................................................... 112
3.4.5. Catalytic Testing.......................................................................... 117
3.5. Conclusions......................................................................................................... 117
3.6. References........................................................................................................... 120

Chapter 4. Creating Ground Truth for Nanocrystal Morphology: A Fully Automated
Pipeline for Unbiased Transmission Electron Microscopy Analysis
4.1. Abstract............................................................................................................... 125
4.2. Introduction......................................................................................................... 126
4.3. Results and Discussion........................................................................................ 130
4.3.1. Machine Learning Pipeline.......................................................... 130
4.3.1.1. Image Processing................................................. 131
4.3.1.2. Creating Morphological Ground Truth................ 136
4.3.1.3. Unsupervised Clustering and Classification
into Shape Groups............................................... 142
4.3.2. Experimental Applications.......................................................... 145
4.3.2.1. Training Classification Algorithms..................... 145
4.3.2.2. Morphological Differentiation of CsPbBr3
Nanocrystals from Different Syntheses............... 149
4.3.2.3. Applicability to Morphologically Diverse
Ensembles........................................................... 166
4.4. Methods............................................................................................................... 170
4.4.1. Nanocrystal Synthesis................................................................. 170
4.4.1.1. Synthesis of CsPbBr3........................................... 170
4.4.1.2. Synthesis of Nickel Sulfides................................ 172
4.4.2. Characterization.......................................................................... 174
4.4.3. TEM Processing Pipeline............................................................ 174
4.4.3.1. Image Processing................................................. 174
4.4.3.2. Ground Truth Creation........................................ 177
vi
4.4.3.3. Unsupervised Clustering Optimization............... 182
4.4.3.4. Optimization of a Classification Algorithm......... 190
4.4.3.5. Classification Algorithms for Experimental
Applications........................................................ 192
4.5. Conclusions......................................................................................................... 195
4.6. Code.................................................................................................................... 197
4.7. References........................................................................................................... 197

Chapter 5. Predictive Synthesis of Copper Selenides through the Construction of a
Multidimensional Phase Map using a Data-Driven Classifier
5.1. Abstract............................................................................................................... 202
5.2. Introduction......................................................................................................... 203
5.3. Results and Discussion........................................................................................ 206
5.3.1. Construction of the Surrogate Model........................................... 207
5.3.2. Mapping the Cu-Se Phase Space via Classification.................... 212
5.3.3. Predictive Phase Determination of Klockmannite CuSe
via Classification Model.............................................................. 221
5.4. Conclusions......................................................................................................... 226
5.5. Experimental Procedures..................................................................................... 227
5.5.1. Materials and General Procedures............................................... 227
5.5.2. Synthesis of Copper(Oleate)2...................................................... 228
5.5.3. Synthesis of Copper Selenide...................................................... 228
5.5.4. Characterization.......................................................................... 229
5.6. References........................................................................................................... 229

Appendix A.
A.1. Coding Variable Values....................................................................................... 233
A.2. Screening Designs............................................................................................... 236
A.2.1. Full Factorial Designs.................................................................. 236
A.2.2. Blocked Designs.......................................................................... 238
A.2.3. Fractional Factorials and Design Resolution............................... 241
A.2.4. Design Augmentation.................................................................. 245
A.2.5. Mixture Designs.......................................................................... 247
A.2.6. Multi-Factor Categorical Designs............................................... 251
A.2.7. Continuous and Categorical Factors............................................ 252
A.2.8. Analyzing a Screening Output..................................................... 256
A.3. Optimization Designs.......................................................................................... 259
A.3.1. Example Optimization Designs................................................... 259
A.3.2. Analyzing an Optimization Output.............................................. 263
A.3.3. Multi-Response Optimizations.................................................... 267
A.4. References........................................................................................................... 271

vii
List of Tables

Table 1.1. List of colloidal nanocrystal syntheses in literature optimized via DoE................. 4

Table 1.2. Example table of reaction bounds and their coded counterparts............................ 11

Table 1.3. A 2
3
full factorial design for three variables and their contrast coefficients.......... 14

Table 2.1. Real and coded values of the reaction parameters for the factors under
investigation in the screening and optimization designs ....................................... 45

Table 2.2. The fractional factorial screening design, based on the guidelines outlined
in Chapter 1.......................................................................................................... 47

Table 2.3. Screening design and corresponding experimental responses for the five factors. 48

Table 2.4. Coded Doehlert optimization matrix for three variables........................................ 55

Table 2.5. Optimization design and corresponding responses................................................ 55

Table 2.6. Analysis of variance for the second-order optimization of size............................. 60

Table 2.7. Analysis of variance for the second-order optimization of polydispersity (%)...... 61

Table 2.8. Analysis of variance for the second order optimization of isolated yield (%)....... 63

Table 2.9. Desirability............................................................................................................ 78

Table 3.1. Coded low (-1), center (0), and high (+1) experimental values for the input
variables investigated in the full factorial screening design................................... 93

Table 3.2. Full Factorial matrix via coded values................................................................... 95

Table 3.3. Full factorial matrix of the corresponding real values............................................ 96

Table 3.4. Real values of the levels investigated for each significant variable in
theDoehlert optimization of maximizing throughput. As depicted from the
number of levels investigated for each variable, the variables most
significantly affecting throughput in decreasing order are concentration,
flow rate, and amount of oleylamine...................................................................... 99

Table 3.5. Coded Doehlert optimization matrix appended with points on each corner
of the design space................................................................................................. 99

Table 3.6. Optimization matrix of the corresponding real values......................................... 100
viii

Table 3.7. Analysis of variance for the screening of throughput.......................................... 113

Table 3.8. Analysis of variance for the screening of isolated yield...................................... 114

Table 3.9. Analysis of variance for the screening of residence time.................................... 115

Table 3.10. Analysis of variance for the optimization of throughput..................................... 116

Table 4.1. Average values of the predictors (defined in Image processing section) for
the seven shape groups of the non-agglomerated nanocrystals of Figure
4.7a...................................................................................................................... 135

Table 4.2. Standard deviations of the predictors (defined in Image processing section)
for the seven shape groups of the non-agglomerated nanocrystals of Figure
4.7a...................................................................................................................... 135

Table 4.3. Average values of the predictors (defined in Image processing section) for
the seven shape groups of the semi-agglomerated nanocrystals of Figure
4.7b...................................................................................................................... 135

Table 4.4. Standard deviations of the predictors (defined in Image processing section)
for the seven shape groups of the semi-agglomerated nanocrystals of Figure
4.7b...................................................................................................................... 135

Table 4.5. Average values of the predictors (defined in Image processing section) for
the seven shape groups of the heavily agglomerated nanocrystals of Figure
4.7c...................................................................................................................... 136

Table 4.6. Standard deviations of the predictors (defined in Image processing section)
for the seven shape groups of the heavily agglomerated nanocrystals of
Figure 4.7c.......................................................................................................... 136

Table 4.7. Parameter distributions used to generate the ground truth morphologies. All
three distributions are generated mutually independently, making this a true
3D parameter space. The limits of the distributions are chosen to produce
only physically plausible geometries (e.g., including the uniform
distribution for curvature down to 0.01 is meaningless since it is
indistinguishable from 0.05)................................................................................ 138

Table 4.8. Clustering optimization....................................................................................... 144

Table 4.9. Poor resolution clustering optimization............................................................... 144

Table 4.10. Average values of predictors (defined in Methods) for the four shape
ix
groups of CsPbBr3 nanocrystals synthesized via hot injection............................. 151

Table 4.11. Standard deviations of predictors (defined in Methods) for each shape
group of CsPbBr3 nanocrystals synthesized via hot injection.............................. 152

Table 4.12. Average values of predictors (defined in Methods) for the four shape
groups of CsPbBr3 nanocrystals synthesized in hot injection batch 1.................. 155

Table 4.13. Standard deviations of predictors (defined in Methods) for each shape
group of CsPbBr3 nanocrystals synthesized in hot-injection batch 1................... 155

Table 4.14. Average values of the predictors (defined in Methods) for the four shape
groups of CsPbBr3 nanocrystals synthesized in hot-injection batch 2.................. 156

Table 4.15. Standard deviations of the predictors (defined in Methods) for each shape
group of CsPbBr3 nanocrystals synthesized in hot-injection batch 2................... 156

Table 4.16. Batch-to-batch morphological comparison of CsPbBr3 nanocrystals
synthesized by the hot-injection method.............................................................. 157

Table 4.17. Average values of the predictors (defined in Methods) for the four shape
groups of CsPbBr3 nanocrystals synthesized via LARP...................................... 162

Table 4.18. Standard deviations of the predictors (defined in Methods) for each shape
group of CsPbBr3 nanocrystals synthesized via LARP........................................ 163

Table 4.19. Efficiency comparison of analysis methods......................................................... 165
Table 5.1. Polymorphs and crystallographic information of binary copper selenides........... 204

Table 5.2. The bounds of the binary Cu-Se phase space investigated for the surrogate
model................................................................................................................... 208

Table 5.3. Screening reactions and corresponding responses for the surrogate model.......... 209

Table 5.4. Full Doehlert optimization design and corresponding responses for the
surrogate model................................................................................................... 210

Table 5.5. Additional Copper Selenide experiments from previous studies......................... 210

Table A.1. Example reaction bounds for five factors............................................................. 233
Table A.2. Coded Doehlert optimization matrix for five factors........................................... 234

Table A.3. Real values of a Doehlert optimization matrix for five factors based on the
reaction bounds in Table A.1............................................................................... 235
x

Table A.4. Example 2-level design for three factors (2
3
factorial)......................................... 236

Table A.5. Example 2-level design for four factors (2
4
)........................................................ 237

Table A.6. Example 2-level factorial design for five factors (2
5
)........................................... 237

Table A.7. A 2
3
full factorial design for three variables and their contrast coefficients......... 238

Table A.8. Example 2IV
4-1
nodal fractional factorial design (½ fraction of the full
factorial in Table A.5)......................................................................................... 242

Table A.9. Example 2III
7-4
nodal fractional factorial design (
1
8
# fraction of the 128 runs
in the full 2
7
factorial).......................................................................................... 244

Table A.10. Alias structure of a 2III
7-4
fractional factorial design............................................. 245

Table A.11. Example 2III
5-2
fractional factorial design (¼ fraction of the full factorial in
Table A.5)........................................................................................................... 245

Table A.12. Example table of a multi-factor categorical experiment....................................... 251

Table A.13. Example table of the multi-factor categorical design matrix................................ 252

Table A.14. Example study of mixed continuous and categorical factors................................ 253

Table A.15. Example 3-factor factorial design with a categorical factor................................. 253

Table A.16. Optimized design matrix for a 3-factor study with categorical and
continuous factors................................................................................................ 255

Table A.17. Real and coded values of the reaction parameters for the factors under
investigation in the screening and optimization designs. Reproduced with
permission from Chapter 2. Copyright 2021 American Chemical Society..........256

Table A.18. Correlation matrix for a 2III
5-2
fractional factorial design..................................... 257

Table A.19. Estimated effects for a single response Y (nanocrystal size)................................ 257

Table A.20. Analysis of variance for a given response Y (nanocrystal size)............................ 258

Table A.21. Central composite design for three factors........................................................... 261

Table A.22. Three-level factorial design for three factors....................................................... 261

Table A.23. Box-Behnken design for three factors.................................................................. 262
xi

Table A.24. Doehlert matrix for three factors.......................................................................... 262

Table A.25. Draper-Lin design for four factors....................................................................... 262

Table A.26. Correlation matrix for estimated effects.............................................................. 263

Table A.27. Estimated effects for a single response Y (nanocrystal size)................................ 264

Table A.28. Analysis of variance for a single response, Y (nanocrystal size).......................... 264

Table A.29. Regression coefficients for nanocrystal size (Y).................................................. 266

Table A.30. Estimation results for nanocrystal size (Y).......................................................... 266

Table A.31. Predicted optimum of nanocrystal size (response value Y).................................. 267

Table A.32. Multi-response optimization design.................................................................... 267

Table A.33. ANOVA of the multi-response model equation................................................... 268

Table A.34. Model coefficients............................................................................................... 268

Table A.35. Alias matrix......................................................................................................... 269

Table A.36. Leverage of each run in the multi-response optimization design......................... 269

Table A.37. The observed and predicted response values of all three responses based on
the quadratic models from their individual optimizations.................................... 270

Table A.38. The observed desirability and the corresponding predicted desirability from
the desirability model for the multi-response optimization.................................. 271
xii
List of Figures

Figure 1.1. (a) A visual example of the OVAT method, and the information elucidated
within a two variable parameter space. (b) A visual example of the
information elucidated from a DoE optimization design throughout the
same two variable parameter space. Circles represent the experiments
performed within the parameter space. Stars represent the optima found by
each respective method............................................................................................ 3

Figure 1.2. (a) 2-level screening design for two variables. (b) 3-level response surface
optimization design for two variables...................................................................... 7

Figure 1.3. First-order screening analysis of three variables on a response Y: (a) Pareto
chart, (b) main effects plot, (c) normal probability plot of the linear
regression, and (d) interaction plot......................................................................... 17

Figure 1.4. Visual representations of 2-factor vs. 3-factor optimization designs in a
parameter space, with spheres representing reactions performed in the
design. (a) Three level factorial, (b) Box-Behnken, (c) central composite,
and (d) Doehlert designs........................................................................................ 23

Figure 1.5. (a) Example response surface of one response and two variables. (b)
Corresponding Pareto chart. (c) Quadratic main effects plot. (d) Quadratic
interaction effects plot........................................................................................... 26

Figure 1.6. Example contour plots for an optimization design with three factors:
Contour plots of the response Y while factor B is held constant at the (a)
low level (coded -1), (b) center (coded 0), and (c) high level (coded +1).
Contour plots of the response Y while factor A is held constant at the (d)
low level (coded -1), (e) center (coded 0), and (f) high level (coded +1).
Contour plots of the response Y while factor C is held constant at the (g)
low level (coded -1), (h) center (coded 0), and (i) high level (coded
+1)......................................................................................................................... 27

Figure 1.7. Desirability plot for a multi-response optimization of three responses using
a 3-factor Doehlert optimization design................................................................. 29

Figure 2.1. (a) Powder XRD pattern of unoptimized nanocrystals, with results from a
Rietveld refinement to the Fm3m (Co,Ni)9S8 structure. Tick marks represent
individual reflections of the (Co,Ni)9S8 structure with the difference pattern
shown below. 𝜆 = 1.5406 Å. (b) HR-TEM image of nanocrystals produced
by the unoptimized synthesis, showing the measured lattice fringes..................... 42

Figure 2.2. (a) Size histogram of the corresponding nanocrystals from the unoptimized
synthesis (N = 303). (b) TEM images of the resulting unoptimized
nanocrystals........................................................................................................... 42
xiii

Figure 2.3. (a) XRD analysis of the boundary reaction performed at 190 ˚C, showing a
mixture of binary and ternary phases present in the resultant nanocrystals.
(b) Phase evolution of the nanocrystals at short times (all other variables
held constant at the base level)............................................................................... 44

Figure 2.4. XRD analysis of the boundary reactions for (a) low temperature condition
(170 ˚C), (b) high temperature condition (190 ˚C), (c) short time condition
(1 h), (d) long time condition (5 h), (e) low Co:Ni precursor ratio (1:1),
(f) high Co:Ni precursor ratio (1:3), (g) low Co:DDT ratio (1:2), (h) high
Co:DDT (1:16), (i) low volume of oleylamine (4 mL), and (j) high volume
of oleylamine (10 mL)........................................................................................... 45

Figure 2.5. Statistical plots for the first order screening of nanocrystal size: (a) Pareto
chart, where the vertical blue line on each graph represents α = 0.05. (b)
main effects plot, (c) linear regression showing the normal probability, and
(d) interaction plot................................................................................................. 50

Figure 2.6. TEM images demonstrating the effect of the primary factor, Co:Ni
precursor ratio, on the size of the resulting nanocrystals........................................ 51

Figure 2.7. Statistical plots for the first-order screening of nanocrystal size distribution:
(a) Pareto chart, where the vertical blue line on each graph represents α =
0.05. (b) Main effects plot, (c) linear regression showing the normal
probability, and (d) interaction plot....................................................................... 52

Figure 2.8. Statistical plots for the first-order screening of nanocrystal isolated yield:
(a) Pareto chart, where the vertical blue line on each graph represents α =
0.05. (b) Main effects plot, (c) linear regression showing the normal
probability, and (d) interaction plot....................................................................... 52

Figure 2.9. (a) Prediction variance plot, illustrating the standard deviation of the
optimization model predictions throughout the entire parameter space. (b)
The design points of the Doehlert optimization matrix in the reaction
parameter space..................................................................................................... 54

Figure 2.10. Statistical plots for the second-order optimization of nanocrystal size: (a)
Pareto chart, (b) main effects plot, (c) linear regression showing the normal
probability, and (d) interaction plot....................................................................... 58

Figure 2.11. (a) Response surface for nanocrystal size as a function of Co:Ni precursor
ratio and Co:DDT ratio, with the minimum size predicted to be 5.2 nm. (b)
TEM image of the CoNi2S4 nanocrystals prepared using the reaction
parameters at the predicted minimum. (c) Corresponding size histogram of
the nanocrystals where s/𝑑
̅ = 18% (N = 320)......................................................... 58

xiv
Figure 2.12. (a) Response surface for the individual optimization of size distribution. (b)
TEM image of resulting nanocrystals. (c) Size distribution of resulting
nanocrystals where 𝜎/𝑑̅ = 11% (N = 338)............................................................... 61

Figure 2.13. Statistical plots for the second-order optimization of nanocrystal size
distribution: (a) Pareto chart, (b) main effects plot, (c) linear regression
showing the normal probability, and (d) interaction plot....................................... 62

Figure 2.14. Response surface for the individual optimization of isolated yield........................ 63

Figure 2.15. Statistical plots for the second-order optimization of nanocrystal isolated
yield: (a) Pareto chart, (b) main effects plot, (c) linear regression showing
the normal probability, and (d) interaction plot..................................................... 64

Figure 2.16. (a) Desirability plot of the simultaneous optimization of all three parameters
with the general optimum conditions indicated in white. (b) TEM image of
the CoNi2S4 nanocrystals synthesized under the optimum conditions. (c)
Corresponding size histogram of the resulting nanocrystals, where s/𝑑
̅ =
14% (N = 1,300). (d) Powder XRD pattern of CoNi2S4 nanocrystals
synthesized under the optimum conditions, with results from a Rietveld
refinement to the Fd3
,
m structure. Tick marks represent individual
reflections of the thiospinel structure with the difference pattern shown
below. l = 1.5406 Å............................................................................................... 66

Figure 2.17. (a) XRD pattern of the CoNi2S4 nanocrystals produced under the optimum
conditions predicted by RSM and (b) HR-TEM image of the corresponding
particles, showing the measured lattice fringes...................................................... 67

Figure 2.18. (a) TEM images CoNi2S4 nanocrystals synthesized under the conditions
indicated at the extrapolated optimum. (b) Size histogram of the resultant
nanocrystals with 𝜎/𝑑
̅ = 15% (N = 639)................................................................ 69

Figure 2.19. (a) Stacked XRD patterns of the CoNi2S4 nanocrystals produced under the
optimum and extrapolated optimum conditions predicted by RSM, in
addition to the single optima conditions of minimized size and
polydispersity. Thesediffraction patterns match the expected CoNi2S4 phase
and are shifted from the (Co,Ni)9S8 nanocrystals that resulted from the
unoptimized reaction. (b,c) HR-TEM images of the nanocrystals resulting
from the extrapolated optimum reaction conditions, showing the measured
lattice fringes. Twinning is observed in the HR-TEM .......................................... 69

Figure 3.1. Schematic of the millifluidic reactor system for the continuous flow
synthesis of MoC1−x nanoparticles. A syringe pump with a heated syringe
(80 °C) is used to drive a precursor mixture into a reactor coil housed in a
furnace. The colors are representative of those empirically observed in the
reaction, with yellow (at reactor inlet) indicating the unreacted Mo(CO)6
xv
precursor solution and black indicating the MoC1−x nanoparticle suspension
isolated into plugs because of in situ gas evolution. An in-line thermal
dissipation-type flow sensor at the reactor outlet reports the relative volume
of liquid and gas—an idealized flow sensor signal is shown with
corresponding liquid (detected by the sensor) and gas (not detected) plugs.
The product is collected in vials that can be isolated with valves to allow for
removal of partial product volumes during runs. The system pressure is
maintained by a fluidic controller (Fluigent), which is locked on a fixed
pressure of 20 psi to minimize in situ gas evolution.............................................. 91

Figure 3.2. Exemplary data for the sensor readout during product collection with a
pump flow rate set at 𝑄
L

= 40 mL h
-1
The data shown here correspond to a
2-min time frame for experiment #1 in the full factorial matrix (Table 3.3).
Data are sampled at 10 Hz. Based on the count of data points above and
below the threshold throughout the experiment (14 min), the gas to liquid
volumetric ratio was found to be 3.74, which translates to 𝑄
G

=150 mL h
-1
.

Therefore, the residence time is calculated is 𝜏 = 𝑉/(𝑄
L

+ 𝑄
G

) = 11.7
Min........................................................................................................................ 92

Figure 3.3. (a) Powder XRD pattern and (b) TEM image of the MoC1–x nanoparticles
synthesized under the base reaction conditions...................................................... 93

Figure 3.4. Standardized Pareto charts for (a) residence time, (b) isolated yield, and (c)
throughput. The variables on the y-axis of the Pareto charts are defined as
A = flow rate (mL h
−1
), B = temperature (°C), C = amount of oleylamine
(vol %), and D = precursor concentration (mM). The vertical line in each
Pareto chart corresponds to α = 5%, and (+) and (−) correspond to an
increase or decrease in the response, respectively, for the high value of a
given factor. Main effects plots for (d) residence time, (e) isolated yield, and
(f) throughput, where (+) and (−) correspond to the high and low levels for
each factor, respectively........................................................................................ 97

Figure 3.5. Calculated response surface function demonstrating the reaction conditions
(precursor concentration, flow rate, and amount of oleylamine) that
correspond to a specific throughput, illustrated by the color legend. The
conditions for maximum throughput are indicated by a star. The bounds of
this parameter space are specific to this flow reactor and synthetic system;
any points outside of the parameter space are not feasible for this system........... 102

Figure 3.6. (a-c) Contour plots of MoC1–x nanoparticle throughput with flow rate held
constant at the (a) low level (coded -1), (b) center (coded 0), and (c) high
level (coded +1). (d-f) Contour plots of throughput with amount of
oleylamine held constant at the (d) low level (coded -1), (e) center (coded
0), and (f) high level (coded +1). (g-i) Contour plots of throughput with
Mo(CO)6 precursor concentration held constant at the (g) low level (coded
-1), (h) center (coded 0), and (i) high level (coded +1)......................................... 103
xvi

Figure 3.7. Prediction variance plot of the design space. Concentration is set at the base
value (coded 0) for visual simplicity.................................................................... 104

Figure 3.8. Visual representation of the design points for the optimization,
corresponding to the runs displayed in Table 3.5................................................. 104

Figure 3.9. (a) Powder XRD pattern and (b) TEM image of the MoC1−x nanoparticles
produced under optimized conditions, and (c) powder XRD pattern and (d)
TEM image of the MoC1−x nanoparticles produced under unoptimized
conditions............................................................................................................ 106

Figure 3.10. (a) HRTEM and (b) SAED pattern of MoC1–x nanoparticles synthesized
under optimized conditions. (c) HRTEM and (d) SAED pattern of MoC1–x
nanoparticles synthesized under batch conditions. The lattice fringes
correspond to the (111) and (200) planes of the 𝛼-phase...................................... 107

Figure 3.11. FT-IR spectra of Mo(CO)6 precursor and the supernatant of the first wash
of the MoC1–x nanoparticles (in the workup procedure). The spectra show
the representative 𝜂(CO) stretching region, demonstrating that there is no
unreacted Mo(CO)6 precursor in the supernatant resulting from the product
stream.................................................................................................................. 107

Figure 3.12. (a) CO2 conversion as a function of TOS and (b) product selectivity taken
as an average of data from 16−20 h. Reaction conditions were 300 °C, 2
MPa, WHSV based on Mo content of 40 h
−1
and H2:CO2 molar ratio in the
feed of 2.7............................................................................................................ 109

Figure 4.1. General workflow of the TEM image preprocessing method. (a) Original
bright field TEM image of CsPbBr3 nanocrystals. (b) Binary image
segmentation after contrast adjustment, filtering, and processing. (c) “Water
wells” of pixel intensities in each connected component to identify and
separate agglomerated nanocrystals. (d) Individual nanocrystal
segmentations plotted as colors overlaid onto the original TEM image, with
their respective bounding boxes used for cropping and implementation into
the neural network. Nanocrystals connected to the edges and joined with the
scale bar are removed........................................................................................... 132

Figure 4.2. (a) Non-overlapping nanocrystals and their morphology classifications. (b)
Semi-overlapping nanocrystals and their corresponding morphology
classifications, with areas of significant overlap circled in green. (c) Heavily
overlapping nanocrystals and their corresponding morphology
classifications, with areas of significant overlap circled in green......................... 134

Figure 4.3. Overview of outputs from model to generate simulated nanocrystal
morphologies to create a ground truth. (a) Demonstration of how varying
xvii
individual parameters in the model while keeping the others constant affects
the generated nanocrystal shape: Upper left, p-norm; upper right, Lratio;
bottom, number of major axes d (size adjusted for clarity). (b) Sampling of
simulated nanocrystal TEM images. The labels for each image are (p, d,
Lratio).................................................................................................................... 138

Figure 4.4. A sampling of abnormal and asymmetric simulated morphologies.................... 139

Figure 4.5. Diagnostic tools for assessing the quality of the neural network training.
(a) Learning curve plotting the root mean square error (RMSE) for
individual nanocrystal images as a function of the number of images on
which the neural network was trained. (b) Q-Q plot for the number of major
axes (d). (c) RMSE-parameter plot for the curvature (p)...................................... 141

Figure 4.6. Scatter plots of the total nanocrystal populations with shape group
classifications indicated by color for (a) the hot-injection synthesis and (b)
the LARP synthesis of CsPbBr3. Groups labels include re-simulation of the
average predicted CNN parameters compared to the experimental
counterpart that generated the prediction and average size of each group.
Average calculated Jaccard coefficient was 0.8 across every group.................... 142

Figure 4.7. Confusion matrix of the predicted class vs. the true class from the
classification model trained on 10 feature variables and three CNN
parameters for (a) the large data set, consisting of 163 TEM images
(corresponding to full_Mdl, test accuracy = 0.9951) and (b) the small data
set, consisting of 52 TEM images (corresponding to full_Mdl2, test
accuracy = 0.9907).............................................................................................. 146

Figure 4.8. Confusion matrix of the predicted class vs. the true class (a) from the
classification model ‘feat_Mdl’, test accuracy = 0.7474, trained on the large
data set of 163 TEM images, using only the 10 morphology features, (b)
from the classification model corresponding to ‘feat_Mdl2’, test accuracy =
0.7401, trained on the large data set of 163 TEM images, using only four of
the 10 morphology features, and (c) from the classification model
corresponding to ‘feat_mdl3’, test accuracy = 0.5571, trained on the small
data set of 52 TEM images, using only the 10 morphology features.................... 148

Figure 4.9. Confusion matrix of the predicted class vs. the true class (a) from the
classification model corresponding to ‘part_Mdl’, test accuracy = 0.9918,
trained on the large data set of 163 TEM images, using the original training
variables but with three random feature variables removed. (b) From the
classification model corresponding to ‘part_Mdl2’, test accuracy = 0.9949,
trained on the large data set of 163 TEM images, using the original training
variables but with six random feature variables removed. (c) From the
classification model corresponding to ‘part_Mdl3’, test accuracy = 0.9899,
trained on the small data set of 52 TEM images, using the original training
xviii
variables but with three random feature variables removed................................. 148

Figure 4.10. (a) Classification of the CsPbBr3 nanocrystals resulting from the hot-
injection synthesis differentiated by color and overlaid onto original TEM
images. Nanocrystals connected to edges are removed. (b) Overlaid shape
groupings (left) compared to the original bright field TEM image (right).
(c) A subset of 210 nanocrystals from each of the four shape groups. (d)
Three exemplary histograms and corresponding normal distributions of
feature descriptors (clusters are color coded)....................................................... 150

Figure 4.11. Unsupervised clustering of the observed CsPbBr3 nanocrystals made via
hot-injection synthesis. (a) Dendrogram of the nanocrystal similarities. (b)
Parallel coordinate plots of each observed nanocrystal with group
assignment coded by color. (c) Pareto chart of the six principal components.
(d) Scatter plots of each observed nanocrystal with the group assignment
coded by color in the 3D parameter space of the top three principal
components. ccc = 0.9432.................................................................................... 151

Figure 4.12. Histograms and their corresponding normal distributions of the input
variables used in shape classification for the CsPbBr3 nanocrystals from the
hot-injection synthesis, with shape groups indicated by color............................. 152

Figure 4.13. Total classified CsPbBr3 nanocrystals from the hot-injection synthesis with
shape groups indicated by color, showing the relative make-up of each
group................................................................................................................... 153

Figure 4.14. Confusion matrix of the predicted class vs. the true class from the
classification model trained on 7,082 CsPbBr3 nanocrystals (5,666 training,
1,416 testing) from 20 TEM images of the hot-injection synthetic product.
Test accuracy = 0.9958........................................................................................ 154

Figure 4.15. Total classified CsPbBr3 nanocrystals from batch 1 of the hot-injection
synthesis showing the relative make-up of each group, with shape groups
indicated by color along with 210 example nanocrystals from each shape
group................................................................................................................... 155

Figure 4.16. Total classified CsPbBr3 nanocrystals from batch 2 of the hot-injection
synthesis showing the relative make-up of each group, with shape groups
indicated by color along with 210 example nanocrystals from each shape
group................................................................................................................... 156

Figure 4.17. (a) Classification of the CsPbBr3 nanocrystals resulting from the LARP
synthesis differentiated by color and overlaid onto original bright field TEM
images. Nanocrystals connected to edges have been removed. (b) Overlaid
shape groupings (left) compared to the original TEM image (right). (c) A
xix
subset of ≤ 210 nanocrystals from each of the six shape groups (d) Three
exemplary histograms and corresponding normal distributions of feature
descriptors (clusters are color coded)................................................................... 158

Figure 4.18. Unsupervised clustering of the observed CsPbBr3 nanocrystals from the
LARP synthesis. (a) Dendrogram of the nanocrystal similarities. (b) Parallel
coordinate plots of each observed nanocrystal with group assignment coded
by color. (c) Pareto chart of the six principal components. (d) Scatter plots
of each observed nanocrystal with the group assignment coded by color in
the 3D parameter space of the top three principal components. ccc =
0.4586.................................................................................................................. 159

Figure 4.19. Total CsPbBr3 nanocrystals classified from the LARP synthesis, showing
the relative percentages of nanocrystals in each group......................................... 160

Figure 4.20. Confusion matrix of the predicted class vs. the true class from the
classification model trained on 6,427 CsPbBr3 nanocrystals from the 31
TEM images of the LARP synthesis (5,124 training, 1,285 testing). Test
accuracy = 0.9930................................................................................................ 160

Figure 4.21. Histograms and their corresponding normal distributions of the input
variables used in shape classification of the CsPbBr3 nanocrystals made via
LARP, with shape groups indicated by color....................................................... 162

Figure 4.22. Output of the Autodetect-mNP pipeline when analyzing CsPbBr3
nanocrystals from the hot-injection synthesis. (a) Total nanocrystals
detected (blue) vs. total nanocrystals that passed the filtering technique used
(white). (b) Total classified nanocrystals with shape group indicated by
color, showing the relative make-up of each group. (c) Histograms of the
input variables for shape classification with shape groups indicated by
color..................................................................................................................... 166

Figure 4.23. Output of the Autodetect-mNP pipeline when analyzing CsPbBr3
nanocrystals from the LARP synthesis. (a) Total nanocrystals detected
(blue) vs. total nanocrystals that passed the filtering technique used (white).
(b) Total classified nanocrystals with shape group indicated by color,
showing the relative make-up of each group. (c) Histograms of the input
variables for shape classification with shape groups indicated by color............... 166

Figure 4.24. Classification of a variety of nickel sulfide nanocrystals, illustrating the
ability of the pipeline to detect and quantify more morphologically diverse
samples. Nanocrystals connected to edges have been removed. (a) Irregular,
concave morphologies of Ni3S2. (b) Morphological diversity observed in
phase pure Ni3S4. (c) Temporal reaction trajectory of morphology changes
in Ni9S8. (d) Magnification differences in TEM images of NiS. (1-10)
Colored segregation of ≤ 210 representative nickel sulfide nanocrystals into
xx
the 10 distinct shape groups................................................................................. 168

Figure 4.25. Unsupervised clustering of the nickel sulfide nanocrystals. (a) Dendrogram
of the nanocrystal similarities. (b) Parallel coordinate plots of each nickel
sulfide nanocrystal with group assignment coded by color. (c) Pareto chart
of the six principal components. (d) Scatter plots of each nickel sulfide
nanocrystal with the group assignment coded by color in the 3D parameter
space of the top three principal components. ccc = 0.9369................................... 169

Figure 4.26. Histograms and their corresponding normal distributions of the input
variables used in shape classification for the nickel sulfide nanocrystals with
shape groups indicated by color........................................................................... 170

Figure 4.27. Diagnostic tools for alternate neural network constructions. (a) Fully
Connected (FC) network. Left: training curve showing underfitting
(insignificant learning), even at learning rates increased by multiple orders
of magnitude. Right: Q-Q plot for d, showing significant bias in the
prediction and an inability to distinguish between images of all number of
major axes. (b) Residual Neural Network (RNN). Left: training curve
showing significant overfitting as the validation RMSE no longer improves
past half the training data, while the training RMSE continues to decrease.
Right: Q-Q plot for d, showing the adverse effects of overfitting when
evaluating the test set........................................................................................... 180

Figure 4.28. Dendrograms of (a) the features only data set, (b) the CNN parameters only
data set, and (c) both data sets combined.............................................................. 186

Figure 4.29. Parallel coordinate plots of each observed nanocrystal with group
assignment coded by color for (a) K-means, (b) GMM, and (c) Linkage Tree
clustering for the features only data set; (d) K-means, (e) GMM, and (f)
Linkage Tree clustering for the CNN parameters only data set; and (g) K-
means, (h) GMM, and (i) Linkage Tree clustering for the combined data set...... 186

Figure 4.30. Scatter plots of each observed nanocrystal with the group assignment coded
by color in the 3D parameter space of (a) the features only data set, (b) the
CNN parameters only data set, and (c) the combined data set.............................. 187

Figure 4.31. Pareto charts for (a) the features only data set, (b) the CNN parameters only
data set, and (c) the combined data set................................................................. 187

Figure 4.32. Dendrograms of (a) the features only data set, (b) the CNN parameters only
data set, and (c) both data sets combined.............................................................. 188

Figure 4.33. Parallel coordinate plots of each observed nanocrystal with group
assignment coded by color for (a) K-means, (b) GMM, and (c) Linkage Tree
xxi
clustering for the features only data set; (d) K-means, (e) GMM, and (f)
Linkage Tree clustering for the CNN parameters only data set; and (g) K-
means, (h) GMM, and (i) Linkage Tree clustering for the combined data set...... 188

Figure 4.34. Scatter plots of each observed nanocrystal, with the group assignment coded
by color in the 3D parameter space of (a) the features only data set, (b) the
CNN parameters only data set, and (c) the combined data set.............................. 189

Figure 4.35. Pareto charts for (a) the features only data set, (b) the CNN parameters only
data set, and (c) the combined data set................................................................. 189

Figure 4.36. Bayesian optimization of classification algorithm and hyperparameters........... 191

Figure 5.1. (a) Coded and color identifiers for each of the 11 unique phase combinations
of copper selenide observed during construction of the surrogate model. (b)
Powder XRD patterns of four resulting phase pure copper selenides:
berzelianite Cu2–xSe (A), umangite Cu3Se2 (E), wurtzite-like Cu2–xSe (G),
and weissite-like Cu2–xSe (I)................................................................................ 211

Figure 5.2. Confusion chart of the classification model predictions, showing correct
predictions in blue and incorrect predictions in orange........................................ 213

Figure 5.3. Relative importance scores of the experimental variables for (a) the entire
surrogate model and separated by (b) Ph2Se2 and (c) Bn2Se2 precursors.............. 214

Figure 5.4. Each reaction in the surrogate model with their respective coded variable
values for (a) diphenyl diselenide precursor and (b) dibenzyl diselenide
precursor. The resultant phase combination for each reaction is indicated by
color..................................................................................................................... 215

Figure 5.5. Visualization of the Cu–Se phase maps for the (a,b) Ph2Se2 and (c,d,)
Bn2Se2 precursors. The data points in (a) and (c) represent experiments ran
in the experimental space for each respective precursor and are color coded
to the phase outcome shown in the legend............................................................ 216

Figure 5.6. (a) Powder XRD patterns of phase combinations that result in the (b) sub-
region of the phase map that is bound by the area of lower temperatures and
higher volumetric ratios of oleylamine with the Ph2Se2 precursor....................... 218

Figure 5.7. Simplified decision tree for the Cu–Se phase map predicted by the
classification algorithm, with Bn2Se2 precursor pathways to phase pure
products indicated by orange arrows and Ph2Se2 precursor pathways to
phase pure products indicated by blue arrows...................................................... 219

Figure 5.8. Classification tree/phase map for the diphenyl dieselenide precursor.................. 220

xxii
Figure 5.9. Classification tree/phase map for the dibenzyl dieselenide precursor.................. 221

Figure 5.10. (a) Cu–Se phase map for the Bn2Se2 precursor, with the sub-region of
interest circled in black. The initial experimental conditions are identified
by a black circle (C) and the target conditions for synthesizing phase pure
klockmannite CuSe are indicated by a star (M). (b) Le Bail refinement of
the XRD pattern collected on the mixture of klockmannite and umangite
Cu3Se2 (C) before phase targeting. (c) Le Bail refinement of the XRD
pattern collected on the phase pure klockmannite CuSe (M) after phase
targeting. (d-f) Response surfaces giving the predicted relative phase purity
of klockmannite CuSe throughout the experimental space, with the initial
experimental conditions and target conditions shown by the black circle (C)
and star (M)......................................................................................................... 222

Figure 5.11. Initial temperature screening of klockmannite phase synthesized via Bn2Se2
over time at (a) 205 ˚C and (b) 215 ˚C and (c) 225 ˚C. Klockmannite stick
pattern is indicated in purple................................................................................ 224

Figure 5.12. Second iteration of temperature screening of klockmannite phase
synthesized via Bn2Se2 over time at (a) 220 ˚C and (b) 230 ˚C.
Klockmannite stick pattern is indicated in purple................................................ 224

Figure 5.13. Aliquot study of klockmannite phase synthesized via Bn2Se2 over time at
223.5 ˚C. Klockmannite stick pattern is indicated in purple................................. 226

Figure A.1. Excel spreadsheet to convert the coded values in Table A.2 into the real
values given in Table A.3.................................................................................... 236

Figure A.2. The design for a simplex lattice cubic or special cubic design, where P =
primary blends, T = tertiary blends, C = the centroid, and the triangle
represents the simplex......................................................................................... 250

Figure A.3. The design for a simplex-centroid where P = primary blends, B = binary
blends, C = the centroid, and the triangle represents the simplex......................... 250
xxiii
List of Schemes

Scheme 1.1. Flow chart of the general sequence of events in DoE to optimize a
nanocrystal synthesis, exemplifying an eight-experiment screening of three
factors (to unconfound main effects and binary interaction effects) and a
subsequent optimization.......................................................................................... 5

Scheme 4.1. Visualization of full pipeline for TEM image analysis......................................... 130

Scheme 4.2. Step-by-step workflow of the image processing.................................................. 177

xxiv
Abstract
Every example of modern technology that is enabled by materials is also limited by
materials; for example, the physicochemical properties of current materials limit the conversion of
sunlight into electricity or fuel, or limit how we generate artificial light. The search for new
materials is driven by the need to improve existing technologies in addition to the discovery of
novel functionality to realize next-generation technologies. Engineered colloidal nanocrystals are
of interest because of their unique size- and shape-dependent chemical and physical properties,
which offer routes to the development of champion materials for applications including catalysis,
plasmonics, photovoltaics, optoelectronics, and thermoelectrics. Control over colloidal
nanocrystal syntheses is essential for materials discovery and the optimization of desired
properties, and therefore plays a key role in the applications of these materials. Despite this, a
significant bottleneck exists in the Edisonian nature of materials synthesis. Traditional one-
variable-at-a-time methods for synthetic optimizations are inefficient, providing one-dimensional
insight into a complex, multidimensional experimental domain, which wastes precious resources
in the process. Moreover, applying the lessons learned from one system to another is exceptionally
challenging because of the disparate nature of material structures and compositions. This
dissertation will address the inefficiencies in colloidal nanocrystal synthesis and development by
utilizing data-driven learning. These multivariate techniques provide improved analytical tools and
robust predictive frameworks that can map reaction coordinates from precursors to the final
crystalline solid in a minimal number of experiments. By constructing a fuller picture of
nanocrystal syntheses, precise control over process-structure-property relationships can be
obtained, better facilitating material discovery and optimization.
xxv
In Chapter 1, a how-to guide is presented for the use of design of experiments (DoE) and
response surface methodology (RSM) in the context of nanocrystal synthesis and optimization.
The theory behind the multivariate statistical analysis techniques is explained, and step-by-step
instructions are described for their proper implementation in experimental research. These
guidelines will be followed, and the utility of these techniques will be directly illustrated in the
chapters that follow. Examples of experimental designs for others to work through and utilize that
correspond to the lessons in this chapter are described and illustrated in Appendix A.
In Chapter 2, a statistical design of experiments (DoE) approach to optimize the
synthesis of novel small, well-defined CoNi2S4 nanocrystals is reported, allowing for control over
the responses of nanocrystal size, size distribution, and isolated yield. After implementing a 2
5−2
fractional factorial design, the statistical screening of five different experimental variables
identified temperature, Co:Ni precursor ratio, Co:thiol ratio, and their higher-order interactions as
the most critical factors in influencing the aforementioned responses. Second-order design with a
Doehlert matrix yielded polynomial functions used to predict the reaction parameters needed to
individually optimize all three responses. A multiobjective optimization to simultaneously
optimization size, size distribution, and isolated yield, predicted the synthetic conditions needed
to achieve a minimum nanocrystal size of 6.1 nm, a minimum polydispersity (σ/d ̅ ) of 10%, and a
maximum isolated yield of 99%, with a desirability of 96%. The resulting model was
experimentally verified by performing reactions under the specified conditions. This work
illustrates the advantage of multivariate experimental design as a powerful tool for accelerating
control and optimization in nanocrystal syntheses.
In Chapter 3, statistical DoE in tandem with response surface methodology is
utilized for a parametric screening analysis to optimize the throughput of a MoC1−x nanoparticle
xxvi
synthesis utilizing a millifluidic flow reactor. A full factorial design was implemented to evaluate
four input variables (reaction temperature, flow rate, solvent fraction of oleylamine, and precursor
concentration) that carry statistically significant effects on three responses (throughput, residence
time, and isolated yield). A Doehlert matrix was implemented to investigate each significant
variable at a higher number of levels to optimize throughput. Our results give a nonintuitive set of
experimental conditions that resulted in an optimized throughput of 2.2 g h
−1
. This translates to a
50-fold increase in throughput compared to the previously reported batch method. The catalytic
performance of the MoC1−x nanoparticles produced under optimized throughput was demonstrated
in the CO2 hydrogenation reaction. This DoE screening analysis and throughput optimization of
MoC1−x synthesis open the door to an increased feasibility for scale-up.
In Chapter 4, the possibilities of data-driven learning are expanded upon in a new direction,
and synthetic image rendering is introduced to solve the ground truth problem of nanocrystal
morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-
dimensional parameter space, a convolutional neural network was trained to link unique
morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather
than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies
nanocrystal morphology from experimental transmission electron microscopy (TEM) images.
Using improved computer vision techniques, 42,650 nanocrystals were identified, assessed, and
labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-
practice manual measurements. A classification algorithm was trained with a prediction accuracy
of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal
shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal
CsPbBr3 nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on
xxvii
morphology. This pipeline provides a simple, efficient, and unbiased method to quantify
nanocrystal morphology and represents a practical route to construct large datasets with an
absolute ground truth for training unbiased morphology-based machine learning algorithms.
In Chapter 5, a demonstration of how data-driven learning can successfully map the
complex phase space of binary copper selenides in a minimal number of experiments is reported.
Combining chimie douce synthetic methods with multivariate analyses via classification
techniques enables predictive phase determination. A surrogate model was constructed with
experimental data derived from a design matrix for four experimental variables: diselenide
precursor C–Se bond strength, time, temperature, and solvent composition. The reactions in the
surrogate model resulted in 11 distinct phase combinations of copper selenide. This data was used
to train a classification model that predicts phase with 95.7% accuracy. Analysis of the resultant
decision tree enabled detailed conclusions to be drawn about how the experimental variables affect
phase and provided prescriptive synthetic conditions for phase isolation. This guided the
accelerated phase targeting of klockmannite CuSe, which could not be isolated as a phase pure
material in the reactions used to construct the surrogate model. The reaction conditions the model
predicted to synthesize klockmannite CuSe were experimentally validated, highlighting the utility
of this approach.

1
Chapter 1. Design of Experiments for Nanocrystal Syntheses: A How-To Guide for Proper
Implementation*

*Published in Chem. Mater. 2022, 34, 9823-9835.

1.1. Abstract
The understanding and control of colloidal nanocrystal syntheses is essential for discovery
and optimization of desired properties, and therefore plays a key role in the applications of these
materials. Typical one variable at a time (OVAT) methods limit the ability of researchers to
achieve such goals by providing one-dimensional insight into a complex, multidimensional
experimental domain, wasting precious resources in the process. Design of experiments (DoE) in
conjunction with response surface methodology (RSM) offers an accelerated route for multivariate
investigation and optimization of nanocrystal syntheses. The method enables systematic analysis
and multi-dimensional modeling of the independent and dependent effects that any number of
factors have on chosen responses, resulting in easy optimization of a large synthetic space in a
fraction of the experiments. Herein, we will outline the general steps to follow when utilizing DoE
and RSM for screening and optimization of nanocrystal syntheses, as well as the background
needed to appropriately design an investigation and understand the results.

1.2. Introduction
Engineered colloidal nanocrystals have become promising materials in photovoltaics,
1,2

thermoelectrics,
3,4
catalysis,
5,6
and biological imaging
7–9
as a result of their many desirable
physical and chemical properties. The controlled synthesis of nanocrystals is required to unlock
the true potential of these materials, since their performance is highly dependent on size,
10,11

polydispersity,
12
morphology,
13
and crystal structure.
14
For example, size, size distribution, and
2
crystal structure directly affect the properties of nanocrystal-based solar cells
15
and light emitting
diodes (LEDs)
16
via tuning the optical band gap.
17
Similarly, a mechanistic understanding of
synthetic control leads to the ability to discovery new nanocrystal materials via kinetic trapping of
metastable phases.
14
However, the thermodynamics and kinetics of nanocrystal formation can be
affected by a variety of factors, including reaction time, temperature and heating rate, solvent
combinations, precursor and ligand types and their ratios, concentrations, orders of addition,
addition rates, etc., as well as the potential binary and ternary interactions between all these
synthetic variables.
18
This results in incredibly complex synthetic parameter spaces, making it
difficult, if not impossible, to understand, predict, and control the properties of synthesized
nanocrystals using chemical intuition alone.
19,20

Most researchers still rely on the one-variable-at-a-time (OVAT) method for synthetic
optimizations, where one variable of interest is systematically changed while all other variables
are held constant. Time and resource costs are considerable with this approach, as it navigates the
synthetic space in an oversimplified, one-dimensional nature. For example, investigating the
effects of two variables A and B (factors) on a particular nanocrystal property Y (response) creates
a 2-dimensional parameter space, defined in Figure 1.1 (where three factors would create a 3-
dimensional parameter space). If one variable is explored while keeping the other constant, the
area of the parameter space covered by OVAT is one-dimensional and may not accurately identify
the true optimum (Figure 1.1a). Additionally, OVAT cannot determine nor quantify the effect of
possible interactions between two or more experimental variables on a given nanocrystal
characteristic, even if there is a qualitative understanding that two variables interact in the synthesis
(e.g., temperature and time in nanocrystal size and ripening).
21,22
Consequentially, the effect of a
factor on a given response Y observed via OVAT method represents a limited region of a much
3
wider experimental domain, and the relationship may not hold true throughout the rest of the
parameter space because of unresolved interactions between the factor and other variables. Such a
disadvantage increases with an increasing number of variables.

Figure 1.1. (a) A visual example of the OVAT method, and the information elucidated within a
two variable parameter space. (b) A visual example of the information elucidated from a DoE
optimization design throughout the same two variable parameter space. Circles represent the
experiments performed within the parameter space. Stars represent the optima found by each
respective method.

This illustrates the need to more fully map the experimental space to achieve efficient and
accurate synthetic control in colloidal nanocrystal syntheses. One effective way to achieve
synthetic control of both dependent and independent variables is via design of experiments (DoE),
which has been an important technique in industrial optimizations since the mid-20
th
century, but
has only more recently begun to develop as a tool in academia.
21,23
Combining DoE with an
optimization technique called response surface methodology (RSM — a predictive modeling
technique that uses regression to explore the relationships between some number of experimental
variables and a response variable) provides insight throughout a high-dimensional parameter space
to formulate a predictive model using a minimal number of experiments.
24
Using the steps that
will be outlined in this chapter, researchers can create models of such parameter spaces that can
be used to predict a response, or multiple responses, at any point within that space. This results in
4
a “bird’s eye view” of the synthetic parameter space, which renders the technique more accurate
and efficient compared to the information acquired through OVAT experimentation. As illustrated
in Figure 1.1b, DoE/RSM accurately identifies two local optima in five experiments, whereas
OVAT misidentifies one global optimum with very low precision in thirteen experiments.
Reinhardt and co-workers explicitly demonstrate superiority of this technique in a study to identify
conditions for perovskite formation.
25
Table 1.1 provides a list of publications where DoE has
been applied to nanocrystal synthesis. Various design methods can be successfully employed for
several response types, proving the feasibility of using DoE in nanocrystal syntheses. Notably,
targeting both direct features, such as nanocrystal size, as well as dependent factors, such as
photoluminescence quantum yield (PLQY), are equally feasible using DoE. In the latter case, DoE
can be used to maximize PLQY; however, the method will not provide insight on the origin of the
increase in PLQY without further characterization of the resulting nanocrystals.

Table 1.1. List of colloidal nanocrystal syntheses in literature optimized via DoE.
Material Target Response Screening Design Optimization Design Software Ref.
CoNi 2S 4 Size, size distribution, yield 2
5-3
Fractional factorial+foldover Doehlert Statgraphics Centurion
26

α-MoC 1–x Synthetic throughput 2
4
Factorial Doehlert Statgraphics Centurion
27

Co 3O 4 Size, shape, phase 2
6
Factorial Central Composite Not specified
28

Ni 2P Size 2
6-3
Fractional factorial Doehlert Statgraphics Centurion
29

CdS Fluorescence intensity 2
5
Factorial Central Composite Stat-Ease Design-Expert
30

CdS UV-vis exciton peak, band gap N/A Computer-generated Statgraphics Plus 4.1
31

CdSe Size, size distribution L 16 Orthogonal array N/A Not specified
32

MAPbI 3 PL peak intensity/position/width 2
8-4
Fractional factorial Face-Centered Not specified
33

MAPbI 3 PL absorption onset J2 Nearly orthogonal array Mixed-level factorial MODDEPro 11
25

TiO 2 Size, agglomeration Definitive screening N/A JMP
34

CuAlS 2 Band gap, PL intensity 2
6-2
Fractional factorial 2
3
Factorial Not specified
35

CuFe 2O 4 Phase purity (%) 2
2
Factorial 2
3
Factorial Not specified
36

α-Al(OH) 3 Yield N/A Box-Behnken Minitab
37

LiFePO 4 Discharge specific capacity 2
3
Factorial 3-Level factorial Not specified
38

CoFe 2O 4 Size 2
4
Factorial Central-Composite Not specified
39

Fe 3O 4 Size N/A Box-Behnken Design Expert
40

Sr:Ca 10(P
O 4) 6(OH) 2 Size, purity, crystallinity N/A 3
3
Factorial Not specified
41

Au Plasmon wavelength, yield, shape 2
8-4
Fractional factorial+foldover Bayesian Screening Not specified
42

Au Diameter, polydispersity Plackett-Burman Central-Composite Not specified
43

Ag Size N/A 3-Level L 9 array Not specified
44

Ag Plasmon FWHM Categorical Box-Behnken Not specified
45

Ag Particle size, conversion N/A Feed-forward NN Not specified
46

5
Herein, we provide a basic tutorial on how to implement DoE in conjunction with RSM for
the optimization of nanocrystal syntheses. First, we will outline the general sequence of steps in a
study utilizing DoE and RSM. We will then introduce how to design a study and set the boundaries
of a design space. Next, we will cover the theory, application, and analysis of design methods to
screen out insignificant variables. The same will be done for higher order optimization designs for
the optimization of target responses. Data sets for hands-on practice are available in Appendix A,
with design decisions that directly correlate to the data in Chapter 2.

Scheme 1.1. Flow chart of the general sequence of events in DoE to optimize a nanocrystal
synthesis, exemplifying an eight-experiment screening of three factors (to unconfound main
effects and binary interaction effects) and a subsequent optimization.

1.3. General Workflow of DoE
As illustrated in Scheme 1.1, the first step of a complete DoE study consists of defining
the research goals: What responses are of interest? Are you seeking to maximize, minimize, or
flexibly tune these responses? What are the possible experimental variables that may influence
6
these responses? With these questions answered, researchers can proceed to finding the reaction
boundaries, or plausible ranges of the variables being investigated via experiments, prior
experience, or experimental limitations (e.g., temperature limits based on solvent boiling points).
This often entails using a modified OVAT approach to define the highest and lowest levels of each
variable that still produce the target material to serve as the upper and lower edges, or bounds, of
the design space. These bounds are coded as -1 and +1 to normalize each variable so their effects
and interactions can be directly compared, regardless of disparities between units and magnitudes
(Figure 1.1). Once the bounds are set, an appropriate two-level screening design can be chosen
and implemented in the DoE software. A series of experiments to perform called a design matrix
is generated by satisfying the statistical requirements of the chosen design. These experiments
screen the effects of the experimental variables at different combinations of their high and low
levels. The number of experiments required depends on the number of different variables being
investigated, the synthetic goals, and the type of design being used. Once the observed responses
from these experiments are collected, a quantitative relationship can be established between the
input variables (factors) and their interactions on target responses via linear regression to create a
first-order linear model (i.e., a polynomial of degree at most one), that can determine factor
significance (Figure 1.2a). The statistically insignificant variables can be removed, enabling the
remaining variables to be efficiently investigated at a higher number of levels throughout the
design space. This provides detail on non-linear effects, or curvatures, in the response that enables
the data to be fit to a polynomial model via RSM (often called “higher-order” models). The model
can predict the optimum of the response throughout the entire parameter space, and outputs the
experimental variable values at said optimum.
24
The 3D response surfaces (Figure 1.2b) and 2D
contour plots (Figure 1.1.) are useful tools for model visualization. The model then must be
7
validated with additional experiments to compare the predicted response to the experimental
response.

Figure 1.2. (a) 2-level screening design for two variables. (b) 3-level response surface
optimization design for two variables.

1.4. Defining The Optimization Problem
The natural way to begin an investigation via DoE and RSM is to define the problem you are
trying to solve. This can be done by answering the following questions about the responses: (1)
What responses do you want to optimize? Common examples of nanocrystal responses include
size, polydispersity, yield, shape, and crystal phase. (2) What types of responses are they? There
are two types of responses: continuous and discrete. Continuous responses have a continuous
quantitative domain throughout the design space (e.g., size, yield, polydispersity). Discrete
responses can be binary (e.g., yes or no), categorical (e.g., shape, phase), or ordered (e.g., small,
medium, large) and are not continuous within the design space. Discrete responses are trickier
because a continuous vector of values is illogical and classic linear regression cannot be used.
Using DoE for such responses often requires them to be quantified in some way in order to make
them continuous (e.g., relative percent phase purity or spectral analyses that give insight on
8
shape).
36,42
(3) Do you want to maximize, minimize, or hit a target for each response? (4) If
multiple responses are to be optimized, are there some responses that are more important than
others? Response values can be weighted according to importance, and multi-response
optimizations require a desirability function (vide infra). (5) Does the optimizing goal make logical
sense? Some responses can be easily optimized together, like the morphological responses of size,
shape, and polydispersity. In most software, a sequence of dialog boxes will be displayed to collect
the answers to these questions.
Once the responses and their optimizing goals have been identified, questions about the
variables must be answered. (1) Which variables may influence the chosen responses? Variables
that are thought to be important can be chosen from prior experience or from a literature
investigation. (2) How many variables does it make sense to investigate? The number of variables
in the design directly impacts the number of experiments required to create an accurate model of
the synthetic system, so time and available resources must be considered. (2.1) What type of
variables are they? There are continuous variables, discrete variables, and mixture components.
Common continuous variables include time, temperature, concentration or solvent volume,
precursor ratios, and heating rate. For any number of k continuous variables, 2
k
number of reactions
are required to fully resolve the interaction effects. Common discrete or categorical variables
include the precursor used, the solvent used, and the reducing agent used. These often require
special categorical designs. A categorical variable with n number of categories will increase the
number of reactions necessary from 2
k
to n⋅2
k
, as each category of the variable must be uniformly
(or orthogonally) combined with every other variable in the contrast matrix (see Screening
Section). Designs with mixture component variables must be constrained to sum to 100% or some
other fixed value (since they are parts of a whole) and are called mixture designs. (2.2) Are any
9
key variables missing? Analyzing the design output gives insight into whether the chosen variables
are the right ones, which is discussed further in the design analysis and interpretation section. (2.3)
Is it possible to reduce the number of experiments while still maintaining acceptable resolution
with these variables (i.e., can the design be optimized)? Tools on how to do this will be discussed
in the design improvement section (vide infra), along with how to amend holes in the design (as
indicated by the answer to question 2.2). Again, most software will prompt you to input the
variables you have chosen to investigate.

1.5. Setting the Experimental Bounds
Once the optimization problem has been determined, the bounds of the experimental design
space must be set. Reaction bounds are the natural boundaries of the chosen variables used to
synthesize any given nanocrystal. Anything above or below the indicated range for a specific
variable should result in illogical conditions or a failed synthesis. These bounds define the edges
of your synthetic parameter space. The parameter bounds play a key role in an accurate
optimization and, if set incorrectly, bring the validity of the investigation into question. If the
ranges of the reaction bounds are too small, key areas of the design space will be omitted and a
local optimum rather than a global optimum is likely. Moreover, some cases may result in zero
statistically significant variables, inhibiting an optimization. If the ranges are too large,
combinations of different variables may result in no reaction or a reaction that produces an
unwanted by-product, making the resultant phase a new categorical response that must be dealt
with. The latter two cases would both require a full re-design, which would be time and resource
intensive, compromising one of the main advantages of DoE.
10
Due to the multivariate nature of DoE, each axis of the parameter space is initially defined
by the units and ranges of each unique variable. For example, a design investigating the effects of
time, temperature, and concentration will have a 3-D parameter space where the x-axis is defined
by temporal units (minutes, hours, etc.), the y-axis is defined by a temperature scale (˚C), and the
z-axis is defined by concentration (e.g., mmol/L), respectively. Since they are all different, these
experimental or “real” values must be normalized to perform statistical calculations and accurately
decipher the effects of each variable and their interactions. To do this, the range of experimental
values for each variable are set to range between -1 and +1, as illustrated for the variable bounds
in Table 1.2. All factors that make up the parameter space will now have axes that are identically
defined, and any point in the reaction space can be indicated by its corresponding set of coded
values (e.g., an experiment where every factor is at its high bound in a 3-D design space would be
represented by the point (+1,+1,+1)). Experimental variable values, Xi, can be coded anywhere
within the -1 to +1 range as xi using eq. 1.1:
𝑥
!
=
"
!
#"
!
"
$"
!
× 𝛼 (1.1)
where 𝑋
!
%
is the real value at the center of the experimental domain for factor i (or the value mid-
way between the lower and upper bound, coded as 0), Δ𝑋
!
is the step variation of the real value (or
the difference between the center value (0) and the value of the upper or lower bound), and a is
equal to the coded value limit for each variable, which in this case is one.
26,47
The “base run”, or
center point of a design, is defined as the coded 0 value of each variable. If a researcher prefers to
consider the “base-run” an experiment whose variable values are not exactly mid-way between the
real value range, this alternative value can be set as 𝑋
!
%
to maintain a code of 0, but the step variation
Δ𝑋
!
will be different in the positive and negative directions when coding all other values. The
coded reactions of a design matrix output by software can be turned back into their corresponding
11
real values by rearranging eq. 1.1 to solve for Xi. See Appendix A for example calculations.
Beyond normalizing the variables, extreme care must be taken to keep all other unexplored
variables in the synthetic procedure constant. Probing a multivariate domain amplifies nuances in
experimental technique, which could be wrongly attributed to factors in the design or add
unwanted error.

Table 1.2. Example table of reaction bounds and their coded counterparts.
Variable Low level High level Coded low
level
Coded high
level
Temperature 170 ˚C 320 ˚C -1 +1
Time 1 min 5 h -1 +1
Concentration (concentration) 4 mmol/L 12 mmol/L -1 +1

1.6. Screening Designs
When designing a multivariable screening experiment, it is important to decide which
screening design will best screen out statistically insignificant variables accurately and efficiently.
Standard screening designs are first-order and investigate each variable at two levels, which limits
them to the determination of linear influence of the variables (Figure 1.2a). This is standard since
the main objective of a screening is to identify inert factors as efficiently as possible for
dimensional reduction in the optimization phase. Employing the minimum number of levels (the
two extremes) not only minimizes the number of screening experiments but maximizes the
information obtained from this phase of DoE. The majority of screening designs are some variation
of a factorial, as their flexibility offers widespread generalizability while remaining simple and
effective.
21,48–50

12
1.6.1. Factorial Design Anatomy
Although almost all DoE software provides the experimental runs once you pick a screening
design, understanding the different designs and how they are made will ensure you choose a design
that properly handles your optimization problem. A full factorial design consists of

N
k

experimental combinations of k number of factors that are investigated at N number of levels per
factor (N = 2 for a screening). To better understand the factorial design, look at the table of signs
in Table 1.3. This is a contrast table for a 2-level screening of three factors (2
3
factorial). The
following discussion will demonstrate how to create, perform, analyze, and improve a screening
design using this table.
The signs in the three columns labeled A, B, and C define the design matrix of a 2
3
full factorial
design. These signs are obtained by identifying every possible combination of the high and low
levels of each factor (A, B, and C), which in this case is eight (N
k
number of total combinations,
2
3
= 8). The eight rows represent the eight experiments you would run once the unique coded value
combinations of A, B, and C in each row were translated back to the real-life conditions via eq.
1.1. For example, run 1 indicates an experiment where factors A, B, and C are all set to their lower
bound, which according to the bounds illustrated in Table 1.2 would be a reaction ran at a
temperature of 170 ˚C, a time of 1 min, and a concentration of 4 mmol/L. Values in between the
high and low bounds come into play later for higher-order optimizations.
Once the experiments are performed, 𝑦
!
represents the observed value of the response that
resulted from each experiment 𝑖. Looking again at Table 1.3, if more than one sample was taken
at each set of experimental conditions, the average value of the response for each run (𝑦 ,
!
) are
treated as single observations and go in the last column of the table. The first column of plus (+)
signs indicates how to obtain the overall or grand average; that is, add the observations 𝑦
!
and
13
divide by 8, since there are eight total experiments in the design. The remaining seven columns
define a set of seven contrasts, used to estimate the variable effects. The single factor columns A,
B, and C represent the main effects, or the quantification of how each factor individually affects
the response. This is followed by four additional columns labeled AB, AC, BC, and ABC, which
are the interaction effects. The values of each effect are calculated as the difference between two
averages (eq. 1.2):
𝐸𝑓𝑓𝑒𝑐𝑡 = 𝑦 ,
&
−𝑦 ,
#
(1.2)
where 𝑦 ,
&
is the average response on the face of the parameter space corresponding to the high
level of the factor and 𝑦 ,
#
is the average response on the opposite face for its low level.
21
The signs
in these columns thus indicate which observations go into 𝑦 ,
&
and which go into 𝑦 ,
#
for each effect.
For example, the value of the main effect A can be calculated from the A column of signs
(−,+,−,+,−,+,−,+) to give (eq. 1.3):
𝐸𝑓𝑓𝑒𝑐𝑡 𝐴 = (− 𝑦 ,
'
+𝑦 ,
(
−𝑦 ,
)
+𝑦 ,
*
−𝑦 ,
+
+𝑦 ,
,
−𝑦 ,
-
+𝑦 ,
.
) 4 ⁄ (1.3)
The divisor (4) transforms the contrast into the difference between the two averages (𝑦 ,
&
−𝑦 ,
#
)
since there are four plus (+) reactions and four minus (–) reactions. The B and C main effects are
calculated the same way from the signs in their columns.
The columns of signs for the four interaction contrasts are obtained by directly multiplying
the signs of their respective factors. Thus, the column of signs for the AB interaction is obtained
by multiplying together the signs for A with those for B, row by row. The estimate of the AB
interaction effect is then calculated the same way as the previous main effect but instead represents
the difference between the variable A effects at the high (𝑦 ,
&
)and low (𝑦 ,
#
) levels of the B factor.
One-half of this difference is called the variable A by variable B interaction, denoted AB.
Similarly, the AC interaction could equally be thought of as one-half the difference in average A
14
effects at the two levels of C. The estimate of the three-factor interaction ABC is calculated
similarly, with the column of signs resulting from the multiplication of columns A, B, and C. The
table of signs for all effects is identically obtained for any 2
k
factorial design. Each contrast column
is perfectly balanced with respect to all others to maintain orthogonality. Orthogonality ensures
that each estimated effect is unaffected by the magnitudes and signs of others. To see this, choose
any one of the seven contrast columns in Table 1.3 and look at the group of four plus (+) signs in
that column. In every one of the other six contrast columns there are two plus (+) signs and two
minus (–) signs opposite of those four plus (+) signs. The same is true for the corresponding group
of four minus (–) signs.
21

Table 1.3. A 2
3
full factorial design for three variables and their contrast coefficients.
Variable Mean A B C AB AC BC ABC Avg. Response
Run 1 + − − − + + + − 𝑦 $
!

Run 2 + + − − − − + + 𝑦 $
"

Run 3 + − + − − + − + 𝑦 $
#

Run 4 + + + − + − − − 𝑦 $
$

Run 5 + − − + + − − + 𝑦 $
%

Run 6 + + − + − + − − 𝑦 $
&

Run 7 + − + + − − + − 𝑦 $
'

Run 8 + + + + + + + + 𝑦 $
(

Divisor 8 4 4 4 4 4 4 4

1.6.2. Design Analysis and Interpretation
It is important to have some method for determining which effects are almost certainly real
and which might readily be explained by chance variation. Most programs employ the analysis of
variance, or ANOVA, for this, but ANOVA does not always make logical sense for 2-level (2
k
)
factorial experiments due to overcomplications.
21
The idea is to determine whether the
discrepancies between the averages of each effect are greater than could be reasonably expected
from the variation that occurs within the effects. The difference between those two things are the
residuals, or what is left due to experimental error and model inaccuracy. These can only be
15
accurately assessed if the design has an adequate amount of error degrees of freedom (d.f.), which
are defined as the number of degrees of freedom that remain after estimating all main and
interaction effects (including quadratic effects for higher order optimization designs). To calculate
these, subtract the total number of observations (eight in this case) from the total number of
estimated effects (seven total effects in Table 1.3, leaving one d.f.). In general, at least three d.f.
must be available if the statistical analyses are to have reasonable statistical power.
21
Replicate
runs of the experimental design and the addition of centerpoints help add necessary degrees of
freedom and reflect information about the run-to-run variability and experimental error. The
difference of the response values (𝑦
!
) of the replicate runs can be squared and divided by two to
obtain estimated variance at each set of conditions (𝑠
!
(
), as shown in in eq. 1.4:
𝑠
!
(
= I𝑦
!
#
−𝑦
!
$
J
(
2 ⁄ (1.4)
while the total average variance estimate in a design with 𝑛 number of replicate runs would be
(eq. 1.5):
𝑠
(
= ∑ (𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒)
(
2(𝑑.𝑓.) ⁄
/
!0'
(1.5)
Because each estimated effect is a difference between two averages of 2
k
observations, the variance
of a specific effect is given as eq. 1.6:
𝑉(𝑒𝑓𝑓𝑒𝑐𝑡)= O
'
(
%
+
'
(
%
P𝑠
(
(1.6)
The square root of this is the standard error (𝑆𝐸) of an effect, or 𝑆𝐸(𝑒𝑓𝑓𝑒𝑐𝑡). A rule of thumb is
that effects greater than two or three times their 𝑆𝐸 are not easily explained by chance alone.
21

More specifically, a “significant” value is a value of the t-statistic that is > t at the 5% level in a t-
table, or 0.05 (i.e., the 95% confidence interval, meaning there is above a 95% chance that the
effect is not resultant from chance alone) where t is defined by eq. 1.7:
𝑡 = 𝑒𝑓𝑓𝑒𝑐𝑡/𝑆𝐸(𝑒𝑓𝑓𝑒𝑐𝑡) (1.7)
16
These values of t are the standardized effects that test the null hypothesis, rendering the calculation
of an F-statistic needed for an ANOVA unnecessary. Linear regression is then used to create a
linear model of the system:
𝑦 = 𝛽
1
∑ 𝛽
𝔦
𝓍
𝔦
+𝜀
4
𝔦0'
(1.8)
where 𝛽0 is the mean, 𝛽i are the coefficients of the linear parameters, 𝑥𝑖 are the independent
variables, k is the number of factors, and 𝜀 is the residuals, or random error term, which is assumed
to follow a normal distribution with a mean value of zero that is independent from its constant
variance. A visualization of this model is depicted in Figure 1.2a.
There are several ways to visualize the statistical significance of the effects. A Pareto chart
(Figure 1.3a) shows the standardized main and interaction effects in decreasing order of their
absolute values (i.e., significance). The vertical line on the plot indicates the 95% confidence
interval. Factors that surpass the vertical line are deemed statistically significant; therefore, the
only significant effect in this study would be the interaction effect between factors A and C. The
main effects plot (Figure 1.3b) illustrates the directionality of the main effects, or their trends from
the low level to the high level. The longer the line, the more significant the effect, as moving from
the low level to the high level of a factor results in a greater change in the response, Y. Here factor
C shows the greatest main effect, although the Pareto chart shows it is not statistically significant
to the 95% confidence level. The positive slope indicates than an increase in factor C increases the
value of the response, which is where the factor’s positive sign comes from in the Pareto chart. A
normal probability plot (Figure 1.3c) displays the effects and residuals (𝜀i) versus quantiles of a
normal distribution, with a fitted line as reference. If the experimental data follows a normal
distribution, the points should lie along the straight line. Deviation from the line indicates large
experimental error (bad model fit), suggesting a need for higher-order optimization designs and
17
the presence of an outlier (a significant effect). Notice factor C falls far off the error line, appearing
significant in this graphical representation. This illustrates the need to analyze several types of
plots for accurate analysis. An interaction plot (Figure 1.3d) visualizes the interaction effects,
where each line represents either the low or high level of one factor in the interaction (indicated
by the signs on the lines) as it is varied from the low level to the high level of the second factor
(indicated on the x-axis). For example, in the BC interaction, the line labeled with plus signs
represents the high level of the B factor as it moves from the low to the high level of the C factor.
If two factors do not interact, moving from the low to the high level of one factor would not make
a difference at the low level versus the high level of another factor, and so the two lines would be
approximately parallel. Analyzing such plots aid in uncovering mechanistic trends in the system.
Analysis of an example screening output can be found in Appendix A.

Figure 1.3. First-order screening analysis of three variables on a response Y: (a) Pareto chart,
(b) main effects plot, (c) normal probability plot of the linear regression, and (d) interaction plot.

18
1.6.3. Design improvement
Considering that multiple batches of product are needed to optimize a nanocrystal synthesis,
batch variability is often a source of error in an experimental design. Blocking is a technique that
can be used to neutralize effects of possible run-to-run differences. In such a case, the runs are
divided into groups, or blocks, to be done together (experiments performed on the same day, or by
the same operator, or from the same batch of starting materials, etc.). However, as the number of
blocks increases, the ability to estimate certain interactions is lost. Splitting the eight reactions into
two blocks would be done by placing all the runs where the ABC contrast is a minus (–) in one
block and all the runs where it is a plus (+) in the other block. This deliberately confounds the
ABC interaction of each block with any run-to-run differences. Due to orthogonality, the only
interaction that would be affected by run-to-run differences is now the ternary ABC interaction,
leaving the main effects and binary interactions unaffected. This is a small sacrifice to eliminate
run-to-run variance, as the higher-order interactions are often assumed to be less important.
21
If
blocks of two experiments were desired, the blocking would come from pairs of ++, +−, −+,
and −− in two of the binary interaction columns. This would confound binary effects but leave
all main effects unaffected. Some specific blocking designs include randomized complete block
design (RCBD) and Latin square design, which are discussed in further detail in Appendix A.
When investigating more than five variables, full factorial designs are not recommended due
to the exponential increase in the number of required experiments.
51
Fractional factorial designs
are often much more efficient, as they allow for a chosen number of main effects and interaction
effects to be elucidated in a fraction of the experiments.
21,49
The specific fraction of experiments
to run is determined in a similar manner to blocking, but instead of confounding the effects with
the blocks, they are confounded amongst themselves to reduce the number of runs. For example,
19
a fourth factor D could be set to equal the ternary interaction ABC column of signs in Table 1.3
to create a ½ fraction of a 2
4
factorial design matrix (D = ABC). This is called the generating
relation and indicates which levels (+1 or -1) of D should be performed in eight runs (½ of 16).
The smaller the fraction, the more higher-level effects will be confounded. Thus, the choice of
fractional factorial design should be strategic. The amount of confounding can be determined from
the generating relation via design resolution, which is defined as the degree to which estimated
main effects are aliased with higher order interaction effects (2-level, 3-level, etc.), which is
elaborated upon in Appendix A. This specific design example has a resolution of IV and can
obtain clear estimates of all main effects, although some of the binary interactions are confounded.
The size of the fraction employed in the design is generally chosen based on minimizing the
number of experiments while maintaining sufficient resolution to solve the optimization problem.
In general, a fractional factorial is denoted by N
k-p
, where p is dictated by equating the desired
fraction of the full factorial to (1 2 ⁄ )
5
. Consider an investigation of five factors at two levels per
factor. The full factorial would consist of 2
5
, or 32 experiments. Here, N = 2 and k = 5, so the
design is a 2
5-p
design requiring 2
5-p
experiments. To employ a ¼ fraction of the full factorial, set
¼ = (1 2 ⁄ )
5
to get p = 2, so it is denoted as a 2
5-2
design that requires eight experiments (¼ of the
full 32). Such a design has a resolution of III, meaning the main effects are confounded with binary
interactions, but can be safely interpreted if all interaction effects are small or non-existent. A half
fraction, or 2
5-1
design consisting of 16 of the 32 experiments, would give a resolution of V. This
would give clear estimates of all main and binary effects. The full factorial would have a resolution
of V+, which has no confounding of any effects. The screening design itself is then capable of
being optimized or augmented after the results are assessed. If the design assessment shows that
there are no significant variables (via Pareto chart), or that the model fit for the screening design
20
is poor (via normal probability plot), it is likely that the design is inadequate or there are key
variables that were left out of the study. Rather than starting over from scratch, design
augmentation is an easy fix to a fractional design that confounds important effects. These holes
can be filled via elements like a single column fold-over, which switches all the signs in a single
column, unconfounding that column from all the other effects. One could also perform another
fraction, or more replicates.
Different kinds of screening designs also exist, including Plackett-Burman, which are 2-level
designs intended for screening many factors in a small number of runs, where the number of runs
is not a power of two. For example, a design is available for studying 11 factors in 12 runs.
50,52

Main effects are confounded with 2-factor interactions, so the design should only be used when
interactions are either not present or known to be small. Folded Plackett-Burman designs are
similar, but the 2-factor interactions are heavily confounded amongst themselves rather than with
the main effects, resolving the main effects but compromising any binary interactions.
52
The
Taguchi method is another technique that employs orthogonal arrays to greatly reduce the number
of experiments needed for studies investigating a very large number of factors.
48,49
Additionally,
Haaland irregular fraction designs are fractional factorial designs in which the number of runs is
not a power of two. Certain irregular fractions, although not completely orthogonal, have attractive
confounding patterns.
52,53
Some software also offer the option of random sampling, where an
indicated amount of experiments are randomly chosen throughout the design space.
Although any screening design will almost always offer valuable insight, different types of
designs are beneficial for different kinds of screening problems and are limited by experimental
resources and the computational tools available, so design choice will be unique for every
experimenter. Example screening designs and design augmentations can be found in Appendix A.
21

1.7. Optimization Designs
1.7.1. Design Characteristics
Once insignificant variables have been removed via screening, the significant variables can
be investigated at a greater number of levels, as illustrated by moving from Figure 1.2a to Figure
1.2b. Again, there are several types of optimization designs with unique characteristics. The
overwhelming majority of DoE optimizations utilize response surface methodology. Other
methods like Bayesian optimizations and simplex optimizations exist, where the former takes into
account prior or exterior information and is more suitable for machine learning, while the latter is
an older stepwise strategy.
54–56
Some characteristics of optimization designs include: (1) Rotatable,
where points are placed so that the variance of the predicted response is the same at all points that
are the same distance from the center of the design (in standard units). This is intuitively reasonable
since the experimenter typically does not know beforehand in which direction the optimum is
likely to be located. Any number of center points may be added (3-5 center points are usually
desirable), which may be placed in a separate block if desired. (2) Orthogonal, which places the
points at an axial distance that ensures all second order terms are orthogonal to one another.
(3) Rotatable and orthogonal, which achieves both properties by proper selection of the number of
center points (𝑛
6
). This is done by setting the axial distance equal to that required to achieve
rotatability, and then selecting a value for 𝑛
6
that makes the axial distance for orthogonality the
same as for rotatability. (4) Face-centered, which places the points at the low and high levels of
the factorial design (i.e., sets a = 1.0). Although less desirable from a statistical perspective, such
a design may be easier to run since it involves only low, medium, and high levels of each factor.
For such designs, 1-2 center points are usually sufficient.
22
Some common response surface designs include: (1) Full three-level design, a simpler design
consisting of all combinations of three levels of each experimental factor (-1,0,1), also known as
a type of orthogonal array design, L9 (Figure 1.4a).
49
(2) Other Taguchi method orthogonal arrays
for 3+ levels. (3) Box-Behnken 3-level designs that include a subset of the runs in the full three
level factorial (Figure 1.4b). (4) Central composite designs consisting of a central two-level
factorial or resolution V fractional factorial design, plus additional points some distance from the
low and high levels used to model curvature with respect to each factor (Figure 1.4c). (5) Doehlert,
or uniform shell, designs, whose dodecahedron shape within the design space allows some factors
to be studied more in depth than others and are superior for studies where some variables clearly
have greater effects than others (Figure 1.4d). Interestingly, this design results from a fully rotated
simplex.
54
(6) Less common designs like Draper-Lin, which are small central composite designs
in which the central portion of the design is less than resolution V.
47,57,58
Each small sphere
illustrated in Figure 1.4 denotes coded experimental conditions within the parameter space to be
performed after conversion to their real-values via eq. 1.1. Examples of optimization design
matrices for different numbers of factors are provided in Appendix A.

23

Figure 1.4. Visual representations of 2-factor vs. 3-factor optimization designs in a parameter
space, with spheres representing reactions performed in the design. (a) Three level factorial, (b)
Box-Behnken, (c) central composite, and (d) Doehlert designs.

1.7.2. Choosing an Optimization Design
When choosing a higher-level design, the optimization problem, the design characteristics,
and the nature of the design space coverage are important considerations. Central composite
designs are popular, as they are easily made orthogonal and rotatable, and their curvature
estimation is good for minimization or maximization of responses. While each factor is being
varied, the other factors are held at their central values, and they typically investigate each factor
24
at five levels. For situations when the experimenter wishes to run only three levels of the factors
yet desires a design that is close to rotatable, Box-Behnken is a good alternative. While the central
composite design with face-centered center points also has three levels, Box-Behnken places most
of the design points at the corners of the experimental region or pre-defined parameter space. If
that region defines the feasible conditions for the process, the face-centered central composite is a
natural choice. However, if one is starting at a particular combination of factors and is simply
searching for better conditions, a more spherical arrangement of design points, such as the Doehlert
design, will be more efficient. Doehlert is particularly superior for studies where some variables
clearly have greater effects than others, as levels of investigation can be tailored to factor
significance, maintaining accuracy while increasing efficiency in a minimization, maximization,
or to hit a target.
47
When the cost of experimentation is high, the experimenter may wish to keep
the number of runs as small as possible and choose simpler designs, such as full 3-level, orthogonal
arrays, or Draper-Lin.

1.7.3. Design Analysis and Interpretation
Once an optimization design is chosen and the experiments are performed, a correlation
matrix shows the extent of confounding amongst the effects in the design. A perfectly orthogonal
design will show a diagonal matrix with 1s on the diagonal and 0s on the off diagonal. Any non-
zero terms off the diagonal imply that the estimates of the effects corresponding to that row and
column will be correlated. If there are no non-zero terms greater than or equal to 0.5, the results
can be interpreted without much difficulty. The response data can be used to fit a multiple linear
regression equation (quadratic polynomial) that models the system (eq. 1.9):
𝑦 = 𝛽
1
+∑ 𝛽
𝔦
𝑥
𝔦
+∑ 𝛽
𝔦𝔦
𝑥
𝔦
(
+∑ 𝛽
!7
4
'8!87
𝑥
!
𝑥
7
+
4
!0'
𝜀
4
!0'
(1.9)
25
where 𝛽0 is the mean, 𝛽i are the coefficients of the linear parameters 𝑖 (main effects), 𝛽ij are the
coefficients of the interaction parameters 𝑖𝑗, 𝛽ii are the coefficients of the quadratic parameters 𝑖𝑖,
𝑥𝑖 are the independent variables, and 𝜀 is the residuals, or random error term. In general, the
magnitude of the coefficients directly correlates to the magnitude of the corresponding effect (if
factor A has a large positive coefficient value it has an equally large positive effect on the
response). This equation predicts the value of a single response anywhere within the indicated
bounds of the system by minimizing the error terms of the residuals and estimating variance. The
smaller the error terms, the greater the prediction accuracy. To assess model fit, software packages
typically include a calculation of variance inflation factors (VIF), which measure the extent to
which the variance of the estimated coefficients are inflated due to lack of orthogonality in the
design. A perfectly orthogonal design would equal 1 for all factors. VIFs above 10 are usually
considered to indicate serious non-orthogonality. A related statistic calculated for model fit is Ri-
squared, which measures the extent to which a coefficient is correlated with other coefficients.
Power is also an important metric, which indicates the probability of identifying an effect of a
given magnitude as being statistically significant at a specific signal-to-noise (S/N) ratio. S/N ratio
is defined as twice the magnitude of the regression coefficient divided by the standard deviation
of the experimental error. For example, if power = 15% at S/N = 0.5, the probability of obtaining
a significant result for effect A, when that effect is half the experimental error, equals 15%. As
previously mentioned, software packages often analyze model variance using an ANOVA table to
partition the variability in the response into separate pieces for each of the effects. Notably, this
includes an R-squared statistic that indicates the percentage of the variability in the response that
is explained by the model. Typically, a model that explains more than 90% of the variability is
26
acceptable, although 95% or above is desired. An example analysis of model fit and a detailed
explanation of how to interpret the results are available in Appendix A.

Figure 1.5. (a) Example response surface of one response and two variables. (b) Corresponding
Pareto chart. (c) Quadratic main effects plot. (d) Quadratic interaction effects plot.

If the fit is adequate, model predictions can be used to map a response surface, which can be
readily visualized for two (Figure 1.5a) or three factors (Figure 1.7). This allows easy
identification of an optimum within the design space. High-dimensional design spaces (more than
three factors) can be visualized via several 2-D contour plots across factor pairs (Figure 1.6).
Similar statistical assessment plots described for screening designs are used for higher-level
designs (Figure 1.5b-d). Notice that the Pareto chart (Figure 1.5b) now includes estimates of the
quadratic effects, denoted by pairs of letters (AA, BB, etc.). Similarly, the effects plots (Figure
1.5c-d) are now quadratics rather than the straight lines seen for the screening stage (Figure
27
1.3b,d). This is because the increase in levels investigated for each factor allows for an estimation
of curvature.

Figure 1.6. Example contour plots for an optimization design with three factors: Contour plots of
the response Y while factor B is held constant at the (a) low level (coded -1), (b) center (coded 0),
and (c) high level (coded +1). Contour plots of the response Y while factor A is held constant at
the (d) low level (coded -1), (e) center (coded 0), and (f) high level (coded +1). Contour plots of
the response Y while factor C is held constant at the (g) low level (coded -1), (h) center (coded 0),
and (i) high level (coded +1).

1.7.4. Multi-Response Optimizations
So far, the optimizations discussed have only dealt with one response variable, such as
nanocrystal size. If there are two or more responses in the experiment that you wish to optimize
together, one must perform a multi-response optimization. To do this, each response must first be
optimized individually and fit to its own multiple linear regression equation. A multiple response
optimization can then be performed to predict the overall most optimal conditions of all the
responses via a desirability function. This function, 𝑑(𝛾 X), quantifies and jointly optimizes each of
28
the quadratic model equations by expressing the desirability of a response value (𝛾 X) on a scale of
0 to 1. This function takes one of three forms depending on whether the response is to be
maximized, minimized, or hit a specified target value. For example, if a response variable is to be
maximized, the desirability function is defined by eq. 1.10:
𝑑 =
⎩
⎪
⎨
⎪
⎧
0
O
9 :# ;<=
>!?> # ;<=
P
@
,
1

𝛾 ] < 𝑙𝑜𝑤
𝑙𝑜𝑤 ≤ 𝛾 X ≤ ℎ𝑖𝑔ℎ
𝛾 X > ℎ𝑖𝑔ℎ
(1.10)
where 𝛾 X is the predicted value of the response variable, low is the value below which the response
is completely unacceptable, and high is a value above where the desirability is at its maximum.
The power value s, named “weight”, defines the shape of the function and it is set by the
experimenter to determine how critical it is for 𝛾 ] to be close to the maximum. The function for
minimization is the mirror image of that for maximization, starting with 1 at the low value and
going to 0 at the high value. The desirability of all the responses can then be combined by creating
a single composite function D known as the global desirability to determine the best joint
responses. If all the responses are considered equally important, then D is given by eq. 1.11:
𝐷 = {𝑑
A
,𝑑
B
,𝑑
C
…𝑑
/
}
#
&
(1.11)
where n is the number of responses being optimized and a desirability of 100% corresponds to
each of the predicted optima in the multiple response optimization being identical to their
individual optima. If the optimizations of the responses are not of equal importance, then each 𝑑
can be weighted accordingly. The desirability results can be modeled visually using 3D contour
plots (Figure 1.7). An example of a multi-response optimization was performed by Williamson
and co-workers
26
and is detailed in Appendix A.

29

Figure 1.7. Desirability plot for a multi-response optimization of three responses using a 3-factor
Doehlert optimization design.

1.7.5. Validating the Model
The final step in a DoE and RSM optimization is to validate the model. That is, perform
the experiments that the model predicts to be the best and see how well the experimental responses
at those values match up to the predicted responses. If the experimental response at the predicted
optimum matches the predicted responses for the experiment (e.g., through an average of triplicate
runs) and the overall analysis of the error and variance of the model render it statistically
significant, then the true optimum of the system has likely been found. If not, a local optimum
could have been found, or the optimization design is missing key experimental points within the
design space. Like screening, design augmentation would then be necessary to provide more data
for the software to use to fit the model more accurately. This could include adding more levels to
each factor or adding a different type of screening design that complements the current model
within the design space. Once the true optimum is found, this response (or responses) can be
compared to the response (or responses) at the unoptimized baseline (coded as 0’s). This indicates
how well the optimization design improved the target responses. The examples of successful DoE
optimizations in Table 1.1 highlight just how effective this method is for predicting optimum
30
conditions throughout the vast parameter spaces that are typically encountered in nanocrystal
syntheses.

1.8. Conclusions
We introduced the general procedure of optimizing a nanocrystal synthesis using DoE
approaches. We illustrated how to accurately establish, perform, and analyze screening and
optimization designs. DoE is a very useful technique to gain synthetic control over nanocrystal
characteristics compared to the traditional OVAT method. We highlight its ability to estimate the
main and interaction effects of variables and predict a true optimum within the parameter space,
while keeping the number of experiments to a minimum.
DoE does have limitations. The reliance on linear regression prevents it from establishing
more complex non-linear models, or classification models for categorical responses, such as
crystal phase. Additionally, revealing the mechanistic insights of a reaction relies on researcher
capability and accurate analysis of the results.
59
As an alternative, machine learning techniques
can find underlying non-linear relationships among the responses and variables but require a much
larger data set that is not always possible in the exploration of experimental parameter space for a
novel synthesis or nanocrystal type. Active learning methods based on Bayesian optimization can
incorporate instant feedback as each experiment proceeds, and thus provide an efficient pathway
toward more complex and categorical optimizations, but typically require an initial DoE screening
as a base or surrogate model.
20,23
Thus, DoE is a necessary tool for many machine learning
techniques, while remaining an effective way to optimize a nanocrystal synthesis. This is
especially true when there is not sufficient data or inexpensive high-throughput experimental
techniques to support or train a machine learning study. DoE is also more user-friendly than
31
machine learning techniques since DoE computer software does not require high-level statistics or
coding knowledge, offering a quick and easy route to similar high-level conclusions. Through the
steps outlined in this guide on the implementation of DoE, we hope more researchers can apply
this powerful statistical tool in their own work and move towards standardizing such tools for
nanocrystal synthetic optimizations.

1.9. References
(1) Yuan, M.; Liu, M.; Sargent, E. H. Colloidal Quantum Dot Solids for Solution-Processed Solar
Cells. Nat. Energy 2016, 1, 16016.

(2) Akkerman, Q. A.; Gandini, M.; Di Stasio, F.; Rastogi, P.; Palazon, F.; Bertoni, G.; Ball, J.
M.; Prato, M.; Petrozza, A.; Manna, L. Strongly Emissive Perovskite Nanocrystal Inks for
High-Voltage Solar Cells. Nat. Energy 2017, 2, 16194.

(3) Yang, H.; Jauregui, L. A.; Zhang, G.; Chen, Y. P.; Wu, Y. Nontoxic and Abundant Copper
Zinc Tin Sulfide Nanocrystals for Potential High-Temperature Thermoelectric Energy
Harvesting. Nano Lett. 2012, 12, 540–545.

(4) Ibáñez, M.; Zamani, R.; LaLonde, A.; Cadavid, D.; Li, W.; Shavel, A.; Arbiol, J.; Morante,
J. R.; Gorsse, S.; Snyder, G. J.; Cabot, A. Cu2ZnGeSe4 Nanocrystals: Synthesis and
Thermoelectric Properties. J. Am. Chem. Soc. 2012, 134, 4060–4063.

(5) Shi, Y.; Lyu, Z.; Zhao, M.; Chen, R.; Nguyen, Q. N.; Xia, Y. Noble-Metal Nanocrystals with
Controlled Shapes for Catalytic and Electrocatalytic Applications. Chem. Rev. 2021, 121,
649–735.

(6) Huang, J.; Buonsanti, R. Colloidal Nanocrystals as Heterogeneous Catalysts for
Electrochemical CO2 Conversion. Chem. Mater. 2019, 31, 13–25.

(7) Zhong, Y.; Ma, Z.; Zhu, S.; Yue, J.; Zhang, M.; Antaris, A. L.; Yuan, J.; Cui, R.; Wan, H.;
Zhou, Y.; Wang, W.; Huang, N. F.; Luo, J.; Hu, Z.; Dai, H. Boosting the Down-Shifting
Luminescence of Rare-Earth Nanocrystals for Biological Imaging beyond 1500 Nm. Nat.
Commun. 2017, 8, 737.

(8) Shao, W.; Chen, G.; Kuzmin, A.; Kutscher, H. L.; Pliss, A.; Ohulchanskyy, T. Y.; Prasad, P.
N. Tunable Narrow Band Emissions from Dye-Sensitized Core/Shell/Shell Nanocrystals in
the Second Near-Infrared Biological Window. J. Am. Chem. Soc. 2016, 138, 16192–16195.

32
(9) Chen, Q.; Wu, J.; Ou, X.; Huang, B.; Almutlaq, J.; Zhumekenov, A. A.; Guan, X.; Han, S.;
Liang, L.; Yi, Z.; Li, J.; Xie, X.; Wang, Y.; Li, Y.; Fan, D.; Teh, D. B. L.; All, A. H.;
Mohammed, O. F.; Bakr, O. M.; Wu, T.; Bettinelli, M.; Yang, H.; Huang, W.; Liu, X. All-
Inorganic Perovskite Nanocrystal Scintillators. Nature 2018, 561, 88–93.

(10) Maier-Flaig, F.; Rinck, J.; Stephan, M.; Bocksrocker, T.; Bruns, M.; Kübel, C.; Powell, A.
K.; Ozin, G. A.; Lemmer, U. Multicolor Silicon Light-Emitting Diodes (SiLEDs). Nano Lett.
2013, 13, 475–480.

(11) Luo, J.-W.; Franceschetti, A.; Zunger, A. Carrier Multiplication in Semiconductor
Nanocrystals: Theoretical Screening of Candidate Materials Based on Band-Structure
Effects. Nano Lett. 2008, 8, 3174–3181.

(12) Yu, Y.; Fan, G.; Fermi, A.; Mazzaro, R.; Morandi, V.; Ceroni, P.; Smilgies, D.-M.; Korgel,
B. A. Size-Dependent Photoluminescence Efficiency of Silicon Nanocrystal Quantum Dots.
J. Phys. Chem. C 2017, 121, 23240–23248.

(13) Guisbiers, G.; Liu, D.; Jiang, Q.; Buchaillot, L. Theoretical Predictions of Wurtzite III-Nitride
Nano-Materials Properties. Phys. Chem. Chem. Phys. 2010, 12 (26), 7203-7210.

(14) Tappan, B. A.; Brutchey, R. L. Polymorphic Metastability in Colloidal Semiconductor
Nanocrystals. ChemNanoMat 2020, 6, 1567–1588.

(15) Ummadisingu, A.; Meloni, S.; Mattoni, A.; Tress, W.; Grätzel, M. Crystal‐Size‐Induced
Band Gap Tuning in Perovskite Films. Angew. Chem. Int. Ed. 2021, 60, 21368–21376.

(16) Kim, Y.-H.; Wolf, C.; Kim, Y.-T.; Cho, H.; Kwon, W.; Do, S.; Sadhanala, A.; Park, C. G.;
Rhee, S.-W.; Im, S. H.; Friend, R. H.; Lee, T.-W. Highly Efficient Light-Emitting Diodes of
Colloidal Metal–Halide Perovskite Nanocrystals beyond Quantum Size. ACS Nano 2017, 11,
6586–6593.

(17) Li, Y.; Wen, J.; Ali, A. M.; Duan, M.; Zhu, W.; Zhang, H.; Chen, C.; Li, Y. Size Structure–
Catalytic Performance Correlation of Supported Ni/MCF-17 Catalysts for COx-Free
Hydrogen Production. Chem. Commun. 2018, 54, 6364–6367.

(18) Muthuswamy, E.; Savithra, G. H. L.; Brock, S. L. Synthetic Levers Enabling Independent
Control of Phase, Size, and Morphology in Nickel Phosphide Nanoparticles. ACS Nano 2011,
5, 2402–2411.

(19) Barim, G.; Smock, S. R.; Antunez, P. D.; Glaser, D.; Brutchey, R. L. Phase Control in the
Colloidal Synthesis of Well-Defined Nickel Sulfide Nanocrystals. Nanoscale 2018, 10,
16298–16306.

(20) Braham, E. J.; Davidson, R. D.; Al-Hashimi, M.; Arróyave, R.; Banerjee, S. Navigating the
Design Space of Inorganic Materials Synthesis Using Statistical Methods and Machine
Learning. Dalton Trans. 2020, 49, 11480-11488.
33
(21) Box, G. E. P.; Hunter, J. S.; Hunter, W. G. Statistics for Experimenters: Design, Innovation,
and Discovery, 2nd ed.; Wiley series in probability and statistics; Wiley-Interscience:
Hoboken, N.J, 2005.

(22) Goodman, E. D.; Carlson, E. Z.; Dietze, E. M.; Tahsini, N.; Johnson, A.; Aitbekova, A.;
Nguyen Taylor, T.; Plessow, P. N.; Cargnello, M. Size-Controlled Nanocrystals Reveal
Spatial Dependence and Severity of Nanoparticle Coalescence and Ostwald Ripening in
Sintering Phenomena. Nanoscale 2021, 13, 930–938.

(23) Cao, B.; Adutwum, L. A.; Oliynyk, A. O.; Luber, E. J.; Olsen, B. C.; Mar, A.; Buriak, J. M.
How To Optimize Materials and Devices via Design of Experiments and Machine Learning:
Demonstration Using Organic Photovoltaics. ACS Nano 2018, 12, 7434-7444.

(24) Bezerra, M. A.; Santelli, R. E.; Oliveira, E. P.; Villar, L. S.; Escaleira, L. A. Response Surface
Methodology (RSM) as a Tool for Optimization in Analytical Chemistry. Talanta 2008, 76,
965–977.

(25) Reinhardt, E.; Salaheldin, A. M.; Distaso, M.; Segets, D.; Peukert, W. Rapid Characterization
and Parameter Space Exploration of Perovskites Using an Automated Routine. ACS Comb.
Sci. 2020, 22, 6–17.

(26) Williamson, E. M.; Tappan, B. A.; Mora-Tamez, L.; Barim, G.; Brutchey, R. L. Statistical
Multiobjective Optimization of Thiospinel CoNi2S4 Nanocrystal Synthesis via Design of
Experiments. ACS Nano 2021, 15, 9422–9433.

(27) Karadaghi, L. R.; Madani, M. S.; Williamson, E. M.; To, A. T.; Habas, S. E.; Baddour, F. G.;
Schaidle, J. A.; Ruddy, D. A.; Brutchey, R. L.; Malmstadt, N. Throughput Optimization of
Molybdenum Carbide Nanoparticle Catalysts in a Continuous Flow Reactor Using Design of
Experiments. ACS Appl. Nano Mater. 2022, 5, 1966-1975.

(28) Mingle, K.; Lauterbach, J. Synthesis-Structure-Activity Relationships in Co3O4 Catalyzed
CO Oxidation. Front. Chem. 2018, 6, 185.

(29) Mora-Tamez, L.; Barim, G.; Downes, C.; Williamson, E. M.; Habas, S. E.; Brutchey, R. L.
Controlled Design of Phase- and Size-Tunable Monodisperse Ni2P Nanoparticles in a
Phosphonium-Based Ionic Liquid through Response Surface Methodology. Chem. Mater.
2019, 31, 1552–1560.

(30) Di Tocco, A.; Porcal, G. V.; Riberi, W. I.; Zon, M. A.; Fernández, H.; Robledo, S. N.;
Arévalo, F. J. Synthesis of Stable CdS Nanocrystals Using Experimental Design:
Optimization of the Emission. New J. Chem. 2019, 43, 12836–12845.

(31) Barglik-Chory, Ch.; Remenyi, Ch.; Strohm, H.; Müller, G. Adjustment of the Band Gap
Energies of Biostabilized CdS Nanoparticles by Application of Statistical Design of
Experiments. J. Phys. Chem. B 2004, 108, 7637–7640.
34
(32) Protière, M.; Nerambourg, N.; Renard, O.; Reiss, P. Rational Design of the Gram-Scale
Synthesis of Nearly Monodisperse Semiconductor Nanocrystals. Nanoscale Res. Lett. 2011,
6, 472.

(33) Baker, R. W.; Forfar, L.; Liang, X.; Cameron, P. J. Using Design of Experiment to Obtain a
Systematic Understanding of the Effect of Synthesis Parameters on the Properties of
Perovskite Nanocrystals. React. Chem. Eng. 2021, 6, 709–719.

(34) Billet, J.; Dujardin, W.; De Keukeleere, K.; De Buysser, K.; De Roo, J.; Van Driessche, I.
Size Tunable Synthesis and Surface Chemistry of Metastable TiO2-Bronze Nanocrystals.
Chem. Mater. 2018, 30, 4298–4306.

(35) Baum, F.; Pretto, T.; Gouvêa, R. A.; Santos, M. J. L. Design of Experiments and Theoretical
Investigation for Photoluminescence Optimization of Copper Aluminum Sulfide
Nanoparticles through Controlling Crystalline Defects. Cryst. Growth Des. 2022, 22, 3669–
3679.

(36) Calvo-de la Rosa, J.; Segarra, M. Optimization of the Synthesis of Copper Ferrite
Nanoparticles by a Polymer-Assisted Sol–Gel Method. ACS Omega 2019, 4, 18289–18298.

(37) Zhang, X.; Zhang, X.; Graham, T. R.; Pearce, C. I.; Mehdi, B. L.; N’Diaye, A. T.; Kerisit, S.;
Browning, N. D.; Clark, S. B.; Rosso, K. M. Fast Synthesis of Gibbsite Nanoplates and
Process Optimization Using Box-Behnken Experimental Design. Cryst. Growth Des. 2017,
17, 6801–6808.

(38) Liu, Z.; Zhu, Y.; Gao, P.; Zhang, Z. Application of the Factorial Design of Experiments to
Hydrothermal Synthesis of Lithium Iron Phosphate. Int. J. Appl. Ceram. Technol. 2020, 17,
1231–1240.

(39) Rondinone, A. J.; Samia, A. C. S.; Zhang, Z. J. A Chemometric Approach for Predicting the
Size of Magnetic Spinel Ferrite Nanoparticles from the Synthesis Conditions. J. Phys. Chem.
B 2000, 104, 7919–7922.

(40) Forge, D.; Roch, A.; Laurent, S.; Tellez, H.; Gossuin, Y.; Renaux, F.; Vander Elst, L.; Muller,
R. N. Optimization of the Synthesis of Superparamagnetic Contrast Agents by the Design of
Experiments Method. J. Phys. Chem. C 2008, 112, 19178–19185.

(41) Kavitha, M.; Subramanian, R.; Vinoth, K. S.; Narayanan, R.; Venkatesh, G.; Esakkiraja, N.
Optimization of Process Parameters for Solution Combustion Synthesis of Strontium
Substituted Hydroxyapatite Nanocrystals Using Design of Experiments Approach. Powder
Technol. 2015, 271, 167–181.

(42) Burrows, N. D.; Harvey, S.; Idesis, F. A.; Murphy, C. J. Understanding the Seed-Mediated
Growth of Gold Nanorods through a Fractional Factorial Design of Experiments. Langmuir
2017, 33, 1891–1907.
35
(43) Fhionnlaoich, N. M.; Yang, Y.; Qi, R.; Galvanin, F.; Guldin, S. DoE-It-Yourself: A Case
Study for Implementing Design of Experiments into Nanoparticle Synthesis. ChemRxiv
(Chemical Engineering and Industrial Chemistry), May 30, 2019, DOI:
10.26434/chemrxiv.8198420.v1 (accessed 2022-09-01).

(44) Kim, K. D.; Han, D. N.; Kim, H. T. Optimization of Experimental Conditions Based on the
Taguchi Robust Design for the Formation of Nano-Sized Silver Particles by Chemical
Reduction Method. Chem. Eng. J. 2004, 104, 55–61.

(45) Viana, A. D.; Nobrega, E. T. D.; Moraes, E. P.; Wanderley Neto, A. O.; Menezes, F. G.;
Gasparotto, L. H. S. Castor Oil Derivatives in the Environmentally Friendly One-Pot
Synthesis of Silver Nanoparticles: Application in Cysteine Sensing. Mater. Res. Bull. 2020,
124, 110755.

(46) Chang, J.-S.; Lee, Y.-P.; Wang, R.-C. Optimization of Nanosized Silver Particle Synthesis via
Experimental Design. Ind. Eng. Chem. Res. 2007, 46, 5591–5599.

(47) Ferreira, S. L. C.; dos Santos, W. N. L.; Quintella, C. M.; Neto, B. B.; Bosque-Sendra, J. M.
Doehlert Matrix: A Chemometric Tool for Analytical Chemistry—Review. Talanta 2004, 63,
1061–1067.

(48) Roy, R. A Primer on the Taguchi Method, 2nd ed.; Society of Manufacturing Engineers, 2010.

(49) Taguchi, G. Introduction to Quality Engineering: Designing Quality into Products and
Processes; The Organization: Tokyo, 1986.

(50) Vanaja, K.; Shobha Rani, R. H. Design of Experiments: Concept and Applications of Plackett
Burman Design. Clin. Res. Regul. Aff. 2007, 24, 1–23.

(51) Gutiérrez Pulido, H.; Vara Salazar, R. de la. Análisis y diseño de experimentos, 2a ed.;
McGraw-Hill: México, 2008.

(52) Statgraphics Centurion, version 19; Statgraphics Technologies, Inc.: The Plains, VA, 2020.

(53) Haaland, P. Statistical Problem Solving. Exp. Des. Biotechnol. 1989, 1–18.

(54) Lundstedt, T.; Seifert, E.; Abramo, L.; Thelin, B.; Nyström, Å.; Pettersen, J.; Bergman, R.
Experimental Design and Optimization. Chemom. Intell. Lab. Syst. 1998, 42, 3–40.

(55) Mekki-Berrada, F.; Ren, Z.; Huang, T.; Wong, W. K.; Zheng, F.; Xie, J.; Tian, I. P. S.;
Jayavelu, S.; Mahfoud, Z.; Bash, D.; Hippalgaonkar, K.; Khan, S.; Buonassisi, T.; Li, Q.;
Wang, X. Two-Step Machine Learning Enables Optimized Nanoparticle Synthesis. Npj
Comput. Mater. 2021, 7, 55.

(56) Braham, E. J.; Cho, J.; Forlano, K. M.; Watson, D. F.; Arròyave, R.; Banerjee, S. Machine
Learning-Directed Navigation of Synthetic Design Space: A Statistical Learning Approach to
36
Controlling the Synthesis of Perovskite Halide Nanoplatelets in the Quantum-Confined
Regime. Chem. Mater. 2019, 31, 3281–3292.

(57) Doehlert, D. H. Uniform Shell Designs. Appl. Stat. 1970, 19, 231-239.

(58) Zhou, B.; Li, Y.; Gillespie, J.; He, G.-Q.; Horsley, R.; Schwarz, P. Doehlert Matrix Design
for Optimization of the Determination of Bound Deoxynivalenol in Barley Grain with
Trifluoroacetic Acid (TFA). J. Agric. Food Chem. 2007, 55, 10141–10149.

(59) Lendrem, D.; Owen, M.; Godbert, S. DOE (Design of Experiments) in Development
Chemistry: Potential Obstacles. Org. Process Res. Dev. 2001, 5, 324–327.

37
Chapter 2. Statistical Multiobjective Optimization of Thiospinel CoNi2S4 Nanocrystal
Synthesis via Design of Experiments*

*Published in ACS Nano 2021, 15, 9422-9433.

2.1. Abstract
Thiospinels, such as CoNi2S4, are showing promise for numerous applications, including
as catalysts for the hydrogen evolution reaction, hydrodesulfurization, and oxygen evolution and
reduction reactions; however, CoNi2S4 has not been synthesized as small, colloidal nanocrystals
with high surface-area-to-volume ratios. Traditional optimization methods to control nanocrystal
attributes such as size typically rely upon one variable at a time (OVAT) methods that are not only
time and labor intensive but also lack the ability to identify higher-order interactions between
experimental variables that affect target outcomes. Herein, we demonstrate that a statistical design
of experiments (DoE) approach can optimize the synthesis of CoNi2S4 nanocrystals, allowing for
control over the responses of nanocrystal size, size distribution, and isolated yield. After
implementing a 2
5−2
fractional factorial design, the statistical screening of five different
experimental variables identified temperature, Co:Ni precursor ratio, Co:thiol ratio, and their
higher-order interactions as the most critical factors in influencing the aforementioned responses.
Second-order design with a Doehlert matrix yielded polynomial functions used to predict the
reaction parameters needed to individually optimize all three responses. A multiobjective
optimization, allowing for the simultaneous optimization of size, size distribution, and isolated
yield, predicted the synthetic conditions needed to achieve a minimum nanocrystal size of 6.1 nm,
a minimum polydispersity (σ/d ̅ ) of 10%, and a maximum isolated yield of 99%, with a desirability
of 96%. The resulting model was experimentally verified by performing reactions under the
38
specified conditions. Our work illustrates the advantage of multivariate experimental design as a
powerful tool for accelerating control and optimization in nanocrystal syntheses.

2.2. Introduction
Transition metal sulfides with an AB2S4 spinel-type structure (where A = 2+ cation and B
= 3+ cation) are emergent materials for a range of applications, such as catalysis,
1–3
energy
harvesting,
4
energy storage and conversion,
5
and electronics.
6–9
The transition metal A and B
cations partially occupy tetrahedral and octahedral sites in the spinel structure, respectively, and
the atoms can also interchange positions, leading to inverse spinel-type structures.
2,4,10
Since the
A and B cations may possess the ability to switch their valency between 2+ and 3+, thiospinels
offer advantages with respect to redox properties, conductivity, and they also possess surface-
abundant sulfur vacancies.
2,8,9
Moreover, thiospinels are more covalent than their oxide
counterparts, and their high catalytic activity can be attributed to the strong synergistic effect
between cobalt ions, nickel ions and the unsaturated sulfur atoms.
1,8,9,11
Despite these properties,
thiospinels of (Ni,Co)3S4 are generally underexplored, with the inverse thiospinel CoNi2S4 (i.e.,
Ni
2+
[Co
3+
Ni
3+
]S
2–
4) having only been studied in a handful of cases as an electrocatalyst for oxygen
evolution and hydrogen evolution and as a catalyst for hydrodesulfurization.
1–3,12

Previous synthetic efforts to generate CoNi2S4 particles have resulted in quasi-amorphous
or ill-defined CoNi2S4 materials on the scale of several tens to hundreds of nm or greater.
1–3,5–7,13

The synthesis of small, colloidal CoNi2S4 nanocrystals, thereby maximizing their surface area-to-
volume ratios, may lead to performance improvements.
1,2
Unfortunately, the synthesis of small,
well-defined colloidal nanocrystals of compositionally complex materials can be difficult as a
result of the large number of synthetic variables that must be controlled in the process to maintain
39
phase purity. Precursor reactivities, precursor ratios, solvent type, capping ligand(s),
concentration, time, temperature, thermal ramping rate, and the interactions between all these
variables are only a few of the common factors that can affect nanocrystal syntheses, with most
process-morphology relationships being non-trivial. Consequently, syntheses are developed
empirically via fragmented knowledge of the underlying sequences of reaction variables.
14
For
these reasons, the nanofabrication of colloidal nanocrystals to give an optimal outcome can be
extremely tedious and laborious.
The development of optimal nanocrystal synthesis, as with all synthetic optimizations, are
traditionally done using the one variable at a time (OVAT) method, where only one variable is
changed at a time while all the variables are held constant to judge their effects. In an experimental
domain, where n synthetic variables create an n-dimensional design space, this one-dimensional
approach, in addition to its time and labor intensity, is incapable of revealing potential higher-
order interactions between the experimental variables (factors) and their effects on specific
nanocrystal characteristics (responses). The inability to map a fuller picture of the synthetic
parameter space makes product optimization and control difficult. For example, we invested years
of research in developing compositional control over binary nickel sulfide nanocrystals using the
OVAT approach, where the key synthetic handle ended up being the interaction between the sulfur
precursor and the surface ligand.
15
Additionally, Brock and coworkers studied the effects of
synthetic levers on the synthesis of binary nickel phosphide nanocrystals, which also illustrates the
extreme synthetic complexity of binary systems with rich phase diagrams, as intricate webs of
numerous reaction pathways were required in order to controllably obtain the desired products.
16,17

One powerful and emerging tool in materials science is the use of data-driven learning for
the synthesis and discovery of new materials.
14,18
Eliminating the reliance upon serendipity for
40
materials discovery and optimization is now possible via data analysis, which can be accomplished
with statistics-based multivariate experimental design techniques. These multivariate methods,
such as statistical design of experiments (DoE), have the ability to screen and dissect experimental
data in a way that not only allows for control and prediction of specific responses in material
synthesis, but also a better understanding of how the synthetic variables affect those responses,
and each other.
19,20
The first stage of statistical DoE involves a simultaneous variation in
magnitude of specific experimental factors that are suspected to influence the desired responses
and a screening of those factors in order to discover which are truly significant in the synthesis and
to what extent.
15
The nature of these first-order designs (e.g., factorial designs, fractional-factorial
designs, Plackett-Burman designs, orthogonal arrays, random sampling
21
) allows for valuable
information to be extracted from a fraction of the experiments that would be necessary using
OVAT methods, leading to significant time and resources saved while providing a deeper
understanding. Once the determinant variables have been identified, response optimization via
second-order designs is possible. Response surface methodology (RSM) is among the most
relevant second-order multivariate techniques used during optimization of a process and
corresponds to a collection of mathematical and statistical techniques based on the fit of a
polynomial equation to the experimental data with the objective of making statistical
predictions.
22,23

Despite the numerous advantages that multivariate experimental design offers for materials
synthesis and optimization, it is not yet widely employed. Of the few examples in the literature
where DoE been successfully applied to the optimization of colloidal nanocrystal syntheses have
mainly focused on unary or binary systems and the responses have been individually optimized
without an overall optimization of the system across all responses.
23–27
Herein, we implement a
41
DoE fractional-factorial screening design in conjunction with a multi-objective RSM Doehlert
optimization for the synthesis of ternary CoNi2S4 nanocrystals. Simultaneous control is exerted
over nanocrystal size, size distribution, and isolated yield to optimize the nanocrystal synthesis.

2.3. Results and Discussion
2.3.1. General Synthesis and Defining the Reaction Parameter Space
The air-free reaction between stoichiometric amounts of Co(acac)2 and Ni(acac)2 in
oleylamine with dodecanethiol (DDT, as the sulfur precursor) hot injected at 180 ˚C yields small,
colloidal ternary nanocrystals after 2 h with an average isolated yield of 70%. Powder XRD of this
unoptimized product returns diffraction peaks that are qualitatively similar to, but shifted
somewhat from, the expected CoNi2S4 thiospinel structure. Rietveld refinement of this diffraction
pattern provided the best fit to a closely related (Co,Ni)9S8 structure (Fm3
,
m),
28,29
which possesses
a nearly close-packed anion sub-lattice, and an experimentally determined lattice parameter of a =
9.759(6) Å (Figure 2.1a). Scherrer peak broadening of the powder XRD pattern is consistent with
a nanocrystalline product possessing an estimated grain size of 8.3 nm. TEM analysis of the
resulting nanocrystals revealed a fairly polydisperse ensemble with an average nanocrystal
diameter of 9.9 ± 2.7 nm (𝜎/𝑑
̅ = 28%, Figure 2.2a), and the particles adopt an irregular
morphology (Figure 2.2b). High-resolution TEM images of individual nanocrystals reveal the
presence of lattice fringes corresponding to the {400} and {220} lattice planes of the cubic
(Co,Ni)9S8 structure (Figure 2.1b). The average d-spacings of the lattice fringes are 0.24 and 0.33
nm, respectively. Analysis of ensemble nanocrystal samples by scanning electron microscope
energy-dispersive X-ray spectroscopy (SEM-EDX) returned an average composition of
Co1.3Ni2.3S3.4.
42
Figure 2.1. (a) Powder XRD pattern of unoptimized nanocrystals, with results from a Rietveld
refinement to the Fm3m (Co,Ni)9S8 structure. Tick marks represent individual reflections of the
(Co,Ni)9S8 structure with the difference pattern shown below. 𝜆 = 1.5406 Å. (b) HR-TEM image
of nanocrystals produced by the unoptimized synthesis, showing the measured lattice fringes.

Figure 2.2. (a) Size histogram of the corresponding nanocrystals from the unoptimized synthesis
(N = 303). (b) TEM images of the resulting unoptimized nanocrystals.

43
Based on these initial results, a Co(acac)2:Ni(acac)2:dodecanethiol ratio of 1:2:4 was used
as the base reaction and starting point for the experimental design and optimization. Under these
conditions, nucleation of the metal precursors takes place around 155 ˚C, indicated by a color
change from colorless to black, and injection of the sulfur precursor between 170-190 ˚C produced
what appears to be a single-phase product after sulfidation and aging at that temperature for ca. 1
h. During the preliminary reactions, hot injection of dodecanethiol below 170 ˚C yielded an ill-
defined, amorphous product and hot injection of dodecanethiol above 190 ˚C yielded a mixture of
resolvable phases by XRD (i.e., Co9S8, Ni3S4, and the desired CoNi2S4 phase) (Figure 2.3). As a
result, temperature was chosen as one of the five screening factors, with the aforementioned
temperatures set as the lower and upper-boundary levels when the viable synthetic parameter space
was designed for investigation via DoE. The other four factors were similarly chosen.
Concentration (or volume of oleylamine) was also investigated, as high concentrations appeared
to play an important role in the formation of an ill-defined and amorphous product at low
temperatures. It was found that reactions at low temperatures in < 4 mL of oleylamine did not yield
nanocrystals, so an oleylamine volume of 4 mL was set as the lower bound. Nanocrystals produced
in solutions of > 10 mL of oleylamine were nearly impossible to isolate by standard work-up, so
10 mL of oleylamine was set as the upper bound. Time was chosen because the desired product
required adequate aging in order give a crystalline single-phase ternary product. Since the desired
product was not formed before 1 h, with XRD analysis showing amorphous products and no
defined diffraction peaks until 30 min, and then only binary Ni3S4 until 50 min, that time was set
as the lower bound (Figure 2.3). The upper bound for time was set to 5 h, as significant
morphological changes were not observed past 3 h by TEM. The non-stoichiometric adjustments
of all precursor ratios (i.e., Co:Ni, Co:DDT) were chosen as the last two variables to screen, as
44
precursor ratio is commonly a key factor in nanocrystal synthesis.
15–17,23
A Co:DDT ratio of 1:2
was chosen as the lower bound as it was observed that sulfur did not incorporate into the crystal
structure at lower amounts. A ratio of 1:16 was chosen as the upper bound as larger volumes of
would have a significant effect on the reaction concentration. The lower bound of Co:Ni precursor
ratio was set to 1:1 since the theoretical yield would be significantly reduced at lower values. The
upper bound of the Co:Ni precursor ratio was set to 1:3, as binary Ni3S4 was obtained as a by-
product with greater amounts of Ni(acac)2. Within these experimental bounds, all reaction products
were confirmed as being qualitatively single-phase ternary products via XRD analysis (Figure
2.4). Size, size distribution (polydispersity), and isolated particle yield were chosen as the
responses to be optimized in the experimental design. A summary of these reaction parameters is
given in Table 2.1 along with their corresponding coded values, which are utilized for simplified
implementation into the analytical software, detailed in the Methods section.

Figure 2.3. (a) XRD analysis of the boundary reaction performed at 190 ˚C, showing a mixture of
binary and ternary phases present in the resultant nanocrystals. (b) Phase evolution of the
nanocrystals at short times (all other variables held constant at the base level).

45
Figure 2.4. XRD analysis of the boundary reactions for (a) low temperature condition (170 ˚C),
(b) high temperature condition (190 ˚C), (c) short time condition (1 h), (d) long time condition
(5 h), (e) low Co:Ni precursor ratio (1:1), (f) high Co:Ni precursor ratio (1:3), (g) low Co:DDT
ratio (1:2), (h) high Co:DDT (1:16), (i) low volume of oleylamine (4 mL), and (j) high volume of
oleylamine (10 mL).

Table 2.1. Real and coded values of the reaction parameters for the factors under investigation in
the screening and optimization designs.
Variable Low level High level Coded low value Coded high value
Co:Ni precursor ratio 1:1 1:3 -1 +1
Co:DDT ratio 1:2 1:16 -1 +1
Temperature of hot injection 170 ˚C 190 ˚C -1 +1
Reaction time 1 h 5 h -1 +1
Volume of oleylamine 4 mL 10 mL -1 +1

2.3.2. Design of Experiments (DoE)
When first designing a multivariable screening experiment, it is important to decide which
design is best for the system under study to screen out insignificant variables accurately and
efficiently in the synthesis. Among the most common designs are factorial designs. A full factorial
design consists of

N
k
possible experimental combinations of k number of factors that are
investigated at N number of levels per factor. In this case, for example, we are investigating five
factors, at two different levels per factor (high and low, coded +1 and -1, respectively), and so the
full factorial would consist of 2
5
(or 32) experiments. The number of experiments can be easily
46
reduced even further to a fraction of the full factorial while retaining a resolution required to attain
the desired information by employing a fractional factorial design. Two-level fractional factorial
first-order designs are standard for screening designs since the main objective is to solely eliminate
inert factors to minimize the number of experiments required for the optimization phase.
Employing the minimum number of levels (only the two extremes) increases the efficiency and
robustness of this phase of DoE. These designs are effective while maintaining widespread
generalizability. Herein, we employ a 2
5-2
fractional factorial for this data set, as described in the
Methods section. In general, a fractional factorial is denoted by N
k-p
, where p is dictated by the
desired fraction of the full factorial being used. For example, when N = 2 and k = 5, a 2
5-p
design
is a O
'
(
P
5
fraction of the 2
5
factorial and will require 2
5-p
experiments. We concluded that the most
efficient design for our data was to use a ¼ fraction of the full factorial (p = 2) with 8 experiments.
The size of the fraction employed in the design is generally chosen based on the generating
relations for the data to minimize the number of experiments while maintaining sufficient
resolution.
A more detailed explanation of the creation of the fractional factorial design that we
employed in this study can be found in Table 2.2. The full factorial, which consists of every
possible experimental combination (five variables investigated at two levels, 2
5
= 32 experimental
combinations), is highlighted in blue in Table 2.2. The generating arrangement for the four blocks
of eight is B1 = ABD and B2 = ACE, which are highlighted in purple in Table 2.2 and represent
ternary interactions between the indicated variables. The generation of blocks indicates that total
error and batch variability are neutralized by strategically confounding the variability linked to
degrees of freedom with high-order ternary interactions that are likely insignificant. The blocks
47
used to determine the experimental runs for the screening study correspond to the (+,+) and (−,+)
blocks, which are highlighted in green and pink, respectively, in Table 2.2.

Table 2.2. The fractional factorial screening design based on guidelines outlined in Chapter 1.
BLOCK
a
A B C D E ABD ACE BCDE
− − − − − − − +
1 + − − − − + + +
− + − − − + − −
2 + + − − − − + −
2 − − + − − − + −
+ − + − − + − −
1 − + + − − + + +
+ + + − − − − +
− − − + − + − −
+ − − + − − + −
− + − + − − − +
1 + + − + − + + +
1 − − + + − + + +
+ − + + − − − +
2 − + + + − − + −
+ + + + − + − −
2 − − − − + − + −
+ − − − + + − −
1 − + − − + + + +
+ + − − + − − +
− − + − + − − +
1 + − + − + + + +
− + + − + + − −
2 + + + − + − + −
1 − − − + + + + +
+ − − + + − − +
2 − + − + + − + −
+ + − + + + − −
− − + + + + − −
2 + − + + + − + −
− + + + + − − +
1 + + + + + + + +
a
A = Co:Ni precursor ratio, B = Co:DDT ratio, C = temperature, D = time, E = volume of
oleylamine.

Block 1, or the (+,+) block, corresponds to the first quarter fraction of the full factorial,
where the defining relation is I = ABD = ACE = BCDE, which has a resolution of III and a
48
projectivity of three. This means that if you remove any two variables you obtain a full 23 factorial
in the remaining three factors. The 3-D design projects a 23 factorial in all five subspaces of three
dimensions; thus, if only three or fewer factors are active for a particular response, we will have a
complete factorial design for whichever these active factors happen to be. The resolution of III
corresponds to the main and binary interactions being aliased with binary and ternary interaction
effects.
Block 2 corresponds to the single column fold-over, which negates the entire D-column
and therefore negates ABD in the defining relation. When the information from the two blocks is
combined, the resulting experimental design has a new defining relation of I = ACE as the effects
of the D (time) column are unconfounded entirely. Main and binary effects are now aliased with a
higher-level of interaction effects.

Table 2.3. Screening Design and Corresponding Experimental Responses for the Five Factors.
a
RUN BLOCK Co:Ni Co:DDT Temperature Time
Volume of
oleylamine
Size via
TEM Polydispersity Isolated yield
(nm) s/𝑑
̅ (%) (%)
1a 1a -1 1 -1 -1 1 5.8 18 48
2a 1a 1 1 1 1 1 5.7 34 85
3a 1a -1 -1 -1 1 1 5.6 21 83
4a 1a 1 -1 1 -1 1 10.1 19 100
5a 1a 1 -1 -1 -1 -1 7.5 42 71
6a 1a -1 1 1 -1 -1 4.0 17 81
7a 1a -1 -1 1 1 -1 5.1 37 88
8a 1a 1 1 -1 1 -1 6.4 25 69
3b 1b -1 -1 -1 1 1 5.5 18 72
8b 1b 1 1 -1 1 -1 6.4 18 50
1c 2a -1 1 -1 1 1 4.8 17 77
2c 2a 1 1 1 -1 1 5.8 26 94
3c 2a -1 -1 -1 -1 1 4.9 19 47
4c 2a 1 -1 1 1 1 10.7 21 100
5c 2a 1 -1 -1 1 -1 12.4 23 90
6c 2a -1 1 1 1 -1 5.0 11 96
7c 2a -1 -1 1 -1 -1 4.7 14 89
8c 2a 1 1 -1 -1 -1 7.3 26 35
3d 2b -1 -1 -1 -1 1 5.3 21 43
5d 2b 1 -1 -1 1 -1 10.4 21 47
6d 2b -1 1 1 1 -1 5.0 16 96
a
Rows shaded in blue are replicates for statistical significance.
49

The screening design depicting each experimental run and the corresponding responses is
given in Table 2.3 (the corresponding real values for each experiment can be referenced in Table
2.1). Since the minimum number of levels are used, the results of the screening design after
analysis of the experimental data yields a first-order equation through a multiple linear regression
that depicts the linear influence of the variables on the targeted responses. The linear equation
allows the significance of the factors to be assessed by analyzing the magnitude of influence each
effect has on the corresponding response. A standardized graphical representation of this analysis,
called a Pareto chart, can be used to determine which factors are statistically relevant to each of
the responses (Figure 2.5). Pareto charts are bar charts where the length of each bar is proportional
to the value of a t-statistic calculated for the corresponding effect, and any bars beyond the vertical
error line represent statistically significant factors at a determined significance level (5%). A main
effects chart is also plotted in Figure 2.5, which is a direct depiction of the linear influence of each
of the factors from the low level to the high level, which helps to visualize the directionality of
each effect. Analysis of these data revealed the significant factors for each response in the order
of their relevancy. When interpreting the Pareto charts, the color of each bar corresponds to the
directionality of each factor’s effect from low to high, so it represents the sign of the slope in the
linear regression for that effect. For example, if the bar is green, the value of the response increases
from the low level to the high level. Considering the optimizing goal for each response, we can
then draw meaningful conclusions as to how each factor is affecting the outcome. For size, the
significant factors were Co:Ni precursor ratio, Co:DDT ratio, and the interaction between Co:Ni
and Co:DDT. Thus, the real values that correspond to a Co:Ni precursor ratio of 1:1, a Co:DDT
ratio of 1:16, and the interaction between those two factors at those levels should produce a small
50
nanocrystal size. To test the robustness of this screening design, we analyzed the sizes of the
nanocrystal products across the most significant factor, which is Co:Ni precursor ratio.

Figure 2.5. Statistical plots for the first order screening of nanocrystal size: (a) Pareto chart, where
the vertical blue line on each graph represents α = 0.05. (b) Main effects plot, (c) linear
regression showing the normal probability, and (d) interaction plot.

Figure 2.6 shows the size dependence that can be qualitatively observed via adjusting the
Co:Ni precursor ratio, verifying its effect as the most significant factor for this response and
showing that nanocrystal diameter can be controlled over a range of mean sizes, from 5 to 11 nm,
under this set of conditions. For size distribution (polydispersity), the only significant factor was
Co:Ni precursor ratio, in which the size distribution should decrease when Co:Ni precursor ratio
is at a low level (Figure 2.7). Although technically insignificant for polydispersity in the screening,
it is interesting to note that polydispersity decreases at the high level of Co:DDT ratio (i.e., 1:16).
This is consistent with the previous observation that lower metal-to-chalcogen ratios have been
shown to lead to slower growth and more monodisperse nanocrystal ensembles.
30
For isolated
51
yield, the significant factor was temperature, with higher temperatures producing a higher isolated
yield (Figure 2.8). Higher temperatures drive precursor conversion, aiding in dodecanethiol
conversion to the reactive sulfur species in addition to facilitating the observed nucleation of
nanoparticle seeds prior to hot injection of the sulfur precursor (vide supra).

Figure 2.6. TEM images demonstrating the effect of the primary factor, Co:Ni precursor ratio, on
the size of the resulting nanocrystals.

52

Figure 2.7. Statistical plots for the first-order screening of nanocrystal size distribution: (a) Pareto
chart, where the vertical blue line on each graph represents α = 0.05. (b) Main effects plot, (c)
linear regression showing the normal probability, and (d) interaction plot.

Figure 2.8. Statistical plots for the first-order screening of nanocrystal isolated yield: (a) Pareto
chart, where the vertical blue line on each graph represents α = 0.05. (b) Main effects plot, (c)
linear regression showing the normal probability, and (d) interaction plot.
53

To perform the optimization using a minimal number of experiments, the factors deemed
statistically insignificant from the first-order design were eliminated. In turn, the factors that were
deemed significant for each response were identified, and to what extent they are significant. The
extent of significance is specifically important in this step because optimization designs are second
order and employ multiple levels per factor to allow for the best fitting of a polynomial function,
which in this case is quadratic. Since the Co:Ni precursor ratio was the most significant factor for
both size and polydispersity (Figures 2.5 and 2.7), that factor was investigated to the greatest
extent (seven levels). The Co:DDT ratio was the second most significant factor for size (Figure
2.5), so that factor was investigated to the second greatest extent (five levels). The hot-injection
temperature was the most significant factor affecting the isolated yield (Figure 2.8) but was not
significant for either of the other two responses, so it was included in the optimization but at the
fewest number of levels (three). Reaction time was not technically a significant factor in any of
the responses, so it was set to a constant value for the optimization. Since it played essentially no
role in the response of polydispersity, and longer reactions times had a positive effect on increasing
yield, but shorter times had a positive effect on minimizing size, we fixed time at the intermediate
base level (coded as 0), which was 2 h. Concentration also was not statistically significant in
affecting any of the three responses but based on the main effects plots it was deemed slightly
more beneficial when a larger volume of oleylamine was used (i.e., lower reaction concentrations),
so the volume of oleylamine used in the synthesis was fixed at 7 mL, a midpoint between the base
level (5 mL) and the high level (10 mL). This made phase two of our DoE a 3-factor optimization
for three responses, with Co:Ni precursor ratio having the highest priority, followed by the
Co:DDT ratio, and then the hot-injection temperature.
54

Figure 2.9. (a) Prediction variance plot, illustrating the standard deviation of the optimization
model predictions throughout the entire parameter space. (b) The design points of the Doehlert
optimization matrix in the reaction parameter space.

2.3.3. Optimization via RSM
A 3-factor optimization of the Co:Ni precursor ratio, the Co:DDT ratio, and the hot-
injection temperature was performed for the three target responses (i.e., nanocrystal size, size
distribution, and isolated yield). As with the screening phase, it is important to choose the correct
optimization design for the data to produce the most efficient and accurate results.
31
The uniform
shell design (tetradecahedron shape in 3D, Figure 2.9b) proposed by Doehlert in 1970 is one of
the most effective and versatile designs, normally requiring fewer experiments than traditional
designs while being equally, or more, effective. The design shape allows some factors to be studied
more in depth than others, since each factor is allowed a distinct number of levels.
32-34
The coded
Doehlert matrix can be found in Table 2.4. The matrix translated into the real experimental values
and the subsequent experimental responses are detailed in Table 2.5.

55

Table 2.4. Coded Doehlert optimization matrix for three variables.
Co:Ni ratio Co:DDT ratio Temperature
0.000 0.000 0.000
0.000 1.000 0.000
0.866 0.500 0.000
0.289 0.500 0.817
0.000 -1.000 0.000
-0.866 -0.500 0.000
-0.289 -0.500 -0.0817
-0.866 0.500 0.000
-0.289 0.500 -0.817
0.866 -0.500 0.000
0.577 0.000 -0.817
0.289 -0.500 0.817
-0.577 0.000 0.817

Table 2.5. Optimization design and corresponding responses.

Co:Ni Co:DDT Temperature Size Polydispersity Yield
(˚C) (nm) s/𝑑
̅ (%) (%)
1:2.0 1:4 180 6.2 22 51
1:2.0 1:16 180 5.9 20 66
1:2.9 1:10 180 7.9 31 100
1:2.3 1:10 188 6.9 26 76
1:2.0 1:2 180 9.9 28 76
1:1.1 1:3 172 6.0 19 78
1:1.7 1:3 180 7.1 20 58
1:1.1 1:10 180 5.0 16 78
1:1.7 1:10 172 7.4 23 42
1:2.9 1:3 180 13.9 22 74
1:2.6 1:4 172 8.4 21 80
1:2.3 1:3 188 6.2 27 91
1:1.4 1:4 188 6.9 31 100
1:2.3 1:10 188 6.0 18 79
1:1.4 1:4 188 6.7 31 76
1:2.0 1:4 180 6.5 26 85
1:2.0 1:4 180 6.2 28 70
1:2.0 1:4 180 6.4 28 75
a
Rows shaded in blue are replicates for statistical significance.

56
All levels span between the boundary parameters that were initially set for each factor. As
discussed in the Methods section, the initial single response optimization analysis fits the data
from the 13 experiments and 4 replicates from the Doehlert matrix to a polynomial function via a
multiple regression technique called the method of least squares to form three individual functions
for each response that contain up to the quadratic interaction terms while minimizing the residual
error (eq. 2.1).
𝑦 = 𝛽
1
+∑ 𝛽
𝔦
𝑥
𝔦
+∑ 𝛽
𝔦𝔦
𝑥
𝔦
(
+∑ 𝛽
!7
4
'8!87
𝑥
!
𝑥
7
+
4
!0'
𝜀
4
!0'
(2.1)

where 𝛽0 = mean, 𝛽I = coefficients of the linear parameters 𝑖, 𝛽ij = coefficients of the interaction
parameters 𝑖𝑗, 𝛽ii = coefficients of the quadratic parameters 𝑖𝑖, 𝑥𝑖 = independent variables, and 𝜀
= residuals.
These functions model the synthetic system to predict the response values based on the
values of the parameters, which allows each response to be optimized based on the inputted targets.
The resulting polynomial equations corresponding to the coded values for size, size distribution,
and isolated yield are as follows (eqs. 2.2-4):

𝑆𝑖𝑧𝑒 (𝑛𝑚) = 6.19 − 1.86625 × 𝐶𝑜:𝐷𝐷𝑇 + 2.30658 × 𝐶𝑜:𝑁𝑖 − 0.662916 ×
𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 1.69 × 𝐶𝑜:𝐷𝐷𝑇
(
− 2.90416 × 𝐶𝑜:𝐷𝐷𝑇 × 𝐶𝑜:𝑁𝑖 + 0.733539 ×
𝐶𝑜:𝐷𝐷𝑇 × 𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 2.12679 × 𝐶𝑜:𝑁𝑖
(
− 1.69544 × 𝐶𝑜:𝑁𝑖 ×
𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 0.389795 × 𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒
(
(2.2)

57
𝑆𝑖𝑧𝑒 𝐷𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 (%) = 21.6667 − 1.8625×𝐶𝑜:𝐷𝐷𝑇 + 2.33834×𝐶𝑜:𝑁𝑖 +
3.21202×𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 1.88333×𝐶𝑜:𝐷𝐷𝑇
(
+ 7.21709×𝐶𝑜:𝐷𝐷𝑇×𝐶𝑜:𝑁𝑖 −
9.89687×𝐶𝑜:𝐷𝐷𝑇×𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 − 0.150009×𝐶𝑜:𝑁𝑖
(
− 8.17395×𝐶𝑜:𝑁𝑖×
𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 2.55435×𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒
(
(2.3)

𝐼𝑠𝑜𝑙𝑎𝑡𝑒𝑑 𝑌𝑖𝑒𝑙𝑑 (%) = 53.3333 − 2.75×𝐶𝑜:𝐷𝐷𝑇 + 8.16562×𝐶𝑜:𝑁𝑖 + 15.384×
𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒+ 17.6667×𝐶𝑜:𝐷𝐷𝑇
(
+ 15.0115×𝐶𝑜:𝐷𝐷𝑇×𝐶𝑜:𝑁𝑖 − 2.8621×
𝐶𝑜:𝐷𝐷𝑇×𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 33.0019×𝐶𝑜:𝑁𝑖
(
− 16.1126×𝐶𝑜:𝑁𝑖×
𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 + 16.1771×𝑇𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒
(
(2.4)

Because all the variables are normalized through the coded values, the relative change of a
variable is directly related to the size of its regression coefficient. Therefore, the equation using
the coded values can be used to compare the absolute effect of the variables on the response of the
nanocrystals. This means that if the model parameters have a large absolute value, the
corresponding variable has a significant effect on the response. Looking at the graphical depiction
of the statistical analysis of eq. 2.2 in Figure 2.10, nanocrystal size had a statistically significant
quadratic dependence on Co:Ni precursor ratio and Co:DDT ratio to the 95% confidence interval.
Using the quadratic terms in eq. 2.2, the minimum nanocrystal size was predicted to be 5.2 nm
when the three parameters (Co:Ni, Co:DDT, temperature) are set to the coded values (-1,-0.016,-
1), respectively, which corresponds to real values of 1:2, 1:3.8 and 170 ˚C. The equation can be
plotted as response surface, which is a visual representation of the model in three dimensions as it
depicts the resulting nanocrystal size as a function of Co:Ni and Co:DDT precursor ratios, as
shown in Figure 2.11a. The temperature factor, which was not significant in the response
58
according to the statistical analysis (Figure 2.10), was fixed to its predicted optimum parameter
value (-1) for visual simplicity.

Figure 2.10. Statistical plots for the second-order optimization of nanocrystal size: (a) Pareto
chart, (b) main effects plot, (c) linear regression showing the normal probability, and (d) interaction
plot.

Figure 2.11. (a) Response surface for nanocrystal size as a function of Co:Ni precursor ratio and
Co:DDT ratio, with the minimum size predicted to be 5.2 nm. (b) TEM image of the CoNi2S4
nanocrystals prepared using the reaction parameters at the predicted minimum. (c) Corresponding
size histogram of the nanocrystals where s/𝑑
̅ = 18% (N = 320).
59

The accuracy of the mathematical fit of the model can be carefully analyzed via analysis
of variance (ANOVA), and replicates of the center point in the design allow for a variance estimate
of each polynomial’s vector value that contains the parameters of the model (Table 2.6-2.8).
Extracting the square root of each component of the vector value gives the standard errors for the
coefficients (regression error and residual error). The sum of squares data allows the residual error
value to be further split into pure error and lack of fit. Based on the evaluation of the fitted model
for size, shown below Table 2.6, the R-squared statistic indicates that the model as fitted explains
80.4% of the variability in size. The adjusted R-squared statistic, which is more suitable for
comparing models with different numbers of independent variables, is 51.0%. The standard error
of the estimate shows the standard deviation of the residuals to be 1.48174. The mean absolute
error (MAE) of 0.650 is the average value of the residuals. The Durbin-Watson (DW) statistic tests
the residuals to determine if there is any significant correlation based on the order in which they
occur in the data file. Since the P-value is greater than 5.0%, there is no indication of serial
autocorrelation in the residuals at the 5.0% significance level. This makes sense, as prediction
variance increases slightly around the edges of the parameter space due to the spherical nature of
the optimization design, depicted in Figure 2.9. This, along with the natural error introduced by
manual measurements of the nanocrystal size and size distribution, likely explains any slight
discrepancies between the predicted optima and the experimental optima, albeit all experimental
results fell well within a single standard deviation of the predicted optima, and therefore are all
statistically significant. To further verify the model, three replicate reactions at the predicted
optimum were performed. For size, this corresponds to a Co:Ni precursor ratio of 1:1, a Co:DDT
ratio of 1:3.8, and a temperature of 170 ˚C with coded values of (-1,-0.016,-1), as previously
60
mentioned. The coordinates of this reaction in the response surface are indicated by a white square
in Figure 2.11a. The average size of the resulting nanocrystals formed from these reaction
parameters was 5.2 ± 1.0 nm. This is in excellent agreement with the predicted minimum of 5.2
nm and represents a significant size minimization from the unoptimized synthesis (i.e., 9.9 nm). A
TEM image, and corresponding size histogram, of the CoNi2S4 nanocrystals produced under these
conditions is given in Figure 2.11.

Table 2.6. Analysis of variance for the second order optimization of size.
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A:Co:DDT ratio 13.932 1 13.932 6.35 0.0453
B:Co:Ni ratio 22.344 1 22.344 10.18 0.0188
C:Temperature 1.945 1 1.945 0.89 0.383
AA 2.760 1 2.760 1.26 0.305
AB 6.325 1 6.325 2.88 0.141
AC 0.323 1 0.323 0.15 0.715
BB 4.579 1 4.579 2.09 0.199
BC 2.084 1 2.084 0.95 0.368
CC 0.0659 1 0.0659 0.03 0.868
Total error 13.173 6 2.196
Total (corr.) 67.136 15
R-squared = 80.378%
R-squared (adjusted for d.f.) = 50.945%
Standard Error of Est. = 1.482
Mean absolute error = 0.650
Durbin-Watson statistic = 2.069 (P = 0.393)
Lag 1 residual autocorrelation = -0.0349

Similarly, when using eq. 2.3, the minimum size distribution that directly corresponds to
the polydispersity of the nanocrystals was predicted to be 𝜎/𝑑
̅ = 4% with coded reaction parameters
(Co:Ni, Co:DDT, temperature) equal to (-1,0.9,-1) with corresponding real values of 1:1, 1:14.8,
and 170 ˚C, respectively (Figure 2.12). The resulting average standard deviation about the mean
nanocrystal diameter from the three validation reactions performed under these conditions was s/𝑑
̅
= 12%, which is higher than the predicted value. This discrepancy is explained by the analysis of
the fit in Table 2.7 showing that 66% of the variability in size distribution could be explained by
61
the model. This is likely because the only factor affecting size distribution in the optimization was
the interaction between Co:DDT ratio and temperature, which was only significant to the 87%
confidence interval (Figure 2.13). This suggests that control over nanocrystal polydispersity is
more complex and must be achieved through higher order interaction effects of the experimental
parameters. Despite this, we did achieve a minimization of polydispersity, as the reactions
performed at the predicted optimum conditions produced the lowest achieved polydispersity of all
the reactions performed in this study and were a significant improvement over the initial conditions
of the representative unoptimized synthesis (where 𝜎/𝑑
̅ = 28%).

Figure 2.12. (a) Response surface for the individual optimization of size distribution. (b) TEM
image of resulting nanocrystals. (c) Size distribution of resulting nanocrystals where 𝜎/𝑑
̅
= 11%
(N = 338).

Table 2.7. Analysis of variance for the second order optimization of polydispersity (%).
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A:Co:DDT ratio 13.876 1 13.876 0.63 0.456
B:Co:Ni ratio 22.964 1 22.964 1.05 0.345
C:Temperature 45.671 1 45.671 2.08 0.199
AA 17.787 1 17.787 0.81 0.402
AB 39.063 1 39.063 1.78 0.230
AC 58.828 1 58.828 2.69 0.152
BB 41.536 1 41.536 1.90 0.218
BC 48.397 1 48.397 2.21 0.188
CC 14.122 1 14.122 0.64 0.453
Total error 131.437 6 21.906
Total (corr.) 390.978 15
62
R-squared = 66.383%
R-squared (adjusted for d.f.) = 15.956%
Standard Error of Est. = 4.680
Mean absolute error = 2.35
Durbin-Watson statistic = 2.674 (P=0.853)
Lag 1 residual autocorrelation = -0.3426

Figure 2.13. Statistical plots for the second-order optimization of nanocrystal size distribution:
(a) Pareto chart, (b) main effects plot, (c) linear regression showing the normal probability, and
(d) interaction plot.

The model of the response for isolated yield (eq. 2.4) predicted a maximum isolated yield
of 100% when the reaction parameters (Co:Ni, Co:DDT, temperature) were set to (-1,-1,1) with
real values of 1:1, 1:2, and 190 ˚C, respectively, as seen in Figure 2.14. Experiments using these
synthetic parameters were performed to again verify the prediction, and the resulting average
isolated yield of those reactions was 99%, which is in excellent agreement with the predictions of
the model, which as fitted explains 72% of the variability in yield after analysis of Table 2.8 and
63
Figure 2.15. Again, this represents a significant improvement over the initial conditions of the
representative unoptimized synthesis (with an isolated yield of only 70%).

Figure 2.14. Response surface for the individual optimization of isolated yield.

Table 2.8. Analysis of variance for the second order optimization of isolated yield (%).
Source Sum of Squares Df Mean Square F-Ratio P-Value
A:Co:DDT ratio 30.25 1 30.25 0.21 0.666
B:Co:Ni ratio 280.029 1 280.029 1.90 0.217
C:Temperature 1047.67 1 1047.67 7.13 0.037
AA 38.533 1 38.533 0.26 0.627
AB 169.0 1 169.0 1.15 0.325
AC 4.920 1 4.920 0.03 0.861
BB 112.133 1 112.133 0.76 0.416
BC 187.826 1 187.826 1.28 0.302
CC 71.305 1 71.305 0.49 0.512
Total error 881.988 6 146.998
Total (corr.) 3103.0 15
R-squared = 71.576%
R-squared (adjusted for d.f.) = 28.941%
Standard Error of Est. = 12.124
Mean absolute error = 6.519
Durbin-Watson statistic = 2.573 (P = 0.793)
Lag 1 residual autocorrelation = -0.313

64

Figure 2.15. Statistical plots for the second-order optimization of nanocrystal isolated yield:
(a) Pareto chart, (b) main effects plot, (c) linear regression showing the normal probability, and
(d) interaction plot.

Once the single response optimizations were performed, the model equations were jointly
optimized to find the overall optimum for all three desired responses across the parameter space
of the factors studied in this reaction. This was done, as described in the Methods section, by
quantifying the desirability of the predictions of each of the individual polynomial models, and
subsequently optimizing that desirability. A desirability of 100% is observed if all three of the
response values predicted from the joint optimization are equal to their individual optimized
predicted response values (i.e., size = 5.2 nm, s/𝑑
̅ = 4%, isolated yield = 100%). It is important to
note that the experimental parameters predicted to give the optimum responses for each individual
response will most likely not be the same as the parameters predicted in the multiple response
optimization, as the values of the parameters that give the smallest possible particle size are not
65
the same parameter values that are predicted to produce the highest isolated yield, for example. A
desirability function serves to provide the most optimized result while finding a compromise
amongst the individual optimal parameter values for all three responses. A graphical representation
of the desirability function across all three parameters is provided in Figure 2.16a, where the
reaction conditions at the optimum are indicated by a white square. The optimal desirability of all
three factors in this system is 90%, with the minimized nanocrystal size being 6.0 nm, the
minimized standard deviation about the mean nanocrystal diameter being s/𝑑
̅ = 14%, and the
maximized isolated yield being 95%. These responses were obtained via predicted coded (Co:Ni,
Co:DDT, temperature) values of (0.866,1.00,0.817), respectively, with corresponding real values
of 1:2.9, 1:16, and 188 ˚C, respectively. Three reactions were run in triplicate using these predicted
optimal experimental conditions to verify the model. A TEM image of the nanocrystals resulting
from a synthesis using the predicted optimal synthetic conditions is provided in Figure 2.16b, with
the corresponding size distribution given in Figure 2.16c. The responses of the verification
reactions were averaged together, with the average nanocrystal size being 5.9 ± 1.0 nm, the average
standard deviation about the mean nanocrystal diameter being 17%, and the average isolated yield
being 95%, all of which are well within a standard deviation of the model response predictions,
showing excellent agreement and predictive power. Rietveld refinement of powder XRD data was
used to confirm that the nanocrystals synthesized under this set of optimal conditions possessed
the desired Fd3
,
m thiospinel structure (a = 9.414(3) Å, Figure 2.16d), with an average composition
of CoNi2.7S3.3 as determined by SEM-EDX. Although the empirical formula is off-stoichiometry,
there is sufficient evidence in the literature demonstrating the presence of non-stoichiometry in
thiospinel structures.
2,4,35-49
In addition, high-resolution TEM images of the resulting nanocrystals
66
possess lattice fringes consistent with the {400} and {220} lattice planes of the CoNi2S4 thiospinel
structure (d = 0.24 and 0.33 nm, respectively, Figure 2.17).

Figure 2.16. (a) Desirability plot of the simultaneous optimization of all three parameters with the
general optimum conditions indicated in white. (b) TEM image of the CoNi2S4 nanocrystals
synthesized under the optimum conditions. (c) Corresponding size histogram of the resulting
nanocrystals, where s/𝑑
̅ = 14% (N = 1,300). (d) Powder XRD pattern of CoNi2S4 nanocrystals
synthesized under the optimum conditions, with results from a Rietveld refinement to the Fd3
,
m
structure. Tick marks represent individual reflections of the thiospinel structure with the difference
pattern shown below. l = 1.5406 Å.

67

Figure 2.17. (a) XRD pattern of the CoNi2S4 nanocrystals produced under the optimum conditions
predicted by RSM and (b) HR-TEM image of the corresponding particles, showing the measured
lattice fringes.

The optimum in Figure 2.16a lies at the high values of all three parameters, but there is
another optimal area in the 3D parameter space observable at the opposite corner of the parameter
space, corresponding to the low values of all of the variables (Co:Ni, Co:DDT, temperature).
Although this second optimal region does not return quite as high of a desirability (~80-90%), this
saddle point-like parameter space relates back to the individual optima of each response and their
different locations in the overall parameter space. With that being said, we can also discover the
factor settings that give nonoptimal results, with the smallest desirability lying around a Co:Ni
ratio of 1:1, a Co:DDT precursor ratio of 1:2, and a temperature of 190 ˚C. This information
provides further insight into the synthesis and a higher level of control, as it highlights the
combination of factors that produce the largest size (13 nm), largest polydispersity (35%), and
lowest isolated yield (38%) resulting from a single set of reaction parameters.
The optimum in Figure 2.16a lies near the edge of the defined parameter space explored
using the Doehlert optimization matrix. This indicates that these conditions may represent a local
optimum, and a better optimum may exist beyond the parameter space studied in the optimization,
as it was just a subset of the entire parameter space defined by the boundary reactions (Figure 2.9)
68
where the edges of the parameter space describe the full range of each variable that can be explored
before introducing multiple phases or nonviable products. As a result, we used the model function
to perform an extrapolation of the design space in search of an improved optimum. After
extrapolation, the new optimum lies at the coded parameter values (1.0,1.0,1.0), which corresponds
to a Co:Ni ratio of 1:3, a Co:DDT precursor ratio of 1:16, and a temperature of 190 ˚C, returning
a desirability of 96%. The new predicted response values give a minimum nanocrystal size of 6.1
nm, a minimum standard deviation about the mean nanocrystal diameter of 10%, and a maximum
isolated yield of 99%; these values represent an improvement over the prior optimum within the
defined parameter space. Nanocrystal syntheses were performed in triplicate under these new
conditions, and the results were compared to the predicted outcomes. The experimental responses
of the post-extrapolation verification reactions resulted in an average nanocrystal size of 6.1 ± 0.9
nm, an average standard deviation about the mean nanocrystal diameter of 15%, and an average
isolated yield of 98%, which are in excellent agreement with the predicted values (Figure 2.18).
Powder XRD was used to confirm that the nanocrystals synthesized under this set of optimal
conditions also possessed the desired thiospinel structure (Figure 2.19), with an average
composition of CoNi2.8S3.2, as determined by SEM-EDX. High-resolution TEM images of the
resulting nanocrystals displayed lattice fringes consistent with the {400} and {311} planes of the
CoNi2S4 thiospinel structure (d = 0.24 and 0.28 nm, Figure 2.19). Overall, these results validate
the predictions made by the model, proving that this method was successful in efficiently and
simultaneously optimizing the synthesis of phase-pure CoNi2S4 nanocrystals over the original
unoptimized reaction for the three nanocrystal responses of size, size distribution, and isolated
yield.

69

Figure 2.18. (a) TEM images CoNi2S4 nanocrystals synthesized under the conditions indicated at
the extrapolated optimum. (b) Size histogram of the resultant nanocrystals with 𝜎/𝑑
̅ = 15% (N =
639).

Figure 2.19. (a) Stacked XRD patterns of the CoNi2S4 nanocrystals produced under the optimum
and extrapolated optimum conditions predicted by RSM, in addition to the single optima
conditions of minimized size and polydispersity. These diffraction patterns match the expected
CoNi2S4 phase and are shifted from the (Co,Ni)9S8 nanocrystals that resulted from the unoptimized
reaction. (b,c) HR-TEM images of the nanocrystals resulting from the extrapolated optimum
reaction conditions, showing the measured lattice fringes. Twinning is observed in the HR-TEM.

70
Two randomly selected points in the reaction parameter space were then selected on the
response surface plot and performed in triplicate to monitor the prediction accuracy outside of the
DoE selected experiments. Point #1 corresponded to the coded values (0.49,-0.12,-0.63) with
respective real values of Co:Ni = 1:2.5, Co:DDT = 1:4.0, and a temperature of 174 ˚C (with a
desirability of 62%). The model predicted a size of 9.0 nm, a size distribution of s/𝑑
̅ = 26% and a
yield of 74%. After averaging the results of all three reactions performed at this parameter setting,
the experimental results corresponded to a size of 8.6 nm, a size distribution of s/𝑑
̅ = 25%, and an
isolated yield of 74%, which were in good agreement of the predicted results (i.e., all within one
standard deviation). Point #2 corresponded to the coded values (0.69,0.76,-0.46) with respective
real values of Co:Ni =1:2.7, Co:DDT = 1:13.1 and a temperature of 175 ˚C (a desirability of 61%).
The model predicted a size of 7.5 nm, a size distribution of s/𝑑
̅ = 30%, and an isolated yield of
88%. After averaging the results of all three reactions performed at this parameter setting, the
experimental results corresponded to a size of 7.4 nm, a size distribution of s/𝑑
̅ = 30%, and an
isolated yield of 81%, which were in good agreement of the predicted results (again, all within one
standard deviation). It is important to note that the excellent agreement of the size distribution is
augmented at larger distributions, as the human error in the manual counting method is masked
much more readily than in very small size distributions, such as those predicted at the optimum
condition. Overall, the model is capable of predicting the responses throughout the entire
parameter space, and the power of the resolution is demonstrated by the accuracy at even very
slightly nuanced overall desirabilities, as seen in the two arbitrary points described above.

71
2.4. Methods
2.4.1. Materials and General Considerations
All chemicals and reagents were used as received without further purification. Cobalt(II)
acetylacetate (Co(acac)2, ³95%,), nickel(II) acetylacetate (Ni(acac)2, ³95%), oleylamine (³70%),
and 1-dodecanethiol (³95%) were purchased from Sigma Aldrich. Oleylamine and 1-dodecanthiol
were dried and degassed prior to use by heating to 120 ˚C under vacuum for 5 h. All nanocrystal
syntheses were performed under a flowing N2 atmosphere using standard Schlenk techniques.

2.4.2. Nanocrystal Synthesis
Ternary nanocrystals were synthesized in a three-neck 50 mL round bottom flask fitted
with a reflux condenser. The reaction flask was charged with Ni(acac)2 (0.50 g, 0.20 mmol) and
Co(acac)2 (0.50-1.50 g, 0.20-0.60 mmol). Degassed oleylamine was injected into the flask (4-10
mL), the reaction mixture was heated to 120 ˚C at 10 ˚C min
–1
under vacuum, and then it was held
under vacuum for 30 min. The reaction mixture was then placed under flowing N2 and temperature
was increased at the same rate to the desired reaction temperature (170-190 ˚C) at a constant ramp
rate of ~14 ˚C min
-1
. Once the final reaction temperature was reached, degassed 1-dodecanthiol
(0.125-1 mL) was hot-injected into the reaction flask and the reaction conditions were maintained
for the desired reaction time (1-5 h).
In all cases, the resulting reaction mixture was quenched by removing it from the heat
source and allowing it to naturally cool to room temperature upon completion. Once cooled, 3-8
mL of hexanes was added to the reaction flask in air followed by bath sonication for 5 min. The
reaction volume was then split equally into two 50 mL centrifuge tubes and the centrifuge tubes
were filled with ethanol for precipitation and washing. The nanocrystals were then separated from
72
the solution via centrifugation for 5 min. The supernatant was discarded, 5 mL of hexanes were
added to the remaining particles, and the solution was bath sonicated for 5 min for redispersion.
The nanocrystals were then washed a second time via the same procedure except this time with
the addition of 10 mL of ethanol and 35 mL of acetone. After redispersion, the nanocrystals were
washed the same way a third time with 45 mL of acetone. After washing 3 times, the nanocrystals
were re-dispersed in 10 mL of hexanes and sonicated for 10 min to form a stable colloidal
suspension. Suspensions of the resulting nanocrystals remained colloidally stable for several
months.

2.4.3. Characterization
Powder X-ray diffraction (XRD) analysis was performed from 10-70˚ 2q with a step size
of 0.7˚/min using a Rigaku Ultima IV diffractometer functioning at 44 mA and 40 kV. Cu Ka
radiation (l = 1.5406 Å) and a silicon zero-diffraction substrate were employed. Rietveld
refinements were performed using GSAS-II.
50
Due to the broad peak shapes within the powder
diffraction patterns of the nanocrystalline materials, the profile parameters U, V, W, X, and Y were
refined. Fixed points and 12-15 Chebyschev variables were used to fit background contributions.
The lattice parameters, atomic positions, thermal displacement parameters, and fractional
occupancies were also refined. Preferred orientation was accounted for through refinement of
fourth-order spherical harmonics, yielding texture indexes of 1.0-1.1. Transmission electron
microscopy (TEM) was performed at an operating voltage of 200 kV on a JEOL JEM-2100,
equipped with a Gatan Orius charge-coupled device camera. Samples for TEM were prepared by
drop-casting dispersions of the nanocrystals in hexanes onto carbon-coated copper grids (carbon
type-B, 200 mesh, Ted Pella). TEM micrographs were processed in ImageJ to analyze nanocrystal
73
size and polydispersity statistics. Average diameters (size) and standard deviations (size
distribution) were derived by manually measuring the longest axis of each particle. A minimum of
N = 300 individual particles were measured per sample and averaged over multiple images.
51
Yield
data was calculated via thermogravimetric analysis (TA Instruments TGA Q50) by measuring a
known amount of powder (minimum 5 mg) from each dried sample in an alumina crucible and
heating the sample to 100 ˚C at 10 ˚C min
–1
under flowing nitrogen, applying a 15 min isotherm,
then heating the sample at the same rate to 600 ˚C with an additional 15 min isotherm. The resulting
product was weighed to correct for the organic ligands in the nanocrystals. The total weight of the
pure nanocrystal product was then compared to the theoretical yield to determine the organic-
corrected isolated yield. Scanning electron microscopy-energy dispersive X-ray spectroscopy
(SEM-EDX) standardless quantification was used for elemental analysis and composition
identification on a JEOL JSM-7001F SEM with an operating voltage of 23 kV and a working
distance of 15 mm.

2.4.4. Design of Experiments (DoE)/Response Surface Methodology (RSM)
We implement DoE to model the synthetic parameter space via a quadratic equation. This
is done by tracking experimental inputs of the chosen factors and analyzing the resultant output
responses. By inputting this data into Statgraphics Centurion XVI software (Statistical Graphics,
Rockville, MD, USA), we were able to eliminate inert or insignificant factors and predict the
specific factors that minimize nanocrystal size and polydispersity, while maximizing the isolated
yield. The DoE process is performed in five steps:
1. Select which factors to test. This was done using prior knowledge of the synthesis of thiospinels
and nanocrystals.
22,23,52,53
Time, temperature, cobalt to nickel (Co:Ni) precursor ratio, cobalt to
74
sulfur, or 1-dodecanethiol, (Co:DDT) ratio, and concentration (volume of oleylamine) were the
five variables screened.
2. Define the reaction parameters. This represents what range of each variable yields the target
material, which was verified via XRD analysis and is detailed in Figure 2.4. Setting these
parameters determines the reaction parameter space under study;
20
therefore, this range should
be as wide as possible to maximize the viable exploration space and give the most accurate
depiction of the synthetic landscape. The reaction parameters of the evaluated factors and their
corresponding coded values are given in Table 2.1.
3. Define a first-order screening design. We implemented a 2
5-2
fractional factorial design for the
five factors, which is a two-level investigation of a quarter fraction of the full factorial with a
resolution of III (i.e., the generating relation for the design is equal to a ternary interaction), in
tandem with a single column fold-over of the time factor in order to unconfound its main and
binary effects.
20
The two-level fractional factorial, in conjunction with the single column fold
over, required a total of 16 experiments and 4 replicates for statistical significance, whereas the
full factorial would have required 32 experiments and 6 replicates. The blocking design of the
fractional factorial, which dictated the specific reactions that had to be performed, is shown in
terms of the coded values in Table 2.2. Detailed descriptions of the subsequent statistical
analysis of the fractional factorial, as well as all related figures can be found below:
3.1. Size
In the ANOVA assessing the linear model of size, the R-squared statistic indicates that the
model as fitted explains 93.3957% of the variability in nanocrystal size. The adjusted R-
squared statistic, which is more suitable for comparing models with different numbers of
independent variables, is 83.4892%. The standard error of the estimate shows the standard
75
deviation of the residuals to be 0.944655. The mean absolute error (MAE) of 0.455044 is
the average value of the residuals. The Durbin-Watson (DW) statistic tests the residuals to
determine if there is any significant correlation based on the order in which they occur in
the data file. Since the P-value is greater than 5.0%, there is no indication of serial
autocorrelation in the residuals at the 5.0% significance level.
3.2. Polydispersity
The ANOVA assessing the linear model of polydispersity showed that the R-squared
statistic indicates that the model as fitted explains 70.9713% of the variability in standard
deviation about the mean nanocrystal diameter. The adjusted R-squared statistic, which is
more suitable for comparing models with different numbers of independent variables, is
27.4284%. The standard error of the estimate shows the standard deviation of the residuals
to be 6.46485. The mean absolute error (MAE) of 3.0862 is the average value of the
residuals. The Durbin-Watson (DW) statistic tests the residuals to determine if there is any
significant correlation based on the order in which they occur in the data file. Since the P-
value is less than 5.0%, there is an indication of possible serial correlation at the 5.0%
significance level. The results are free of any possible serial correlations at a P-value
greater than 12.5%.
3.3. Yield
The ANOVA assessing the linear model of yield showed that the R-squared statistic
indicates that the model as fitted explains 83.925% of the variability in isolated yield. The
adjusted R-squared statistic, which is more suitable for comparing models with different
numbers of independent variables, is 59.8124%. The standard error of the estimate shows
the standard deviation of the residuals to be 13.3614. The mean absolute error (MAE) of
76
5.87679 is the average value of the residuals. The Durbin-Watson (DW) statistic tests the
residuals to determine if there is any significant correlation based on the order in which
they occur in the data file. Since the P-value is greater than 5.0%, there is no indication of
serial autocorrelation in the residuals at the 5.0% significance level.
4. Perform an optimization via second-order design. Once the inert, or insignificant factors, were
screened out, a surface response optimization of Co:Ni precursor ratio, Co:DDT ratio, and
temperature was performed through a second-order Doehlert design for three factors.
32-34
In this
optimization, the design consists of 13 reactions and two replicates of the central point for added
degrees of freedom and two other random points as a means to estimate the experimental variance
(Table 2.5). In this case, the total error has six degrees of freedom, while there are three degrees
of freedom for pure error. In general, at least three or four error degrees of freedom need to be
available when testing the statistical significance of estimated effects. The correlation matrix
shows the extent of the confounding amongst the effects. A perfectly orthogonal design will show
a diagonal matrix with 1s on the diagonal and 0s off the diagonal. Any non-zero terms off the
diagonal imply that the estimates of the effects corresponding to that row and column will be
correlated. In this case, there were 12 pairs of effects with non-zero correlations. However, since
none were greater than or equal to 0.5, the results can be interpreted without much difficulty. The
experimental results from the specified reactions were fitted to a model quadratic function for each
response (eqs. 2.2-4). This function predicts which values of the input parameters give optimal
target responses. The table illustrating the coded values for each reaction can be found in Table
2.4 and their corresponding real values are given in Table 2.5. For statistical calculations, the
experimental variables, Xi, have been coded as xi according to (eq. 2.5):
33

𝑥
!
=
"
!
#"
!
"
$"
!
× 𝛼 (2.5)
77
where 𝑋
!
%
is the real value at the center of the experimental domain (or the value used in the base
synthesis, coded as 0), Δ𝑋
!
is the step variation of the real value (or the difference between the
base value and its upper or lower limit real value counterparts coded +1 or -1, respectively) and a
is equal to the coded value limit for each variable, which in this case is 1.
33
The predicted optimal
parameters and their corresponding response values are given in the Results and Discussion.
5. Simultaneously optimize all responses to find the most optimal conditions for the desired product.
After each response was individually optimized, a multiple response optimization was performed.
This technique enables the simultaneous optimization of all three variables using a desirability
function 𝑑(𝛾) to quantify and jointly optimize each of the response’s quadratic model equations.
The desirability function expresses the desirability of a response value equal to 𝛾 on a scale of 0
to 1. This function takes one of three forms depending on whether the response is to be maximized,
minimized, or hit a specified target value. In this case, we only sought to maximize isolated yield
and minimize size and size distribution. If a response variable is to be maximized, the desirability
function is defined by eq. 2.6:
𝑑 =
⎩
⎪
⎨
⎪
⎧
0
O
9 :# ;<=
>!?> # ;<=
P
@
,
1

𝛾 ] < 𝑙𝑜𝑤
𝑙𝑜𝑤 ≤ 𝛾 X ≤ ℎ𝑖𝑔ℎ
𝛾 X > ℎ𝑖𝑔ℎ
(2.6)
where 𝛾 X is the predicted value of the response variable, low is the value below which the response
is completely unacceptable, and high is a value above which the desirability is at its maximum.
The parameter s defines the shape of the function. The function for minimization is the mirror
image of that for maximization, starting with 1 at the low value and going to 0 at the high value.
We then combine the desirability of the three responses by creating a single composite function
D. Since all of the responses are considered equally important, D is given by eq 2.7:
78
𝐷 = {𝑑
'
,𝑑
(
,𝑑
)
}
'/)
(2.7)
where a desirability of 100% corresponds to each of the predicted optima in the multiple response
optimization being identical to their individual optima. The desirability results were then modeled
visually using 3D contour plots.
Table 2.9 displays the calculated desirability of the responses at each run in the experiment.
Based on the observed responses, the most desirable results were obtained for run 8. Based on the
predicted responses from the fitted model, the most desirable results correspond to run 4.

Table 2.9. Desirability.
Observed Predicted Observed Predicted Observed Predicted
Run Size (nm) Size (nm) Polydispersity (%) Polydispersity (%) Yield (%) Yield (%)
1 6.44 6.363 27.4 27.4 75.0 76.667
2 5.86 6.014 19.5 21.688 66.0 68.25
3 7.94 8.014 31.4 26.244 100.0 94.696
4 5.95 5.722 18.3 21.269 79.0 82.054
5 9.9 9.746 27.6 25.413 76.0 73.75
6 5.96 5.886 18.9 24.056 78.0 83.304
7 7.11 7.338 19.5 16.531 58.0 54.946
8 5.04 6.534 16.0 15.944 78.0 67.554
9 7.36 5.712 22.8 20.669 42.0 50.196
10 13.89 12.396 21.8 21.856 74.0 84.446
11 8.41 9.83 20.8 25.9 80.0 74.857
12 6.18 7.828 27.0 29.131 91.0 82.804
13 6.93 6.085 30.8 28.45 100.0 90.571
14 6.66 6.085 31.2 28.45 76.0 90.571
15 6.18 6.363 28.6 27.4 85.0 76.667
16 6.47 6.363 26.2 27.4 70.0 76.667

2.5. Conclusions
A statistical DoE approach in conjunction with RSM was used to successfully optimize the
synthesis of phase-pure CoNi2S4 nanocrystals. A first-order fractional factorial design was
employed to screen the effects of five different variables relevant in the synthesis of the
nanocrystals at two levels, allowing robust information to be drawn about the role of each variable
in influencing three product characteristics: namely, nanocrystal size, size distribution, and
79
isolated yield. The statistical screening also allowed us to identify higher-order binary interaction
effects between the variables that significantly affected the desired outcomes; for example, the
interaction of Co:Ni precursor ratio and Co:DDT ratio allows for a minimization of nanocrystal
size. These higher-order interactions are very challenging to identify using common OVAT
methods. Not only were we able to uncover the magnitude of each variable’s effects on the product
using this screening design, but we also elucidated the directionality of those effects, offering
essential insight into determining factors at play.
A Doehlert uniform-shell optimization design for the three statistically significant variables
(Co:Ni precursor ratio, Co:DDT ratio, and temperature) was able to analyze each of the three
factors at a greater number of levels in order to fit the experimental data to a quadratic equation
that modelled each response in the system as a function of the investigated factors via multiple
regression techniques. The model equations were graphically represented as response surfaces and
subsequently optimized to find the optimum value for each respective response. These three
functions were then inputted to a desirability function to optimize all three responses
simultaneously and find an optimum for the overall synthesis.
Despite DoE only having accurate predictive power up to quadratic coefficients, we
successfully demonstrated that it is still possible to screen several variables at once with an
adequate resolution for extracting enough significant information to accurately predict
characteristics of a nanocrystal synthesis using binary interaction effects.
14
It has also been
suggested that these kinds of statistical DoE methods only possess the predictive power for the
optimization of a single optimizing goal, but again, we demonstrate with this work that the
optimization of multiple responses simultaneously is possible with a careful choice of design, even
for a complex material system like that of a ternary thiospinel.
80
The nature of the statistical designs employed here can easily be generalized and
implemented in the optimization of a vast array of material syntheses, beyond the current suggested
limitations. These techniques may also be expanded upon to achieve an even greater degree of
experimental and statistical efficiency if they are used in conjunction with automated high-
throughput reactors, which allow for the collection of a large amount of experimental data in a
short amount of time. We are currently in the process of combining such techniques and moving
in the direction of active learning modules for the acquisition of large, normalized experimental
data sets for the optimization and mechanistic analysis of material systems.

2.6. References
(1) Wang, D.; Zhang, X.; Du, Z.; Mo, Z.; Wu, Y.; Yang, Q.; Zhang, Y.; Wu, Z. CoNi2S4
Nanoparticles as Highly Efficient Electrocatalysts for the Hydrogen Evolution Reaction in
Alkaline Media. Int. J. Hydrog. Energy 2017, 42, 3043–3050.

(2) Guo, K.; Ding, Y.; Luo, J.; Yu, Z. Nickel Cobalt Thiospinel Nanoparticles as
Hydrodesulfurization Catalysts: Importance of Cation Position, Structural Stability, and
Sulfur Vacancy. ACS Appl. Mater. Interfaces 2018, 10, 19673–19681.

(3) Liu, Q.; Jin, J.; Zhang, J. NiCo2S4@Graphene as a Bifunctional Electrocatalyst for Oxygen
Reduction and Evolution Reactions. ACS Appl. Mater. Interfaces 2013, 5, 5002–5008.

(4) Zhao, Q.; Yan, Z.; Chen, C.; Chen, J. Spinels: Controlled Preparation, Oxygen
Reduction/Evolution Reaction Application, and Beyond. Chem. Rev. 2017, 117, 10121–
10211.

(5) Yuan, D.; Huang, G.; Yin, D.; Wang, X.; Wang, C.; Wang, L. Metal-Organic Framework
Template Synthesis of NiCo2S4@C Encapsulated in Hollow Nitrogen-Doped Carbon Cubes
with Enhanced Electrochemical Performance for Lithium Storage. ACS Appl. Mater.
Interfaces 2017, 9, 18178–18186.

(6) Hu, W.; Chen, R.; Xie, W.; Zou, L.; Qin, N.; Bao, D. CoNi2S4 Nanosheet Arrays Supported
on Nickel Foams with Ultrahigh Capacitance for Aqueous Asymmetric Supercapacitor
Applications. ACS Appl. Mater. Interfaces 2014, 6, 19318–19326.

81
(7) Hua, M.; Chen, T.; Ma, H. Design of a Functionalized Carbon Cloth Substrate for a Ni and
Co-Based High-Performance Supercapacitor. ACS Appl. Energy Mater. 2019,
acsaem.9b00515.

(8) Yu, L.; Zhang, L.; Wu, H. B.; Lou, X. W. D. Formation of NixCo3−xS4 Hollow Nanoprisms
with Enhanced Pseudocapacitive Properties. Angew. Chem. Int. Ed. 2014, 53, 3711–3714.

(9) Chauhan, M.; Reddy, K. P.; Gopinath, C. S.; Deka, S. Copper Cobalt Sulfide Nanosheets
Realizing a Promising Electrocatalytic Oxygen Evolution Reaction. ACS Catal. 2017, 7,
5871–5879.

(10) Shoemaker, D. P.; Li, J.; Seshadri, R. Unraveling Atomic Positions in an Oxide Spinel with
Two Jahn−Teller Ions: Local Structure Investigation of CuMn2O4. J. Am. Chem. Soc. 2009,
131, 11450–11457.

(11) Vaughan, D. J.; Burns, R. G.; Burns, V. M. Geochemistry and Bonding of Thiospinel Minerals.
Geochim. Cosmochim. Acta 1971, 35, 365–381.

(12) Anantharaj, S.; Ede, S. R.; Sakthikumar, K.; Karthick, K.; Mishra, S.; Kundu, S. Recent Trends
and Perspectives in Electrochemical Water Splitting with an Emphasis on Sulfide, Selenide,
and Phosphide Catalysts of Fe, Co, and Ni: A Review. ACS Catal. 2016, 6, 8069–8097.

(13) Liu, P.; Sui, Y.; Wei, F.; Qi, J.; Meng, Q.; Ren, Y.; He, Y. One-Step Hydrothermal Synthesis
of CoNi2S4 for Hybrid Supercapacitor Electrodes. World Sci. 2019, 14, 14.

(14) Braham, E. J.; Davidson, R. D.; Al-Hashimi, M.; Arróyave, R.; Banerjee, S. Navigating the
Design Space of Inorganic Materials Synthesis Using Statistical Methods and Machine
Learning. Dalton Trans. 2020, 49, 11480-11488.

(15) Barim, G.; Smock, S. R.; Antunez, P. D.; Glaser, D.; Brutchey, R. L. Phase Control in the
Colloidal Synthesis of Well-Defined Nickel Sulfide Nanocrystals. Nanoscale 2018, 10,
16298–16306.

(16) Li, D.; Senevirathne, K.; Aquilina, L.; Brock, S. L. Effect of Synthetic Levers on Nickel
Phosphide Nanoparticle Formation: Ni5P4 and NiP2. Inorg. Chem. 2015, 54, 7968–7975.

(17) Muthuswamy, E.; Savithra, G. H. L.; Brock, S. L. Synthetic Levers Enabling Independent
Control of Phase, Size, and Morphology in Nickel Phosphide Nanoparticles. ACS Nano 2011,
5, 2402–2411.

(18) Kim, E.; Huang, K.; Saunders, A.; McCallum, A.; Ceder, G.; Olivetti, E. Materials Synthesis
Insights from Scientific Literature via Text Extraction and Machine Learning. Chem. Mater.
2017, 29, 9436–9444.

(19) Lendrem, D.; Owen, M.; Godbert, S. DOE (Design of Experiments) in Development
Chemistry: Potential Obstacles. Org. Process Res. Dev. 2001, 5, 324–327.
82
(20) Box, G. E. P.; Hunter, J. S.; Hunter, W. G. Statistics for Experimenters. Design, Innovation,
and Discovery, 2nd ed.; Wiley-Interscience: Hoboken, NJ, 2005.

(21) Lundstedt, T.; Seifert, E.; Abramo, L.; Thelin, B.; Nyström, Å.; Pettersen, J.; Bergman, R.
Experimental Design and Optimization. Chemom. Intell. Lab. Syst. 1998, 42, 3–40.

(22) Bezerra, M. A.; Santelli, R. E.; Oliveira, E. P.; Villar, L. S.; Escaleira, L. A. Response Surface
Methodology (RSM) as a Tool for Optimization in Analytical Chemistry. Talanta 2008, 76,
965–977.

(23) Mora-Tamez, L.; Barim, G.; Downes, C.; Williamson, E. M.; Habas, S. E.; Brutchey, R. L.
Controlled Design of Phase- and Size-Tunable Monodisperse Ni2P Nanoparticles in a
Phosphonium-Based Ionic Liquid through Response Surface Methodology. Chem. Mater.
2019, 31, 1552–1560.

(24) Fhionnlaoich, N. M.; Yang, Y.; Qi, R.; Galvanin, F.; Guldin, S. DoE-It-Yourself: A Case Study
for Implementing Design of Experiments into Nanoparticle Synthesis. ChemRxiv. 2019, DOI:
10.26434/chemrxiv.8198420.v1.

(25) Burrows, N. D.; Harvey, S.; Idesis, F. A.; Murphy, C. J. Understanding the Seed-Mediated
Growth of Gold Nanorods through a Fractional Factorial Design of Experiments. Langmuir
2017, 33, 1891–1907.

(26) Viana, A. D.; Nobrega, E. T. D.; Moraes, E. P.; Wanderley Neto, A. O.; Menezes, F. G.;
Gasparotto, L. H. S. Castor Oil Derivatives in the Environmentally Friendly One-Pot
Synthesis of Silver Nanoparticles: Application in Cysteine Sensing. Mater. Res. Bull. 2020,
124, 110755.

(27) Rondinone, A. J.; Samia, A. C. S.; Zhang, Z. J. A Chemometric Approach for Predicting the
Size of Magnetic Spinel Ferrite Nanoparticles from the Synthesis Conditions. J. Phys. Chem.
B 2000, 104, 7919–7922.

(28) Chauke, H. R.; Nguyen-Manh, D.; Ngoepe, P. E.; Pettifor, D. G.; Fries, S. G. Electronic
Structure and Stability of the Pentlandites Phys. Rev. B 2002, 66, 155105.
(29) Kwan, K. Preparation and Properties of Co9-xMxS8 (M = Ni, Rh, Ru, and Fe). Bull. Korean
Chem. Soc. 1986, 7, 102–105.

(30) Abe, S.; Čapek, R. K.; De Geyter, B.; Hens, Z. Tuning the Postfocused Size of Colloidal
Nanocrystals by the Reaction Rate: From Theory to Application. ACS Nano 2012, 6, 42–53.

(31) Kiefer, J. & Wolfowitz, J. Optimum Designs in Regression Problems, Ann. Math. Statist. 1959,
30, 271–294.

(32) Zhou, B.; Li, Y.; Gillespie, J.; He, G.-Q.; Horsley, R.; Schwarz, P. Doehlert Matrix Design for
Optimization of the Determination of Bound Deoxynivalenol in Barley Grain with
Trifluoroacetic Acid (TFA). J. Agric. Food Chem. 2007, 55, 10141–10149.
83

(33) Ferreira, S. L. C.; dos Santos, W. N. L.; Quintella, C. M.; Neto, B. B.; Bosque-Sendra, J. M.
Doehlert Matrix: A Chemometric Tool for Analytical Chemistry—Review. Talanta 2004, 63,
1061–1067.

(34) Doehlert, D. H. Uniform Shell Designs. Appl. Stat. 1970, 19, 231.

(35) Brik, M. G.; Suchocki, A.; Kamińska, A. Lattice Parameters and Stability of the Spinel
Compounds in Relation to the Ionic Radii and Electronegativities of Constituting Chemical
Elements. Inorg. Chem. 2014, 53, 5088–5099.

(36) Narayanasamy, A.; Sivakumar, N. Influence of Mechanical Milling and Thermal Annealing
on Electrical and Magnetic Properties of Nanostructured Ni–Zn and Cobalt Ferrites. Bull.
Mater. Sci. 2008, 3, 373–380.

(37) Ma, T. Y.; Dai, S.; Jaroniec, M.; Qiao, S. Z. Synthesis of Highly Active and Stable Spinel-
Type Oxygen Evolution Electrocatalysts by a Rapid Inorganic Self-Templating Method.
Chem. Eur. J. 2014, 20, 12669–12676.

(38) Dziembaj, R.; Molenda, M. Stabilization of the Spinel Structure in Li1+δMn2−δO4 Obtained by
Sol–Gel Method. J. Power Sources 2003, 119–121, 121–124.

(39) Choi, H. W.; Kim, S. J.; Rim, Y.-H.; Yang, Y. S. Effect of Lithium Deficiency on Lithium-
Ion Battery Cathode LixNi0.5Mn1.5O4. J. Phys. Chem. C 2015, 119, 27192–27199.

(40) Peng, Z.; Jia, D.; Al-Enizi, A. M.; Elzatahry, A. A.; Zheng, G. Electrocatalysts: From Water
Oxidation to Reduction: Homologous Ni-Co Based Nanowires as Complementary Water
Splitting Electrocatalysts. Adv. Energy Mater. 2015, 5.

(41) Shi, J.; Lei, K.; Sun, W.; Li, F.; Cheng, F.; Chen, J. Synthesis of Size-Controlled CoMn2O4
Quantum Dots Supported on Carbon Nanotubes for Electrocatalytic Oxygen
Reduction/Evolution. Nano Res. 2017, 10, 3836–3847.

(42) Chang, S.-K.; Zainal, Z.; Tan, K.-B.; Yusof, N. A.; Wan Yusoff, W. M. D.; Prabaharan, S. R.
S. Synthesis and Electrochemical Properties of Nanostructured Nickel-Cobalt Oxides as
Supercapacitor Electrodes in Aqueous Media: Synthesis and Electrochemical Properties of
Nanostructured Ni-Co Oxides. Int. J. Energy Res. 2015, 39, 1366–1377.

(43) Bao, J.; Zhang, X.; Fan, B.; Zhang, J.; Zhou, M.; Yang, W.; Hu, X.; Wang, H.; Pan, B.; Xie,
Y. Ultrathin Spinel-Structured Nanosheets Rich in Oxygen Deficiencies for Enhanced
Electrocatalytic Water Oxidation. Angew. Chem. Int. Ed. 2015, 54, 7399–7404.

(44) Sathya, A.; Guardia, P.; Brescia, R.; Silvestri, N.; Pugliese, G.; Nitti, S.; Manna, L.; Pellegrino,
T. CoxFe3–xO4 Nanocubes for Theranostic Applications: Effect of Cobalt Content and Particle
Size. Chem. Mater. 2016, 28, 1769–1780.

84
(45) Lee, E.; Jang, J.-H.; Kwon, Y.-U. Composition Effects of Spinel Mn Co3−O4 Nanoparticles on
Their Electrocatalytic Properties in Oxygen Reduction Reaction in Alkaline Media. J. Power
Sources 2015, 273, 735–741.

(46) Giovannelli, F.; Marsteau, V.; Zaghrioui, M.; Autret, C.; Delorme, F. Low Temperature
Synthesis of Co3O4 and (Co1−xMnx)3O4 Spinels Nanoparticles. Adv. Powder Technol. 2017,
28, 1325–1331.

(47) Zhu, H.; Zhang, S.; Huang, Y.-X.; Wu, L.; Sun, S. Monodisperse MxFe3–xO4 (M = Fe, Cu, Co,
Mn) Nanoparticles and Their Electrocatalysis for Oxygen Reduction Reaction. Nano Lett.
2013, 13, 2947–2951.

(48) Nguyen-Cong, H.; de la Garza Guadarrama, V.; Gautier, J. L.; Chartier, P. Oxygen Reduction
on NixCo3−xO4 Spinel Particles/Polypyrrole Composite Electrodes: Hydrogen Peroxide
Formation. Electrochimica Acta 2003, 48, 2389–2395.

(49) Rı
́ os, E.; Nguyen-Cong, H.; Marco, J. F.; Gancedo, J. R.; Chartier, P.; Gautier, J. L. Indirect
Oxidation of Ethylene Glycol by Peroxide Ions at Ni0.3Co2.7O4 Spinel Oxide Thin Film
Electrodes. Electrochimica Acta 2000, 45, 4431–4440.

(50) Toby, B. H. EXPGUI, a Graphical User Interface for GSAS. J. Appl. Crystallogr. 2001, 34,
210–213.

(51) Fhionnlaoich, N. M.; Guldin, S. Information Entropy as a Reliable Measure of Nanoparticle
Dispersity. Chem. Mater. 2020, 32, 3701-3706.

(52) Cao, B.; Adutwum, L. A.; Oliynyk, A. O.; Luber, E. J.; Olsen, B. C.; Mar, A.; Buriak, J. M.
How to Optimize Materials and Devices via Design of Experiments and Machine Learning:
Demonstration Using Organic Photovoltaics. ACS Nano 2018, 12, 7434.

(53) Moosavi, S. M.; Chidambaram, A.; Talirz, L.; Haranczyk, M.; Stylianou, K. C.; Smit, B.
Capturing Chemical Intuition in Synthesis of Metal-Organic Frameworks. Nat. Commun.
2019, 10, 539.

(54) Box, G. E. P.; Hunter, J. S.; Hunter, W. G. Statistics for Experimenters. Design, Innovation,
and Discovery, 2nd ed.; Wiley-Interscience: Hoboken, NJ, 2005.

85
Chapter 3. Throughput Optimization of Molybdenum Carbide Nanoparticle Catalysts in a
Continuous Flow Reactor Using Design of Experiments*

*Published in ACS Appl. Nano Mater. 2022, 5, 1966-1975.

3.1. Abstract
Transition metal carbides (TMCs) have attracted significant attention because of their
applications toward a wide range of catalytic transformations. However, the practicality of their
synthesis is still limited because of the harsh conditions in which most TMCs are prepared.
Recently, a solution-phase synthesis of phase-pure α-MoC1−x nanoparticles was presented. While
this synthetic route yielded nanoparticles with exceptional catalytic performance, the reaction
parameter space was not explored, and catalyst throughput was not optimized for scale-up.
Continuous flow platforms coupled with statistical design of experiments (DoE) can provide a
powerful method for understanding the reaction parameter space for optimizations. Here, we
demonstrate the use of statistical DoE in tandem with response surface methodology for a
parametric screening analysis to optimize the throughput of a MoC1−x nanoparticle synthesis
utilizing a millifluidic flow reactor. A full factorial design was implemented to evaluate four input
variables (reaction temperature, flow rate, solvent fraction of oleylamine, and precursor
concentration) that carry statistically significant effects on three responses (throughput, residence
time, and isolated yield). A Doehlert matrix was implemented to investigate each significant
variable at a higher number of levels to optimize throughput. Our results give a nonintuitive set of
experimental conditions that resulted in an optimized throughput of 2.2 g h
−1
. This translates to a
50-fold increase in throughput compared to the previously reported batch method. The catalytic
performance of the MoC1−x nanoparticles produced under optimized throughput was demonstrated
86
in the CO2 hydrogenation reaction. This DoE screening analysis and throughput optimization of
MoC1−x synthesis open the door to an increased feasibility for scale-up.

3.2. Introduction
The increasing demand for sustainable routes to produce fuels and chemicals motivates the
development of low-cost, Earth-abundant catalysts that maintain stability under catalytic
conditions.
1
Transition metal carbides (TMCs) possess inherent multifunctionality that enables
exceptional catalytic performance for a wide range of transformations, including hydrogenation,
2−6

isomerization,
7−10
deoxygenation,
11
and hydrodeoxygenation reactions.
12
Levy and Boudart were
among the first to report that TMCs possess similarities in catalytic behavior and electronic
structure to noble metals, such as Pt.
13
However, these early TMCs were orders of magnitude less
active than noble metals, driving efforts to synthesize TMCs with higher active surface areas.
High-temperature carburization has been demonstrated as a route to synthesize higher surface area
TMC powders; however, this approach requires harsh conditions with temperatures typically
exceeding 600 ˚C that limit the practicality of high-throughput industrial-scale manufacturing.
14−16

Molybdenum carbide (particularly the α-MoC1−x and β-Mo2C phases) is an attractive TMC
because of superior reactivity and the potential to obtain higher specific surface areas (177−210
m
2
g
−1
).
17−19
While synthetic routes to high-surface area α-MoC1−x have been demonstrated, they
still rely on a high-temperature carburization step.
20
We recently addressed some of these
shortcomings by developing the first mild solution-phase synthetic route to produce phase-pure α-
MoC1−x nanoparticles.
21
This synthetic approach was enabled by the thermolytic decomposition of
Mo(CO)6 at comparatively low temperatures (i.e., 320 ˚C) and was further extended to other TMC
nanoparticles, including β-WC, using an analogous W(CO)6 precursor. The resulting α-MoC1−x
87
nanoparticle catalysts exhibited a twofold increase in activity on a per-site basis and an increase in
selectivity toward C2+ hydrocarbon products, as compared to a lower surface area bulk α- MoC1−x
catalyst for the thermocatalytic hydrogenation of CO2.
21
Despite the mild synthetic conditions
employed to make this catalyst, the volumetric scaling of nanoparticle synthesis by batch processes
is not ideal because of batch-to-batch variability and nonuniform reactor conditions.
22
More
specifically to the batch MoC1−x nanoparticle synthesis, Mo(CO)6 precursor sublimation
contributes to relatively low isolated yield, and significant gas evolution during the reaction
introduces safety concerns, ultimately making process intensification and scale-up untenable.
Consequently, this lab-scale batch reaction resulted in a relatively low isolated yield of 40−50%
after a 1 h reaction time, for a throughput of only ca. 0.047 g h
−1
per reaction, reflecting the by-
hand nature of the process.
21
Continuous flow systems for nanoparticle syntheses are an attractive alternative approach
that eliminates many of the aforementioned problems encountered in traditional batch processes.
23

The superior heat and mass transport characteristics provided by the small, millimeter-scale
channels allow for greater control over heating by virtue of reduced thermal masses and higher
surface area-to-volume ratios. In contrast to batch syntheses, where the inhomogeneous nucleation
and growth rates that result from local thermal inhomogeneities introduce batch-to-batch
variability (which is only further exacerbated when volumetric scale-up is attempted), continuous
flow approaches enable high-temperature reactions to be performed with improved product
consistency and reproducibility,
22,24−27
while generating higher product yields with shorter reaction
times.
21,28,29
Our previous unoptimized attempt to synthesize MoC1−x nanoparticles in flow resulted
in a throughput of 0.77 g h
−1
.
21

88
While our previous study demonstrated the successful synthesis of MoC1−x nanoparticles
via continuous flow methods, the synthetic parameter space was undefined, resulting in an
unoptimized system. The experimental variable space for nanoparticle syntheses is high
dimensional and complex by nature, and, when coupled with the additional operational parameters
of a flow reactor, optimizations become nearly impossible via traditional one variable at a time
(OVAT) methods, which are prohibitively slow and costly for all but the simplest reactions. With
that being said, continuous flow methodologies are amenable to the rapid screening of reaction
conditions for the multivariate optimization of synthetic conditions.
30−32
High-throughput
experimentation has been used to increase the efficiency of such investigations, including self-
optimizing flow reactors that combine in situ analysis with feedback algorithms like SNOBFIT to
rapidly map a response surface and identify an target optimum
33−36
or Bayesian optimizations that
combine a mixture of theoretical, literature, and experimental screening data to learn general
predictive schemes for a system.
37,38
However, in our case, in situ analysis of the target response
(throughput) is implausible because the resulting MoC1−x nanoparticles lack a spectral signature,
which rules out implementation of automated self-optimization. This is not a unique case, as
materials with meaningful spectral signals are the exception rather than the rule. More complex
machine learning models, such as Bayesian optimizations, utilize a balance of both exploiting
existing data and exploring a design space, so yield optimizations require large libraries of existing
theoretical or literature data to pull from to effectively train a model.
33−38
This is impractical for
our system as this is novel chemistry with no library of preexisting data to pull from, and an
optimization must therefore be purely exploratory.
Considering the limitations of the system under study, statistical design of experiments
(DoE) in conjunction with response surface methodology (RSM) is the most efficient option for
89
response surface creation and optimization. As a well-known, powerful optimization method that
has been proven effective for synthetic optimizations of nanoparticles, the combination of the two
techniques offers more precise analysis of a unique system that is required here, as the simplicity
of the regression techniques enables elucidation of the parameter space based solely on in-house
experimental data and ex situ nanoparticle characterization in a minimal number of
experiments.
39−41
This removes the difficulties encountered when considering the implementation
of machine learning approaches while still efficiently providing a comprehensive evaluation of the
multivariate design space (including interaction effects) and an accurate model of the response
surface for identification of the optimal reaction conditions.
40−46
In addition, coupling such
techniques with the advantages of flow systems further advances the speed and efficiency of
reaction parameter space elucidation for nano- particle optimization. Herein, we utilize statistical
DoE for a parametric screening analysis to explore the experimental variables governing the
MoC1−x nanoparticle synthesis in a continuous flow millifluidic reactor in tandem with RSM to
optimize MoC1−x nanofabrication with the goal of identifying the reaction conditions that
maximize the product throughput. We successfully use this approach to maximize the throughput
of catalytically active MoC1−x nanoparticles, discovering nonintuitive reaction conditions that lead
to maximum throughput, thereby demonstrating a robust route to scale-up.

3.3. Results and Discussion
3.3.1. Continuous Flow Millifluidic Synthesis and Reactor Setup
The colloidal MoC1−x nanoparticles were synthesized through a solution-phase thermolytic
decomposition of Mo(CO)6 in oleylamine (acting as a surface ligand
47
) and octadecene (high
boiling solvent), as previously described.
21
High-throughput continuous flow synthesis was
90
performed using the millifluidic reactor configuration illustrated in Figure 3.1. A syringe pump
was used to deliver a precursor Mo(CO)6 solution at a constant flow rate (determined as described
in the Experimental Procedures and Methods section) into a custom-made borosilicate
millifluidic glass reactor placed inside a convection furnace. As the precursor enters the heated
zone, it spontaneously decomposes to release CO gas and separate the liquid into isolated plugs
(Figure 3.1). Based on a transient heat transfer approximation (see Experimental Procedures
and Methods), and assuming a constant surface heat flux, the reagent solution reaches a reaction
temperature of 240 ˚C at ca. 2.5 s after entering the furnace and asymptotically approaches the
final temperature of 340 °C after ca. 13 s. The time required for the fluid to approach the steady-
state temperature is negligible relative to the minimum residence time achieved in this work (ca.
6 min) and is significantly faster than ramping times in the batch “heating up” process (ca. 10 min).
Segmented flow prevents axial dispersion while recirculation within the liquid plugs facilitates
rapid mixing.
48,49
As gas evolution resulting from precursor decomposition can drive the flow in
either direction, a one-way valve is placed upstream of the furnace to prevent the backflow of the
reaction solution. In addition, the reactor is maintained at 20 psig to minimize in situ gas evolution.
The pressurized system elevates the reaction mixture boiling point above the highest temperature
used in the matrix of experimental conditions. As the product emerges from the furnace, the
reaction is thermally quenched, and the nanoparticle suspension passes through an in-line flow
sensor to determine the gas-to-liquid plug volume ratio (Figure 3.1). The residence time of the
reaction is calculated based on the ratio of gas to liquid. Exemplary data and calculations are given
in Figure 3.2 and the Experimental Procedures and Methods section. Although the plug size
varies over time during an experiment, this variation does not result in run-to-run inconsistencies
91
as demonstrated by experimental replicates. Beyond the flow sensor, the product is directed
through a set of manually operated valves to collection reservoirs.

Figure 3.1. Schematic of the millifluidic reactor system for the continuous flow synthesis of
MoC1−x nanoparticles. A syringe pump with a heated syringe (80 ˚C) is used to drive a precursor
mixture into a reactor coil housed in a furnace. The colors are representative of those empirically
observed in the reaction, with yellow (at reactor inlet) indicating the unreacted Mo(CO)6 precursor
solution and black indicating the MoC1−x nanoparticle suspension isolated into plugs because of in
situ gas evolution. An in-line thermal dissipation-type flow sensor at the reactor outlet reports the
relative volume of liquid and gas—an idealized flow sensor signal is shown with corresponding
liquid (detected by the sensor) and gas (not detected) plugs. The product is collected in vials that
can be isolated with valves to allow for removal of partial product volumes during runs. The system
pressure is maintained by a fluidic controller (Fluigent), which is locked on a fixed pressure of 20
psi to minimize in situ gas evolution.

92
Figure 3.2. Exemplary data for the sensor readout during product collection with a pump flow rate
set at 𝑄L
= 40 mL h
-1
. The data shown here correspond to a 2-min time frame for experiment #1 in
the full factorial matrix (Table 3.3). Data are sampled at 10 Hz. Based on the count of data points
above and below the threshold throughout the experiment (14 min), the gas to liquid volumetric
ratio was found to be 3.74, which translates to 𝑄G
=150 mL h
-1
. Therefore, the residence time is
calculated is 𝜏 = 𝑉/(𝑄
L

+ 𝑄
G

) = 11.7 min.

The base reaction, denoted with the coded valued “0” in Table 3.1, gives phase-pure, face-
centered cubic α-MoC1−x nanoparticles (ICDD PDF # 03-065-8092). This reaction was run at a
flow rate of 25 mL h
−1
, a temperature of 315 ˚C, a Mo(CO)6 precursor concentration of 352 mM,
and a 53% volume fraction of oleylamine in octadecene. These reaction conditions resulted in an
isolated yield of 69%, a residence time of 14 min, and a product throughput of 0.62 g h
−1
. The
powder X-ray diffraction (XRD) pattern of the resulting product shows significant peak
broadening, consistent with small nanoparticles. Scherrer analysis of the XRD pattern indicated a
crystallite size of 2 nm. This size is qualitatively similar to the size of the multipodal nanoparticles
observed by transmission electron microscopy (TEM). These data, shown in Figure 3.3, are fully
consistent with previous reports of this material, with the α-phase being sufficiently distinct from
the β- and γ-phases of molybdenum carbide.
21

93
Table 3.1. Coded low (-1), center (0), and high (+1) experimental values for the input variables
investigated in the full factorial screening design.
Input variables Flow rate
(mL h
-1)
Concentration
(mM)
Temperature
(˚C)
Amount of
oleylamine (vol %)
Coded low (-1) 10 78 290 5
Coded center point (0) 25 352 315 53
Coded high (+1) 40 625 340 100

Figure 3.3. (a) Powder XRD pattern and (b) TEM image of the MoC1−x nanoparticles synthesized
under the base reaction conditions.

3.3.2. Design of Experiments
The first step in employing DoE and predictive RSM is to determine the experimental input
variables that may affect the outcome of a specific system (e.g., throughput, isolated yield, or
residence time). These variables are typically chosen via prior knowledge of the system and
assessment of the current literature.
21,50
The input variables chosen for this system are reaction
temperature (˚C), volume fraction of oleylamine in octadecene (vol%), Mo(CO)6 precursor
concentration (mM), and syringe pump flow rate (mL h
−1
). Preliminary reactions are then
performed to determine the high and low bounds of each variable, the highest and lowest values
where the reaction produces MoC1−x nanoparticles, as assessed by powder XRD. This creates the
bounds/edges of the parameter space specific to this flow reactor and synthetic system that can be
94
evaluated as part of the DoE. Any points outside of the bounded design space presented here are
not feasible for this system. The bounds were established by analogous batch reactions, except for
the flow rate, which was bounded by the mechanical limitations of the syringe pump and
millifluidic reactor apparatus. Each variable was chosen because of its typical importance in
conventional nanoparticle syntheses. The input variables and their corresponding high and low
levels (coded +1 or −1, respectively) are given in Table 3.1. Similar MoC1−x crystallite sizes of
the α-phase, as assessed by Scherrer analysis of the powder XRD data, were observed for all
screening reactions. The responses, or the output data that are subsequently analyzed, were then
chosen based on the nature of this system and the most important experimental aspects when
considering scale-up optimization, which in this case were isolated yield, residence time (which is
related in a complex way to the syringe pump flow rate due to gas evolution), and product
throughput.
Once the high and low bounds are set, the combinations of k factors (variables) can be
investigated at N number of levels to create a factorial matrix in a statistical screening design. To
rapidly screen the effects of each factor and the combinations of the factors on the responses, an
investigation at two levels (N = 2) is standard, as it is the most robust and efficient. Considering
that the goal of an initial screening is to assess significance and not to resolve the fine details of
the system, a design involving solely the absolute high and low extremes (i.e., the boundary
conditions) of each variable enables an adequate investigation of the entire expanse of the reaction
space in the smallest possible number of experiments. A full factorial design will then consist of
N
k
combinations.
50
For this system, the full factorial consisted of 2
4
(16) experiments, because four
variables, or factors, were investigated at two levels each. The coded and real values of the full
factorial matrix are given in Tables 3.2 and 3.3, respectively. In addition to the experiments given
95
in the matrix, two replicates of the center point of the design space, also known as the base reaction
(coded as 0), and two replicates chosen at random from the matrix were performed to help quantify
run-to-run variance and increase statistical significance. As a result of the complexity of the
system, we chose to perform the full factorial to ensure maximum data resolution (V), which
ensures minimal to no confounding of the variable interactions.
50

Table 3.2. Full Factorial matrix via coded values.
Input Variables Responses
Block Flow
Rate
(mL h
-1
)
Temperature
(˚C)
Amount of
Oleylamine
(% vol)
Concentration
(mM)
Residence
Time (min)
Isolated
Yield (%)
Throughput
(g h
-1
)
1 1 -1 1 -1 11.7 0.69 0.219
1 -1 1 1 -1 18.3 0.55 0.044
1 -1 -1 -1 -1 19.1 0.60 0.048
1 1 1 -1 -1 9.3 0.36 0.115
1 0 0 0 0 14.7 0.69 0.618
1 0 0 0 0 13.0 0.69 0.618
1 -1 -1 1 1 24.0 0.95 0.605
1 1 1 1 1 18.6 0.71 1.809
1 1 -1 -1 1 6.0 0.33 0.841
1 -1 1 -1 1 21.0 0.23 0.147
1 -1 -1 1 -1 24.8 0.72 0.057
1 1 1 1 -1 9.7 0.70 0.223
1 1 -1 -1 -1 12.5 0.80 0.254
1 -1 1 -1 -1 34.0 0.22 0.017
1 1 -1 1 1 9.5 0.93 2.370
1 -1 1 1 1 30.0 0.20 0.127
1 -1 -1 -1 1 25.0 0.18 0.115
1 1 1 -1 1 6.9 0.05 0.127
1 -1 -1 1 -1 29.7 0.79 0.063
1 1 1 1 -1 11.4 0.70 0.223

96
Table 3.3. Full factorial matrix of the corresponding real values.
Block Flow Rate (mL h
-1
) Temperature (°C) Amount of Oleylamine (% vol) Concentration (mM)
1 40 290 100 78
1 10 340 5 625
1 40 290 5 625
1 10 340 100 78
1 25 315 52.5 351.5
1 25 315 52.5 351.5
1 40 340 100 625
1 10 290 10 78
1 40 340 10 78
1 10 290 100 625
1 10 290 100 78
1 40 340 100 78
1 40 290 5 78
1 10 340 5 78
1 40 290 100 625
1 10 340 100 625
1 10 290 5 625
1 40 340 5 625

The results of the full factorial screening are illustrated through Pareto charts and main
effects plots in Figure 3.4. The Pareto chart is used to determine which factors are statistically
relevant; that is, the length of each bar is proportional to the value of a t-statistic calculated for the
corresponding effect. Any bars beyond the vertical error line represent statistically significant
factors at a determined significance level (α = 5%). For example, the sole factor that affected the
residence time was the flow rate, with higher flow rates yielding shorter residence times (Figure
3.4a). For isolated yield, higher fractions of oleylamine and lower temperatures had significant
effects on the resulting isolated yield, as depicted in Figure 3.4b. Finally, a high precursor
concentration, high flow rate, high fractions of oleylamine, and the binary interactions between
those three variables significantly affected the throughput (Figure 3.4c). The main effects plots
help visualize the change in each indicated response from the low level to the high level for each
97
of the factors (Figure 3.4d−f), with a steeper slope corresponding to a more significant effect on
the given response.

Figure 3.4. Standardized Pareto charts for (a) residence time, (b) isolated yield, and (c) throughput.
The variables on the y-axis of the Pareto charts are defined as A = flow rate (mL h
−1
), B =
temperature (˚C), C = amount of oleylamine (vol %), and D = precursor concentration (mM). The
vertical line in each Pareto chart corresponds to α = 5%, and (+) and (−) correspond to an increase
or decrease in the response, respectively, for the high value of a given factor. Main effects plots
for (d) residence time, (e) isolated yield, and (f) throughput, where (+) and (−) correspond to the
high and low levels for each factor, respectively.

Based on the significant impact of multiple variables on throughput from the screening
data, an optimization was performed with the goal of maximizing throughput as a single response
98
with important relevance to scale-up. Because only three variables (i.e., fraction of oleylamine,
precursor concentration, and flow rate) significantly affected throughput, the reactor temperature
was fixed as a constant to the low level (-1) which corresponds to a real value of 290 °C. The low
level was chosen because of its apparent positive, though not statistically significant, effect on the
throughput (see Figure 3.4f), in addition to the decreased energy consumption required for lower
temperature reactions. To perform an optimization of the experimental conditions and create a
response surface of the parameter space, a second-order design is needed.
51,52
We employed the
Doehlert, or uniform shell, design for this optimization because it allows for the critical factors to
be investigated in differing amounts of detail based on their significance.
53
That is, significant
factors that have a clear hierarchy in regard to their impact on the response can be assigned varying
levels of investigation, weighing their significance accordingly. This enables a more thorough
investigation of the variables without increasing the number of experiments and will increase the
accuracy of the fitted model. The spherical nature of the uniform-shell model also allows for
smooth movement throughout the parameter space and more efficient model predictions in fewer
experiments.
53
As illustrated in Figure 3.4c, the variables that significantly affected throughput
had a clear ranking of significance with the amount of oleylamine having the smallest impact, flow
rate having a moderate impact, and precursor concentration having the greatest impact. We
therefore decided to investigate the corresponding variables at three, five, and seven levels,
respectively. Table 3.4 provides the real values of each investigated level for each variable. The
entire Doehlert matrix (coded and real values) is given in Tables 3.5 and 3.6, respectively). The
Doehlert matrix for these three factors corresponded to 13 experiments, as well as two random
replicates in the design space and two replicates of the center point, or base reaction.

99
Table 3.4. Real values of the levels investigated for each significant variable in the Doehlert
optimization of maximizing throughput. As depicted from the number of levels investigated for
each variable, the variables most significantly affecting throughput in decreasing order are
concentration, flow rate, and amount of oleylamine.
Input variables Level 1

Level 2 Level 3 Level 4 Level 5 Level 6 Level 7
Concentration (mM) 115 194 273 352 431 509 588
Flow rate (mL h
-1
) 10 18 25 33 40
Amount of oleylamine
(vol %)
14 53 92

Table 3.5. Coded Doehlert optimization matrix appended with points on each corner of the design
space.
Run Flow Rate (mL h
–1
)
Amount of Oleylamine
(%vol)
Concentration
(mM)
Throughput (g h
–1
)
1 0 0 0 0.608927
2 1 0 0 0.888317
3 -1 0 0 0.254317
4 -0.5 0 -0.866 0.198324
5 0.5 0 -0.866 0.353127
6 -0.5 -0.817 -0.289 0.233224
7 0.5 -0.817 -0.289 0.342895
8 0.5 0.817 0.289 0.926832
9 -0.5 0.817 0.289 0.760111
10 0.5 0 0.866 0.837875
11 -0.5 0 0.866 0.608544
12 0 -0.817 0.577 0.467106
13 0 0.817 -0.577 0.345411
14 0.5 0 -0.866 0.364519
15 0.5 0 -0.866 0.364519
16 0 0.817 -0.577 0.340477
17 0 0.817 -0.577 0.330608
18 1 -1 -1 0.25
19 -1 -1 1 0.11
20 -1 -1 -1 0.05
21 1 1 1 2.4
22 -1 1 1 0.61
23 1 1 -1 0.22
24 1 -1 1 0.84
25 -1 1 -1 0.06
26 -1 1 -1 0.06
27 0 0 0 0.77
28 0 0 0 0.76

100
Table 3.6. Optimization matrix of the corresponding real values.
Run Flow Rate (mL h
–1
) Amount of Oleylamine (%vol) Concentration (mM)
1 25 52.5 351.5
2 40 52.5 351.5
3 10 52.5 351.5
4 17.5 52.5 114.65
5 32.5 52.5 114.65
6 17.5 13.7 272.46
7 32.5 13.7 272.46
8 32.5 91.3 430.54
9 17.5 91.3 430.54
10 32.5 52.5 588.35
11 17.5 52.5 588.35
12 25 13.7 509.31
13 25 91.3 193.69
14 32.5 52.5 114.65
15 32.5 52.5 114.65
16 25 91.3 193.69
17 25 91.3 193.69
18 40 5 78
19 10 5 625
20 10 5 78
21 40 100 625
22 10 100 625
23 40 100 78
24 40 5 625
25 10 100 78
26 10 100 78
27 25 52.5 351.5
28 25 52.5 351.5

3.3.3. Optimization via RSM: Maximizing Throughput

After successful collection of the experimental data representing the Doehlert matrix, RSM
can be performed to optimize the throughput. A surface response function was fitted to the
aforementioned data to serve as a predictive polynomial model of the curvature of the surface in
three dimensions (eq. 3.1). This is done by depicting the response (i.e., throughput) in the 3D
parameter space as a function of the three input variables (i.e., concentration, flow rate, and amount
of oleylamine). The function is produced, initially, by performing a linear regression to fit the input
experimental data to the polynomial via regression coefficients. The function is continually
101
enhanced by performing an exchange algorithm.
54
This algorithm tests all pairs of experimental
runs, consisting of one that has been selected in the design space and one that has not, making any
exchanges that would increase the efficiency of the model. Exchanges continue until no further
improvements can be made by switching one run that has been selected with one run that has not.
This adjusts the function to minimize the residual error term, which is a numerical value that
represents the observed-predicted value of each design point as each data point is introduced. Once
the resulting function is an optimal representation of the design space, it can then be extrapolated
beyond the input data values to predict the likely throughput response for every data point within
the 3D parameter space, providing a coherent model of our synthetic system.
𝑇ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡 (𝑔 ℎ
#'
) = 0.578 + 0.319𝐴 + 0.235𝐵 + 0.370𝐶 + 0.0491𝐴
(

+0.105𝐴𝐵 + 0.249𝐴𝐶 − 0.0577𝐵
(
+ 0.253𝐵𝐶 − 0.0179𝐶
(
(3.1)
where A is the flow rate (mL h−1), B is the amount of oleylamine (vol%), and C is the Mo(CO)6
concentration (mM).
Because all the variables are normalized through the coded values, the relative change of a
variable is directly related to the size of its regression coefficient (eq. 3.1). Therefore, the equation
using the coded values can be used to compare the absolute effect of the variables on the response
of the throughput. This means that if the model coefficients have a large absolute value, the
corresponding variable has a significant effect on the response.
40
As shown by the statistical
analysis of eq. 3.1 (see the Experimental Procedures and Methods), throughput had a
statistically significant quadratic dependence on all three factors to the 95% confidence interval.
Using the quadratic terms in eq. 3.1, the model predicts a maximum throughput of 2.1 g h
−1
when
the three parameters (i.e., precursor concentration, flow rate, and amount of oleylamine) are set to
the coded values (1, 1, 1), respectively, which correspond to real values of 625 mM, 40 mL h
−1
,
102
and 100%. These predicted conditions for a maximum throughput are nonintuitive for this
synthetic system (specifically, 100% oleylamine and low temperature) and could not be
ascertained through chemical intuition or previous literature. Therefore, the implementation of
DoE is imperative for this optimization. A visual representation of the parameter space based on
the fitted model predictions of throughput plotted in three dimensions is given in Figure 3.5, with
the optimum condition indicated with a star. Additionally, two-dimensional representations of the
surface response are depicted through contour plots in Figure 3.6.

Figure 3.5. Calculated response surface function demonstrating the reaction conditions (precursor
concentration, flow rate, and amount of oleylamine) that correspond to a specific throughput,
illustrated by the color legend. The conditions for maximum throughput are indicated by a star.
The bounds of this parameter space are specific to this flow reactor and synthetic system; any
points outside of the parameter space are not feasible for this system.

103

Figure 3.6. (a-c) Contour plots of MoC1−x nanoparticle throughput with flow rate held constant at
the (a) low level (coded -1), (b) center (coded 0), and (c) high level (coded +1). (d-f) Contour plots
of throughput with amount of oleylamine held constant at the (d) low level (coded -1), (e) center
(coded 0), and (f) high level (coded +1). (g-i) Contour plots of throughput with Mo(CO)
6 precursor
concentration held constant at the (g) low level (coded -1), (h) center (coded 0), and (i) high level
(coded +1).

Replicates of the center point in triplicate create a basis for the constant value in the
polynomial function (see eq. 3.1). Additionally, the center point replicates along with the replicates
of two random points in the design allow for a variance estimate of the polynomial function’s
vector value, which is a simplified representation of the 3D function that contains the coefficients
of each term in the model and their directionality (positive or negative effect). This essentially
allows for the assessment and correction of run-to-run variability, or sampling error, in the fitting
of the model via experimental data and helps to calculate model accuracy. This is done by
extracting the square root of each component of the vector value, giving the standard errors for the
coefficients (regression error and residual error). The sum of squares data allows the residual error
value to be further split into pure error and lack of fit. Based on the evaluation of the fitted model,
104
the fit was able to explain 92% of the variability in throughput with no serial autocorrelation of
the residuals at a 5% significance level, as indicated by R
2
. The prediction variance increases
slightly around the edges of the parameter space (Figure 3.7) because of the spherical nature of
the optimization design, as depicted in Figure 3.8. This indicates an accurate model of the design
space; a more detailed description of the model’s fit and prediction accuracy can be found in the
Experimental Procedures/Methods Section (3.4).

Figure 3.7. Prediction variance plot of the design space. Concentration is set at the base value
(coded 0) for visual simplicity.

Figure 3.8. Visual representation of the design points for the optimization, corresponding to the
runs displayed in Table 3.5.

Prediction Variance Plot
Concentration=0.06
-1
0
1
Flow Rate
-1
0
1
Amount of Oleylamine
-1
0
1
Concentration
Stnd. error
0.0
0.3
0.6
0.9
1.2
1.5
1.8
2.1
2.4
2.7
3.0
Stnd. error
Design Points
-1
-0.6
-0.2
0.2
0.6
1
Flow Rate
-1
-0.6
-0.2
0.2
0.6
1
% OA m
-1
-0.6
-0.2
0.2
0.6
1
Concentration
105
Several subsequent reactions were performed to validate the predicted model. First, the
optimized reaction conditions that were predicted to maximize throughput were performed in
triplicate. The XRD pattern and TEM image of the resulting optimized MoC1−x nanoparticles are
provided in Figure 3.9a,b, respectively, and are consistent with the α-phase. Selected area electron
diffraction of these MoC1−x nanoparticles corroborates the assignment of the α-phase. The lattice
fringes of MoC1−x nanoparticles were observed through high-resolution TEM and suggest single
crystalline particles (Figure 3.10). The measured d-spacing (0.21 nm) corresponds to the (200)
plane, in agreement with previous reports.
21
The throughput optimization did not affect the MoC1−x
nanoparticle phase or crystallite size. The average isolated yield of this reaction, including the
replicates, was 87 ± 3%. The average experimental throughput achieved was 2.2 ± 0.04 g h
−1
,
compared to the predicted throughput under these reaction conditions, which was 2.1 ± 0.3 g h
−1
.
The optimized throughput represents a 3.5× increase compared to the base reaction, which afforded
an average throughput of 0.62 g h
−1
. In addition, triplicate reactions were caried out with conditions
from a random, unoptimized, point on the surface. The conditions of the unoptimized reaction
consisted of a flow rate of 30 mL h
−1
, a Mo(CO)6 precursor concentration of 100 mM, and an 86%
volume fraction of oleylamine in octadecene. The average isolated yield resulting from these
reaction conditions, including replicates, is 86 ± 8%. The average experimental throughput
achieved under these conditions was 0.26 ± 0.02 g h
−1
, which matches the predicted throughput of
0.25 ± 0.3 g h
−1
from the response surface under these specific conditions. The powder XRD
pattern and TEM image of the resulting nanoparticle product from the unoptimized reaction are
shown in Figure 3.9c,d, respectively. The predicted and experimental throughput values from both
the optimized synthetic conditions and a random set of conditions, both performed in triplicate,
fall within standard error of the predicted throughput, thus, successfully validating the model. To
106
determine if there is any unreacted Mo(CO)6 precursor in the product stream, Fourier transform
infrared spectra were obtained on the Mo(CO)6 precursor and compared to those of the supernatant
from the first wash in the MoC1−x nanoparticle work-up procedure (Figure 3.11). From this
analysis, we conclude that there is no evidence of unreacted Mo(CO)6 precursor in the product
stream when the isolated yield is not quantitative. This suggests that in those cases the precursor
converts to clusters or ultrasmall nanoparticles that are lost in the nanoparticle work-up.
Nonetheless, the surface response can model throughput in our experimental parameter space,
including regions where the isolated yield is lower, as evidenced by the excellent agreement
between the model and experimental data.

Figure 3.9. (a) Powder XRD pattern and (b) TEM image of the MoC1−x nanoparticles produced
under optimized conditions, and (c) powder XRD pattern and (d) TEM image of the MoC1−x
nanoparticles produced under unoptimized conditions.
107

Figure 3.10. (a) HRTEM and (b) SAED pattern of MoC1−x nanoparticles synthesized under
optimized conditions. (c) HRTEM and (d) SAED pattern of MoC1−x nanoparticles synthesized
under batch conditions. The lattice fringes correspond to the (111) and (200) planes of the 𝑎-phase.

Figure 3.11. FT-IR spectra of Mo(CO)
6 precursor and the supernatant of the first wash of the
MoC1−x nanoparticles (in the workup procedure). The spectra show the representative 𝜂(CO)
stretching region, demonstrating that there is no unreacted Mo(CO)
6 precursor in the supernatant
resulting from the product stream.

108
Finally, the MoC1−x nanoparticles were supported on a carbon support and evaluated as
catalysts in the thermocatalytic reduction of CO2 using procedures and reaction conditions similar
to those of our previous report (300 ˚C, 2 MPa, weight hourly space velocity (WHSV) based on
Mo content of 40 h
−1
and H2:CO2 molar ratio in the feed of 2.7).
21
Two catalysts were tested, one
synthesized under the optimized throughput conditions described here (termed “Optimized Flow”
MoC1−x/C), and one synthesized via the previously reported batch method (“Batch” MoC1−x/C).
As presented in Figure 3.12a for the conversion as a function of time-on-stream (TOS), both
catalysts demonstrated an induction period where conversion increased from ca. 17−21% over a
period of 15 h TOS and then maintained stable activity without signs of deactivation over the next
5 h. This activity trend was nearly identical to the performance observed previously for our C-
supported nanoparticle MoC1−x catalyst.
21
Major products from the CO2 hydrogenation reaction
under these conditions were CO, methane, and a mixture of C2+ hydrocarbons (Figure 3.12b). The
C2+ hydrocarbons were composed mostly of C2−5 alkanes, which are characteristic of the products
from CO2 conversion through the Fischer−Tropsch reaction on molybdenum carbide catalysts.
Both catalysts exhibited a similar product distribution to the previously reported study,
21
with
comparable selectivity to C2+ hydrocarbon products observed over the optimized flow MoC1−x/C
catalyst (11.7%) and the batch MoC1−x/C catalyst (10.7%).

109

Figure 3.12. (a) CO2 conversion as a function of TOS and (b) product selectivity taken as an
average of data from 16−20 h. Reaction conditions were 300 ˚C, 2 MPa, WHSV based on Mo
content of 40 h
−1
and H2:CO2 molar ratio in the feed of 2.7.

3.4. Experimental Procedures/Methods
3.4.1. Continuous Flow Synthesis of MoC1−x Nanoparticles
Oleylamine (70% technical grade) and 1-octadecene (90%) were purchased from Sigma-
Aldrich and dried by heating to 120 ˚C under vacuum for ca. 5 h prior to use. Mo(CO)6 (98%) was
purchased from Sigma-Aldrich and used as received. In a typical synthesis, the custom- made glass
millifluidic reactor was placed inside a furnace (Paragon Kiln Frog), equilibrated to the desired
reaction temperature, and pressurized to 20 psig with N2(g) prior to each synthesis. A precursor
solution was prepared by mixing an appropriate portion of Mo(CO)6 with oleylamine and/or
octadecene, based on the desired vol% of oleylamine relative to the total solvent volume, and
heating the mixture to 140 ˚C for 1 h under N2. The solution was cooled to 100 ˚C and carefully
transferred to a glass syringe fitted with heat tape maintained at 80 ˚C to maintain a homogeneous
precursor solution. The 50 mL glass syringe (Hamilton) was fastened to a syringe pump (Harvard
Apparatus PHD 2000). The syringe was connected to a 37 mL custom-made coiled borosilicate
glass reactor (1.8 mm ID) inside of the furnace using PTFE tubing (1/16” ID, 1/8” OD) outside of
110
the furnace. A check valve was integrated in the inlet tubing to prevent any backflow that may
arise from the gas evolution that is characteristic in this reaction. The reactor was maintained at 20
psig (Fluigent) throughout the reaction to minimize the evolution of CO. The resulting MoC1−x
nanoparticle product suspension (10 mL) was combined with hexanes in a 1:5 (vol/vol) ratio of
hexanes/ reaction mixture, before being transferred equally into two 50 mL centrifuge tubes. The
reaction mixtures were vortex-mixed and bath- sonicated before being precipitated each with 35
mL acetone through centrifugation (6000 rpm, 20 min). The colorless supernatant was decanted
and discarded, and the black nanoparticle pellet was redispersed by adding 0.5 mL CHCl3. After
vortex mixing and bath sonicating the product suspension, the nanoparticles were reprecipitated
using 39 mL of ethanol through centrifugation (6000 rpm, 10 min). This washing step with CHCl3
and ethanol was performed once more. The resulting nanoparticle pellet was redispersed in CHCl3
and dried overnight under flowing N2(g) for further characterization. Nanoparticle work-up was
performed identically for all samples. The purification procedure does not have a significant effect
on throughput, as the isolated yield of MoC1−x after one purification cycle is the same as that after
three purification cycles.
3.4.2. Product Characterization
XRD patterns were collected on a Rigaku Ultima IV diffractometer operating at 40 mA
and 44 kV with a Cu Ka X-ray source (λ = 1.5406 Å). Samples used for XRD were prepared by
drop-casting CHCl3 suspensions of nanoparticles onto a glass sample holder. TEM and high-
resolution TEM (HRTEM) images were acquired with a JEOL JEM2100F (JEOL Ltd.) microscope
operating at 200 kV. Selected area electron diffraction (SAED) images were acquired with an FEI
Talos F200C G2 (Thermo Fisher Scientific) microscope operating at 200 kV. Each sample was
drop- cast on 400 mesh Cu grids coated with a lacey carbon film (Ted Pella, Inc.) and dried
111
overnight under vacuum at room temperature. TGA of the MoC1−x nanoparticles was performed
on a TGA Q50 instrument. The organic-corrected isolated yield of the nanoparticles from each
reaction was gravimetrically calculated via TGA. To determine the organic ligand content, ca. 5
mg of the resulting nanoparticles was isolated after workup and drying and heated to 650 °C under
flowing N
2 at a heating rate of 10 ˚C min
-1
. FT-IR spectra were collected using a Bruker Vertex
80 spectrophotometer using 16 scans, 4 cm
-1
resolution, 4000-400 cm
-1
range, and absorbance units
as the operational parameters.
3.4.3. Reactor Temperature Control
The temperature of the furnace (Paragon W18) is maintained by a built-in controller system
(Sentinel). Multiple temperature points were evaluated for a given precursor batch (specific
concentration and amount of oleylamine). For every precursor batch, the temperature was initially
set to the lower coded value. Once the product was collected, and the system flushed, the
temperature was raised to the higher coded value and the system was allowed to equilibrate before
reacting the remaining precursor. By using different flow rates at the higher and lower temperature
values, two flow rates and two temperatures could be evaluated for each precursor batch.
A thermal insulator at the inlet of the furnace blocked convective heat emerging from the
furnace and prevented thermal expansion of the connectors and fittings going to the reactor. The
insulation also minimized undesired precursor preheating prior to reactor entry. Maintaining a
homogenous and stable temperature profile across the reactor is essential for accurate
representation of the reaction condition under investigation. This is particularly crucial for
screening reactions as misinterpreted input variables may influence the set reaction bounds
hindering accurate optimization procedures.

112
3.4.4. Experimental Screening Design and Surface Response Methodology
Design of experiments and response surface methodology were implemented in the system
using previous methods described by Williamson et al.
1
The evaluated factors include reaction
temperature, flow rate, amount of oleylamine, and precursor concentration. The bounds of these
factors, or the range of each variable that still produces viable MoC1−x nanoparticles in the flow
reactor, were set based on the limits of the flow reactor and exploratory boundary reactions, which
were verified via powder XRD analysis. The bounds and their upper and lower limits, coded +1
and -1, respectively, are shown in Table 3.1. Statgraphics Centurion XVI software (Statistical
Graphics, Rockville, MD, USA)
2
as then used to create and analyze a screening design of these
four variables to elucidate their effects on throughput, residence time, and isolated yield. The
implemented design was a full factorial, which consisted of 2
4
(16) reactions with a resolution V,
along with four replicates for statistical significance. The screening matrix can be found in Tables
3.3 and 3.4. The degrees of freedom that will be available for estimating experimental error
include: total error, which includes degrees of freedom that could have been used to estimate
effects that are not in the current model, and pure error which comes only from replicated runs. In
this case, the total error has nine degrees of freedom, while there are three degrees of freedom for
pure error. The correlation matrix shows the correlations amongst the columns of the design
matrix. Any non-zero terms imply that the estimates of the effects corresponding to that row and
column will be correlated. In this case, there are 21 pairs of columns with non-zero correlations.
However, since none are ≥ 0.5, the results can be interpreted without much difficulty. Leverage is
a statistic which measures how influential each run is in determining the coefficients of the
estimated model. In this case, an average run would have a leverage value equal to 0.55. Runs with
113
high leverage will have an unusually large impact on the fitted model compared to other runs.
There are no runs with more than 3 times the average leverage, indicating a balanced design.
To analyze the results of the screening design, we use the ANOVA. The analysis of
throughput is depicted in Table 3.7. The R
2
statistic indicates that the model as fitted explains
90.9% of the variability in throughput. The adjusted R
2
statistic, which is more suitable for
comparing models with different numbers of independent variables, is 80.7%. The standard error
of the estimate shows the standard deviation of the residuals to be 0.273. The mean absolute error
(MAE) of 0.161 is the average value of the residuals. The Durbin-Watson (DW) statistic tests the
residuals to determine if there is any significant correlation based on the order in which they occur
in the data file. Since the P-value is > 5.0%, there is no indication of serial autocorrelation in the
residuals at the 5.0% significance level.

Table 3.7. Analysis of variance for the screening of throughput.
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A: Flow rate (mL h
–1
) 1.42 1 1.42 19.03 0.0018
B: Temperature (°C) 0.309 1 0.309 4.13 0.0725
C: Amount of oleylamine (%vol) 1.06 1 1.06 14.14 0.0045
D: Concentration (mM) 1.76 1 1.76 23.48 0.0009
AB 0.0467 1 0.0467 0.62 0.4496
AC 0.454 1 0.454 6.07 0.0359
AD 0.953 1 0.953 12.75 0.0060
BC 0.0144 1 0.0144 0.19 0.6708
BD 0.12 1 0.12 1.61 0.2364
CD 0.825 1 0.825 11.04 0.0089
Total error 0.673 9 0.0748
Total (corr.) 7.36 19
R-squared = 90.9%
R-squared (adjusted for d.f.) = 80.7%
Standard error of est. = 0.273
Mean absolute error = 0.161
Durbin-Watson statistic = 1.67 (P=0.2204)
Lag 1 residual autocorrelation = 0.128

114
The analysis of isolated yield is depicted in Table 3.8. The R
2
statistic indicates that the
model as fitted explains 77.4% of the variability in isolated yield. The adjusted R
2
statistic, which
is more suitable for comparing models with different numbers of independent variables, is 52.3%.
The standard error of the estimate shows the standard deviation of the residuals to be 18.7. The
mean absolute error (MAE) of 10.2 is the average value of the residuals. The Durbin-Watson (DW)
statistic tests the residuals to determine if there is any significant correlation based on the order in
which they occur in the data file. Since the P-value is > 5.0%, there is no indication of serial
autocorrelation in the residuals at the 5.0% significance level.

Table 3.8. Analysis of variance for the screening of isolated yield.
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A: Flow rate (mL h
–1
) 579 1 579 1.66 0.2300
B: Temperature (°C) 3.18E3 1 3.18E3 9.11 0.0145
C: Amount of oleylamine
(%vol)
5.05E3 1 5.05E3 14.47 0.0042
D: Concentration (mM) 835 1 835 2.39 0.1563
AB 92.8 1 92.8 0.27 0.6184
AC 63.5 1 63.5 0.18 0.6797
AD 0.0343 1 0.0343 0.00 0.9923
BC 3.57 1 3.57 0.01 0.9216
BD 34.7 1 34.7 0.10 0.7596
CD 1.09E3 1 1.09E3 3.13 0.1106
Total error 3.14E3 9 349
Total (corr.) 1.39E4 19
R-squared = 77.4%
R-squared (adjusted for d.f.) = 52.3%
Standard Error of Est. = 18.7
Mean absolute error = 10.2
Durbin-Watson statistic = 2.02 (P=0.4947)
Lag 1 residual autocorrelation = -0.0486

The analysis of residence time is depicted in Table 3.9. The R
2
statistic indicates that the
model as fitted explains 80.0% of the variability in residence time. The adjusted R
2
statistic, which
is more suitable for comparing models with different numbers of independent variables, is 57.8%.
The standard error of the estimate shows the standard deviation of the residuals to be 5.4. The
115
mean absolute error (MAE) of 2.99 is the average value of the residuals. The Durbin-Watson (DW)
statistic tests the residuals to determine if there is any significant correlation based on the order in
which they occur in the data file. Since the P-value is > 5.0%, there is no indication of serial
autocorrelation in the residuals at the 5.0% significance level.

Table 3.9. Analysis of variance for residence time.
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A: Flow rate (mL h
–1
) 919 1 919 31.49 0.0003
B: Temperature (°C) 7.2 1 7.2 0.25 0.6313
C: Amount of oleylamine (%vol) 26.2 1 26.2 0.90 0.3682
D: Concentration (mM) 1.88 1 1.88 0.06 0.8055
AB 0.0145 1 0.0145 0.00 0.9827
AC 10.9 1 10.9 0.37 0.5557
AD 0.254 1 0.254 0.01 0.9277
BC 2.47 1 2.47 0.08 0.7777
BD 11.1 1 11.1 0.38 0.5522
CD 51.4 1 51.4 1.76 0.2172
Total error 263 9 29.2
Total (corr.) 1.31E3 19
R-squared = 80.0%
R-squared (adjusted for d.f.) = 57.8%
Standard Error of Est. = 5.4
Mean absolute error = 2.99
Durbin-Watson statistic = 2.66 (P=0.9163)
Lag 1 residual autocorrelation = -0.4

Surface response methodology was used to optimize the system based on the input goal of
maximizing throughput. A Doehlert matrix for the three significant input variables identified by
the screening (amount of oleylamine, precursor concentration, and flow rate) was implemented to
investigate each variable at a higher number of levels based on their significance in affecting
throughput – precursor concentration was investigated at seven levels, flow rate was investigated
at five levels, and amount of oleylamine was investigated at three levels, for a total of 13 reactions.
Two random points in the 3D parameter space as well as the center point (e.g., the base reaction,
coded (0,0,0)) were performed in triplicate to assess and accurately account for the run variability.
116
The coded matrix along with the corresponding real values for the optimization is given in Tables
3.5 and 3.6. The resulting data from the illustrated optimization reactions was used to create a
quadratic equation that modeled the parameter space of throughput as a function of the three input
variables, and optimal reaction conditions were predicted. The model predictions were tested in
triplicate and compared to the predicted values for accuracy and precision.
Statistical models were fit to the response variable. Models with P-values below 0.05, of
which there is one, indicate that the model as fit is statistically significant at the 5.0% significance
level. Also of interest, is the R
2
statistic, which shows the percentage of variation in the response
that has been explained by the fitted model (Table 3.10). The adjusted R
2
statistic, which is more
suitable for comparing models with different numbers of independent variables, is 87.7%. The
standard error of the estimate shows the standard deviation of the residuals to be 0.161. The mean
absolute error (MAE) of 0.0985 is the average value of the residuals. The Durbin-Watson (DW)
statistic tests the residuals to determine if there is any significant correlation based on the order in
which they occur in the data file. Since the P-value > 5.0%, there is no indication of serial
autocorrelation in the residuals at the 5.0% significance level.

Table 3.10. Analysis of variance for the optimization of throughput.
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A: Flow rate (mL h
–1
) 1.32 1 1.32 50.86 0.0000
B: Amount of oleylamine (%vol) 0.743 1 0.743 28.67 0.0000
C: Concentration (mM) 1.92 1 1.92 74.11 0.0000
AA 0.00674 1 0.00674 0.26 0.6163
AB 0.103 1 0.103 3.99 0.0611
AC 0.605 1 0.605 23.32 0.0001
BB 0.0113 1 0.0113 0.44 0.5166
BC 0.603 1 0.603 23.26 0.0001
CC 0.000879 1 0.000879 0.03 0.8560
Total error 0.467 18 0.0259
Total (corr.) 5.68 27
R-squared = 91.8 percent
R-squared (adjusted for d.f.) = 87.7 percent
Standard Error of Est. = 0.161
Mean absolute error = 0.0984
Durbin-Watson statistic = 2.26 (P=0.582)
Lag 1 residual autocorrelation = -0.166
117

3.4.5. Catalytic Testing
MoC1−x nanoparticles were supported on Vulcan XC 72 R carbon from a batch preparation
and the optimized throughput flow preparation, with Mo contents of 3.54 and 5.10 wt%,
respectively. Activity and selectivity of the MoC1−x nanoparticle catalysts were evaluated for the
CO2 hydrogenation reaction following similar conditions described in our previous report.
3
Specifically, 0.70–1.0 g of supported catalyst was loaded in a 1⁄4” ID stainless steel reactor and
pretreated under 95% H
2
/5% Ar flow (50 sccm) at 450 ˚C for 2 h before cooling to 300 ˚C for
reaction. Flowrates for CO2 and 95% H2/5% Ar were adjusted to achieve the same weight-hourly
space velocity (WHSV) of ca. 40 gCO gMo
-1
h
-1

based on metal composition (i.e., weight loading
of Mo) for each catalyst and the same feed gas composition of 26:70:4 mol% for CO
2
:H2:Ar,
respectively (corresponding to a molar H
2
:CO
2 ratio in the feed of 2.7). Product analysis was
performed online by an Agilent Technologies 7890B gas chromatograph equipped with flame
ionization detectors (FIDs) and a thermal conductivity detector (TCD). Conversion was calculated
as Σ(molar flow rate of C in all products)/(molar flow rate of inlet CO
2
). The C-selectivity of
product i was calculated as (molar flow rate of C in product (i)/Σ(molar flow rate of C in all
products).

3.5. Conclusions
We successfully utilized a millifluidic continuous flow system coupled with DoE and
predictive RSM to optimize the throughput of α- MoC1−x nanoparticle production. This is the first
report, to the best of our knowledge, of a colloidal nanoparticle synthetic throughput optimization
using DoE in a continuous flow system. Through the investigation of four experimental
118
variables—temperature, volume fraction of oleylamine, precursor concentration, and syringe
pump flow rate—we performed a first-order full factorial design, which allowed for robust
information to be accessed regarding the statical significance of each variable in affecting three
different responses (i.e., isolated yield, residence time, and throughput). This comprehensive
statistical screening provided insight into singular variable effects on each response, as well as
binary interaction effects between the variables, which are nearly impossible to identify using
traditional OVAT methods. For example, throughput (the response with the most immediate
relevance for scale-up) was affected by three of the four experimental variables (the high levels of
the volume fraction of oleylamine, precursor concentration, and flow rate), in addition to the binary
interactions between each of these variables. All three of these variables and their interactions were
significant to the 95% confidence interval. The deconvolution of these variable interactions,
including the secondary positive effects on throughput from the interactions of the high levels of
each variable and the insignificance of temperature, would not be possible without the use of this
kind of data driven screening method.
Upon completion of the screening design, a Doehlert uniform-shell optimization design to
maximize throughput with the three statistically significant variables (vide supra) was employed,
keeping reaction temperature constant at the low level. Each of these factors was investigated at
different levels, based on their significance, to fit the experimental data to a quadratic equation
that modeled throughput as a function of the investigated factors via linear regression (RSM). The
resulting model equation was graphically represented as a response surface function with the
ability to identify the optimized reaction conditions to maximize the throughput, as well as the
synthetic conditions for any given throughput. In this case, a DoE optimization is superior to other
simplex algorithms and machine learning methods because of the novelty of this material. Because
119
there is only one previous report of colloidal α-MoC1−x nanoparticles in the literature,
21

optimization designs that require large data sets would not be feasible.
55,56
Using DoE in
conjunction with RSM for this system results in incredibly accurate predictive power of the desired
responses with a minimal number of reactions, given the limited prior knowledge of the design
space as a whole.
We achieved an average experimental optimized throughput of 2.2 g h
−1
. This throughput
represents a threefold increase over the throughput achieved with the base reaction conditions.
Coupling DoE with continuous flow methods also drastically increases throughput over the lab-
scale batch reaction, which had a throughput of only 0.047 g h
−1
, for a total improvement of almost
50×. This impressive increase in throughput demonstrates that the optimization through DoE and
predictive RSM unlocks the robust exploration of a given synthetic parameter space in a way that
is not possible using OVAT methods. The achievable throughput can be further increased through
parallelization of the millifluidic reactor.
29,57
Considering a 16-channel parallel reactor, the
achieved throughput with linear scaling could reach 52 g day
−1
, or 0.4 kg week
−1
. This provides a
clear pathway to scaling the production to industrially relevant quantities.
58,59
Catalytic testing in
the CO2 hydrogenation reaction confirmed comparable activity and selectivity of the MoC1−x
nanoparticles produced using the conditions for optimized throughput compared to our initial
report of low throughput batch synthesis.
21
The implications of this study further promote the
potential of scaling up colloidal nanoparticle catalysts at the industrial level.

120
3.6. References
(1) Marinas, A.; Sheldon, R. A. Utilisation of Biomass for Fuels and Chemicals: The Road to
Sustainability. Catal. Today 2011, 167, 1−2.

(2) Aoki, Y.; Tominaga, H.; Nagai, M. Hydrogenation of CO on Molybdenum and Cobalt
Molybdenum Carbide Catalysts—Mass and Infrared Spectroscopy Studies. Catal. Today
2013, 215, 169−175.

(3) Ranhotra, G. S.; Haddix, G. W.; Bell, A. T.; Reimer, J. A. Catalysis over Molybdenum Carbides
and Nitrides: I Catalyst Characterization. J. Catal. 1987, 108, 24−39.

(4) Ranhotra, G. S.; Bell, A. T.; Reimer, J. A. Catalysis over Molybdenum Carbides and Nitrides:
II. Studies of CO Hydrogenation and C2H6
Hydrogenolysis. J. Catal. 1987, 108, 40−49.

(5) Liu, C.; Lin, M.; Fang, K.; Meng, Y.; Sun, Y. Preparation of Nanostructured Molybdenum
Carbides for CO Hydrogenation. RSC Adv. 2014, 4, 20948−20954.

(6) Shou, H.; Davis, R. J. Multi-Product Steady-State Isotopic Transient Kinetic Analysis of CO
Hydrogenation over Supported Molybdenum Carbide. J. Catal. 2013, 306, 91−99.

(7) Ribeiro, F. H.; Boudart, M.; Dalla Betta, R. A.; Iglesia, E. Catalytic Reactions of N-Alkanes
on β-W
2
C and WC: The Effect of Surface Oxygen on Reaction Pathways. J. Catal. 1991,
130, 498−513.

(8) Phamhuu, C.; Ledoux, M. J.; Guille, J. Reactions of 2- and 3- Methylpentane,
Methylcyclopentane, Cyclopentane, and Cyclohexane on Activated Mo
2
C. J. Catal. 1993,
143, 249−261.

(9) Iglesia, E.; Baumgartner, J. E.; Ribeiro, F. H.; Boudart, M. Bifunctional Reactions of Alkanes
on Tungsten Carbides Modified by Chemisorbed Oxygen. J. Catal. 1991, 131, 523−544.

(10) Ribeiro, F. H.; Dalla Betta, R. A.; Boudart, M.; Baumgartner, J.; Iglesia, E. Reactions of
Neopentane, Methylcyclohexane, and 3,3-Dimethylpentane on Tungsten Carbides: The
Effect of Surface Oxygen on Reaction Pathways. J. Catal. 1991, 130, 86−105.

(11) Sullivan, M. M.; Chen, C.-J.; Bhan, A. Catalytic Deoxygenation on Transition Metal Carbide
Catalysts. Catal. Sci. Technol. 2016, 6, 602−616.

(12) Lin, Z.; Chen, R.; Qu, Z.; Chen, J. G. Hydrodeoxygenation of Biomass-Derived Oxygenates
over Metal Carbides: From Model Surfaces to Powder Catalysts. Green Chem. 2018, 20,
2679−2696.

(13) Levy, R. B.; Boudart, M. Platinum-Like Behavior of Tungsten Carbide in Surface Catalysis.
Science 1973, 181, 547−549.

121
(14) Volpe, L.; Boudart, M. Compounds of Molybdenum and Tungsten with High Specific Surface
Area: I Nitrides. J. Solid State Chem. 1985, 59, 332−347.

(15) Lee, J. S.; Oyama, S. T.; Boudart, M. Molybdenum Carbide Catalysts: I Synthesis of
Unsupported Powders. J. Catal. 1987, 106, 125−133.

(16) Iglesia, E.; Ribeiro, F. H.; Boudart, M.; Baumgartner, J. E. Synthesis, Characterization, and
Catalytic Properties of Clean and Oxygen-Modified Tungsten Carbides. Catal. Today 1992,
15, 307− 337.

(17) Vo, D.-V. N.; Arcotumapathy, V.; Abdullah, B.; Adesina, A. A. Non-Linear ASF Product
Distribution over Alkaline-Earth Promoted Molybdenum Carbide Catalysts for Hydrocarbon
Synthesis. Catal. Today 2013, 214, 42−49.

(18) Vo, D.-V. N.; Nguyen, T.-H.; Kennedy, E. M.; Dlugogorski, B. Z.; Adesina, A. A.
Fischer−Tropsch Synthesis: Effect of Promoter Type on Alumina-Supported Mo Carbide
Catalysts. Catal. Today 2011, 175, 450−459.

(19) Alaba, P. A.; Abbas, A.; Huang, J.; Daud, W. M. A. W. Molybdenum Carbide Nanoparticle:
Understanding the Surface Properties and Reaction Mechanism for Energy Production
towards a Sustainable Future. Renew. Sustain. Energy Rev. 2018, 91, 287−300.

(20) Hyeon, T.; Fang, M.; Suslick, K. S. Nanostructured Molybdenum Carbide: Sonochemical
Synthesis and Catalytic Proper- ties. J. Am. Chem. Soc. 1996, 118, 5492−5493.

(21) Baddour, F. G.; Roberts, E. J.; To, A. T.; Wang, L.; Habas, S. E.; Ruddy, D. A.; Bedford, N.
M.; Wright, J.; Nash, C. P.; Schaidle, J. A.; Brutchey, R. L.; Malmstadt, N. An Exceptionally
Mild and Scalable Solution-Phase Synthesis of Molybdenum Carbide Nanoparticles for
Thermocatalytic CO2 Hydrogenation. J. Am. Chem. Soc. 2020, 142, 1010−1019.

(22) Zhang, L.; Xia, Y. Scaling up the Production of Colloidal Nanocrystals: Should We Increase
or Decrease the Reaction Volume? Adv. Mater. 2014, 26, 2600−2606.

(23) Roberts, E. J.; Karadaghi, L. R.; Wang, L.; Malmstadt, N.; Brutchey, R. L. Continuous Flow
Methods of Fabricating Catalytically Active Metal Nanoparticles. ACS Appl. Mater.
Interfaces 2019, 11, 27479−27502.

(24) Jensen, K. F. Microreaction Engineering – Is Small Better? Chem. Eng. Sci. 2001, 56,
293−303.

(25) Focus. Lab Chip 2004, 4 (), 11N−15N, DOI: 10.1039/ b403638g.

(26) Sui, J.; Yan, J.; Liu, D.; Wang, K.; Luo, G. Continuous Synthesis of Nanocrystals via Flow
Chemistry Technology. Small 2020, 16, No. 1902828.

122
(27) deMello, A. J. Control and Detection of Chemical Reactions in Microfluidic Systems. Nature
2006, 442, 394−402.

(28) Roberts, E. J.; Habas, S. E.; Wang, L.; Ruddy, D. A.; White, E. A.; Baddour, F. G.; Griffin,
M. B.; Schaidle, J. A.; Malmstadt, N.; Brutchey, R. L. High-Throughput Continuous Flow
Synthesis of Nickel Nanoparticles for the Catalytic Hydrodeoxygenation of Guaiacol. ACS
Sustainable Chem. Eng. 2017, 5, 632−639.

(29) Riche, C. T.; Roberts, E. J.; Gupta, M.; Brutchey, R. L.; Malmstadt, N. Flow Invariant Droplet
Formation for Stable Parallel Microreactors. Nat. Commun. 2016, 7, 10780.

(30) Maceiczyk, R. M.; deMello, A. J. Fast and Reliable Metamodeling of Complex Reaction
Spaces Using Universal Kriging. J. Phys. Chem. C 2014, 118, 20026−20033.

(31) Li, S.; Baker, W. R.; Lignos, I.; Yang, Z.; Stavrakis, S.; Howes, P. D.; de Mello, A. J.
Automated Microfluidic Screening of Ligand Interactions during the Synthesis of Cesium
Lead Bromide Nanocrystals. Mol. Syst. Des. Eng. 2020, 5, 1118−1130.

(32) Miller, P. W.; Jennings, L. E.; deMello, A. J.; Gee, A. D.; Long, N. J.; Vilar, R. A Microfluidic
Approach to the Rapid Screening of Palladium-Catalysed Aminocarbonylation Reactions.
Adv. Synth. Catal. 2009, 351, 3260−3268.

(33) Holmes, N.; Akien, G. R.; Blacker, A. J.; Woodward, R. L.; Meadows, R. E.; Bourne, R. A.
Self-Optimisation of the Final Stage in the Synthesis of EGFR Kinase Inhibitor AZD9291
Using an Automated Flow Reactor. React. Chem. Eng. 2016, 1, 366−371.

(34) Abdel-Latif, K.; Epps, R. W.; Bateni, F.; Han, S.; Reyes, K. G.; Abolhasani, M. Self-Driven
Multistep Quantum Dot Synthesis Enabled by Autonomous Robotic Experimentation in
Flow. Adv. Intell. Syst. 2021, 3, No. 2000245.

(35) Mateos, C.; Nieves-Remacha, M. J.; Rincón, J. A. Automated Platforms for Reaction Self-
Optimization in Flow. React. Chem. Eng. 2019, 4, 1536−1544.

(36) Hall, B. L.; Taylor, C. J.; Labes, R.; Massey, A. F.; Menzel, R.; Bourne, R. A.; Chamberlain,
T. W. Autonomous Optimisation of a Nanoparticle Catalysed Reduction Reaction in
Continuous Flow. Chem. Commun. 2021, 57, 4926−4929.

(37) Shields, B. J.; Stevens, J.; Li, J.; Parasram, M.; Damani, F.; Alvarado, J. I. M.; Janey, J. M.;
Adams, R. P.; Doyle, A. G. Bayesian Reaction Optimization as a Tool for Chemical
Synthesis. Nature 2021, 590, 89−96.

(38) Schweidtmann, A. M.; Clayton, A. D.; Holmes, N.; Bradford, E.; Bourne, R. A.; Lapkin, A.
A. Machine Learning Meets Continuous Flow Chemistry: Automated Optimization towards
the Pareto Front of Multiple Objectives. Chem. Eng. J. 2018, 352, 277−282.

123
(39) Fhionnlaoich, N. M.; Yang, Y.; Qi, R.; Galvanin, F. DoE-It-Yourself: A Case Study for
Implementing Design of Experiments into Nanoparticle Synthesis, DOI:
10.26434/chemrxiv.8198420.v1.

(40) Williamson, E. M.; Tappan, B. A.; Mora-Tamez, L.; Barim, G.; Brutchey, R. L. Statistical
Multiobjective Optimization of Thiospinel CoNi2S4 Nanocrystal Synthesis via Design of
Experiments. ACS Nano 2021, 15, 9422−9433.

(41) Mora-Tamez, L.; Barim, G.; Downes, C.; Williamson, E. M.; Habas, S. E.; Brutchey, R. L.
Controlled Design of Phase- and Size- Tunable Monodisperse Ni2P Nanoparticles in a
Phosphonium-Based Ionic Liquid through Response Surface Methodology. Chem. Mater.
2019, 31, 1552−1560.

(42) Weissman, S. A.; Anderson, N. G. Design of Experiments (DoE) and Process Optimization.
A Review of Recent Publications. Org. Process Res. Dev. 2015, 19, 1605−1633.

(43) Braham, E. J.; Davidson, R. D.; Al-Hashimi, M.; Arróyave, R.; Banerjee, S. Navigating the
Design Space of Inorganic Materials Synthesis Using Statistical Methods and Machine
Learning. Dalton Trans. 2020, 49, 11480−11488.

(44) Burrows, N. D.; Harvey, S.; Idesis, F. A.; Murphy, C. J. Understanding the Seed-Mediated
Growth of Gold Nanorods through a Fractional Factorial Design of Experiments. Langmuir
2017, 33, 1891−1907.

(45) Liu, Z.; Zhu, Y.; Gao, P.; Zhang, Z. Application of the Factorial Design of Experiments to
Hydrothermal Synthesis of Lithium Iron Phosphate. Int. J. Appl. Ceram. Technol. 2020, 17,
1231−1240.

(46) Braham, E.J.; Cho, J.; Forlano, K.M.; Watson, D.F.; Arroỳave, R.; Banerjee, S. Machine
Learning-Directed Navigation of Synthetic Design Space: A Statistical Learning Approach
to Controlling the Synthesis of Perovskite Halide Nanoplatelets in the Quantum-Confined
Regime. Chem. Mater. 2019, 31, 3281−3292.

(47) Mourdikoudis, S.; Liz-Marzán, L. M. Oleylamine in Nano- particle Synthesis. Chem. Mater.
2013, 25, 1465−1476.

(48) Song, H.; Tice, J. D.; Ismagilov, R. F. A Microfluidic System for Controlling Reaction
Networks in Time. Angew. Chem., Int. Ed. 2003, 42, 768−772.

(49) Yen, B. K. H.; Günther, A.; Schmidt, M. A.; Jensen, K. F.; Bawendi, M. G. A Microfabricated
Gas−Liquid Segmented Flow Reactor for High-Temperature Synthesis: The Case of CdSe
Quantum Dots. Angew. Chem., Int. Ed. 2005, 44, 5447−5451.

(50) Box, G. E. P.; Hunter, J. S.; Hunter, W. G. Statistics for Experimenters: Design, Innovation,
and Discovery, 2nd ed.; Wiley series in probability and statistics; Wiley-Interscience:
Hoboken, N.J, 2005.
124

(51) Vera Candioti, L.; De Zan, M. M.; Cámara, M. S.; Goicoechea, H. C. Experimental Design
and Multiple Response Optimization. Using the Desirability Function in Analytical Methods
Development. Talanta 2014, 124, 123−138.

(52) Bezerra, M. A.; Santelli, R. E.; Oliveira, E. P.; Villar, L. S.; Escaleira, L. A. Response Surface
Methodology (RSM) as a Tool for Optimization in Analytical Chemistry. Talanta 2008, 76,
965−977.

(53) Doehlert, D. H. Uniform Shell Designs. Appl. Stat. 1970, 19, 231.

(54) Meyer, R. K.; Nachtsheim, C. J. The Coordinate-Exchange Algorithm for Constructing Exact
Optimal Experimental Designs. Technometrics 1995, 37, 60−69.

(55) Fath, V.; Kockmann, N.; Otto, J.; Röder, T. Self-Optimising Processes and Real-Time-
Optimisation of Organic Syntheses in a Microreactor System Using Nelder−Mead and
Design of Experiments. React. Chem. Eng. 2020, 5, 1281−1299.

(56) Breen, C. P.; Nambiar, A. M. K.; Jamison, T. F.; Jensen, K. F. Ready, Set, Flow! Automated
Continuous Synthesis and Optimization. Trends Chem. 2021, 3, 373−386.

(57) Wang, L.; Karadaghi, L. R.; Brutchey, R. L.; Malmstadt, N. Self-Optimizing Parallel
Millifluidic Reactor for Scaling Nanoparticle Synthesis. Chem. Commun. 2020, 56,
3745−3748.

(58) Dong, Z.; Wen, Z.; Zhao, F.; Kuhn, S.; Noël, T. Scale-up of Micro- and Milli-Reactors: An
Overview of Strategies, Design Principles and Applications. Chem. Eng. Sci. X 2021, 10, No.
100097.

(59) Schaidle, J. A.; Habas, S. E.; Baddour, F. G.; Farberow, C. A.; Ruddy, D. A.; Hensley, J. E.;
Brutchey, R. L.; Malmstadt, N.; Robota, H. Transitioning Rationally Designed Catalytic
Materials to Real “Working” Catalysts Produced at Commercial Scale: Nanoparticle
Materials. In. Catalysis 2017, 213−281.

125
Chapter 4. Creating Ground Truth for Nanocrystal Morphology: A Fully Automated
Pipeline for Unbiased Transmission Electron Microscopy Analysis*

*Published in Nanoscale, 2022, 14, 15327-15339.

4.1. Abstract
Control over colloidal nanocrystal morphology (size, size distribution, and shape) is
important for tailoring the functionality of individual nanocrystals and their ensemble behavior.
Despite this, traditional methods to quantify nanocrystal morphology are laborious. New
developments in automated morphology classification will accelerate these analyses but the
assessment of machine learning models is limited by human accuracy for ground truth, causing
even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic
image rendering to solve the ground truth problem of nanocrystal morphology classification. By
simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space,
we trained a convolutional neural network to link unique morphologies to their simulated
parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An
automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology
from experimental transmission electron microscopy (TEM) images. Using improved computer
vision techniques, 42,650 nanocrystals were identified, assessed, and labeled with quantitative
parameters, offering a 600-fold improvement in efficiency over best-practice manual
measurements. A classification algorithm was trained with a prediction accuracy of 99.5%, which
can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The
resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr3
nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on
morphology. This pipeline provides a simple, efficient, and unbiased method to quantify
126
nanocrystal morphology and represents a practical route to construct large datasets with an
absolute ground truth for training unbiased morphology-based machine learning algorithms.

4.2. Introduction
Engineered colloidal nanocrystals are of interest because of their unique size- and shape-
dependent chemical and physical properties.
1,2
Control over nanocrystal morphology (including
nanocrystal size, size distribution, and shape) is critical to tailor and ultimately maximize the
functionality of individual nanocrystals and their ensembles.
3–6
That is, it is well established that
nanocrystal morphology has a direct impact on a myriad of functional properties, including
catalytic behavior, optoelectronic and plasmonic effects, and biological uptake for drug delivery
and imaging.
7–11
In the overwhelming majority of cases, the resulting morphologies of colloidally
prepared nanocrystals are assessed by transmission electron microscopy (TEM), which produces
an image with a 2D projection of the 3D nanocrystal. Despite the significance of nanocrystal
morphology, the published best practice for determining nanocrystal size and size distribution, for
example, relies on the analysis of very small populations (N » 300 nanocrystals).
12
This is because,
despite recent advances in analysis methods, image analysis is most frequently done by hand,
which introduces human bias, is slow and laborious, and necessarily results in small population
statistics. With advances in instrumentation, it is now possible to acquire increasingly larger
volumes of TEM images in shorter periods of time, which further exacerbates these analytical
challenges and bottlenecks.
Alternatively, image analysis can be done using public domain image processing programs,
such as ImageJ,
13
Ilastik,
14
MIPARÔ,
15
‘binary DoG’ method,
16
Py-EM and SerialEM.
17
These
provide some improvements in terms of scaling up data analysis, but nanocrystal detection is only
127
conclusive for well-defined and -separated nanocrystals. Advancements to account for noisy
backgrounds, nanocrystal agglomeration, and lower-quality images have been proposed via
incorporation of modules from different platforms, as exemplified by groups like Qian et al.,
18

Park et al.,
19
and Cervera et al.
20
However, outputs are limited to ensemble averages of a few
geometric descriptors, such as 2D area, and sacrifice irregular and concave morphologies for
segmentation accuracy.
20,21
This often leads to arbitrary, nongeometric shape descriptors being
reported in the literature, including zoomorphic terms such as “nanourchins” and
“nanotadpoles”,
22,23
because of the qualitative nature of parsing shapes. These challenges
complicate the description and quantification of nanocrystal morphology by TEM analysis.
24–27

The incorporation of image processing platforms with machine learning (ML) algorithms
presents a solution to these challenges by opening the door for automated nanocrystal morphology
classification in a statistical manner. Such advancements will enable high-throughput morphology
assessments and could lead to universal reporting to aid the incorporation of literature data into
ML datasets.
28
One of the first attempts at automated morphology classification was done by
Laramy et al.,
29
who measured the distance from the center of each nanocrystal to its edge as a
function of angle, 𝑑(𝜃). Boselli et al.
11
and Lee et al.
21
expanded on this work by quantitatively
classifying morphologies into a predetermined number of shape groups via different image
processing techniques. Most recently, AutoDetect-mNP by Wang et al. offered an unsupervised
algorithm for automated morphology analysis.
30
However, such techniques are still in their
infancy.
31–34
Generalizability limitations and human bias remain problematic, with the solution to
one usually coming at the cost of the other.
11,16,29,30
Generalizability and highly accurate
nanocrystal detection can be achieved via complex image processing techniques, but such methods
are either incapable of differentiating between the shapes and shape attributes of individual
128
nanocrystals in the sample ensemble for subsequent classification or typically require supervision
with human-generated ground truth labels to be incorporated into ML platforms
30,15,17,18,35–39

Unsupervised cluster algorithms, such as AutoDetect-mNP, create shape group classifications
based on minimized probability statistics, such as Hu moments, that ostensibly removes bias. This
is sometimes coined “soft classification” instead of the predetermined groups characteristic of
classification algorithms, but the unsupervised nature of the image processing typically lacks the
complexity needed to classify a diverse array of morphologies without multiple manual
interventions.
16,30
Either way, there is currently no standardized method of quantitatively analyzing
diverse ensembles of nanocrystals to uniquely identify shapes and the distribution of shapes.
28,40

With this in mind, a barrier in the progression of automated image analysis and
classification for colloidal nanocrystals is the absence of an unbiased morphological ground truth
when training the ML models.
36
With the ground truth for training and testing remaining in the
hands of human experts, even unsupervised models are still inherently limited to human accuracy
and rooted in human bias since the evaluation is reliant on the comparison to the status quo biased
methods. Herein, we create a ground truth for nanocrystal morphology classification by exploiting
the unbiased nature of deep learning in conjunction with the analytical assurance of simulation.
The general workflow of automated morphology classification has three main components:
(1) nanocrystal detection (segmentation), (2) feature extraction, and (3) classification. Recent
studies have utilized synthetic image rendering as a ground truth to solve the annotation problem
in segmentation of nanocrystal images via deep learning, proving to be an important step in
removing the bias in initial nanocrystal detection.
37,41,37
We extend this method to feature
extraction, thereby creating a ground truth for morphology classification. By simulating a
theoretically infinite number of individual nanocrystal morphologies via a quantifiable,
129
parametrically continuous mathematical function, we train a convolutional deep neural network
(CNN) to perform the simulation in reverse by predicting the point in parameter space that best
corresponds to each pixelated, noisy nanocrystal image. This links experimental morphologies to
a unique combination of parameters and thus defines nanocrystal morphology quantitatively rather
than qualitatively. We then create an unsupervised, automated pipeline that accurately processes,
quantitatively defines, and classifies the morphologies of nanocrystals from experimental TEM
images, utilizing the neural network output as ground truth for classification (Scheme 4.1). The
pipeline applies computer vision techniques to separate and identify ranges of concave, convex,
and agglomerated nanocrystals. To the best of our knowledge, this is the first example of a non-
human ground truth in nanocrystal morphology classification, and therefore the first example of a
pipeline that is designed to eliminate human bias. The pipeline demonstrates accuracy across a
wide range of nanocrystal morphologies and efficiently creates a viable dataset large enough to
train subsequent ML models to near perfect accuracy. This pipeline can be easily implemented by
any researcher, with a simple 5-step guide included in the MATLAB code on Github for those
with little-to-no coding background.

130

Scheme 4.1. Visualization of full pipeline for TEM image analysis.

4.3. Results and Discussion
4.3.1. Machine Learning Pipeline
The components for our algorithm include a neural network trained on simulated images
of individual nanocrystals to create a morphological ground truth, which is used to evaluate the
output of a parallel image processing algorithm that detects, separates, crops, and extracts the
morphological features of nanocrystals in experimental TEM images. The trained neural network
predicts and assigns three shape parameter values for each detected experimental nanocrystal that,
cohesively, best define the overall morphology in 2D virtual space according to the simulation
function. These shape parameters are then utilized to cluster the nanocrystals into shape groups in
an unsupervised manner, and act as ground truth labels to train morphology classification
algorithms (Scheme 4.1).

131
4.3.1.1. Image Processing
The first step in automated morphology classification is image processing. Efficient and
accurate segmentation of colloidal nanocrystals in the foreground of an experimental TEM image
is essential for accurate nanocrystal detection, feature extraction, cropping, and eventual
classification. Furthermore, the quality of the TEM images inputted into a neural network directly
affects its accuracy and precision. As a result, segmentation is a widely studied area of image
processing, with techniques ranging from simple thresholding to complex neural networks to
predict an accurate image segmentation.
37,41–44
We opt for a simple but effective approach to
increase the pipeline generalizability (Figure 4.1), described in more detail in the Methods
section. Processing concave shapes has been an ongoing problem in shape classification as
concavity is commonly used as a way to filter out overlapping nanocrystals, which inhibits the
analysis of naturally concave morphologies.
19,29,30,35,45
We address this challenge by analyzing the
connectivity and intensities of each pixel in a segmented image via MATLAB’s watershed
function, which identifies connected components as “water wells” that fill to a certain level before
the pixels begin to intersect with neighboring nanocrystals (Figure 4.1c). Each “well” therefore
indicates an individual nanocrystal center, whose edges are uniformly eroded in the segmentation
to remove any overlap with other nanocrystals while maintaining the integrity of the nanocrystal
morphology. The eroded nanocrystal mask is subsequently rebuilt in an equal but opposite manner
via dilation to obtain the correct size and shape before cropping and feature analysis. Once
segmentation is complete, 30 different features are calculated for each nanocrystal, as defined in
the Methods section. Extremely agglomerated samples where individual nanocrystals cannot be
distinguished even qualitatively can be filtered out via implementing a threshold for extreme
outliers. Although this occasionally filters out “accurate” nanocrystals, it removes the inaccurate
132
segmentations without limiting the shapes that can be accurately processed, and in our experience
the loss of the accurate nanocrystal renderings remains insignificant to the overall result. Bounding
box and centroid data are then used to visualize the accuracy of the segmentation and crop each
TEM image into smaller images of individual nanocrystals (Figure 4.1d). Lastly, the cleaned,
cropped images are compressed to normalize image size for implementation into the neural
network, in accordance with Scheme 4.1.

Figure 4.1. General workflow of the TEM image preprocessing method. (a) Original bright field
TEM image of CsPbBr3 nanocrystals. (b) Binary image segmentation after contrast adjustment,
filtering, and processing. (c) “Water wells” of pixel intensities in each connected component to
identify and separate agglomerated nanocrystals. (d) Individual nanocrystal segmentations plotted
as colors overlaid onto the original TEM image, with their respective bounding boxes used for
cropping and implementation into the neural network. Nanocrystals connected to the edges and
joined with the scale bar are removed.

133
In addition to visualizing the bounding boxes for each detected nanocrystal (Figure 4.1d),
we assessed the ability of the pipeline to handle nanocrystal agglomeration and misclassification.
Nanocrystals from a single batch of product were imaged via TEM with three different degrees of
agglomeration: non-overlapping, semi-overlapping, and almost entirely overlapping (Figure 4.2).
Each image was input into the pipeline separately to assess the effects of agglomeration on the
output. Across the three levels of agglomeration, each image was similarly classified into seven
shape groups with average sizes within a standard deviation of each other. The non-overlapping
sample had an average size of 20.0 ± 13.8 nm, the semi-overlapping sample had an average size
of 18.0 ± 8.3 nm, and the very agglomerated sample had an average size of 16.1 ± 7.4 nm. The
decreasing size and polydispersity with increasing agglomeration is concurrent with accurate
nanocrystal detection, as more agglomeration means a denser population of nanocrystals, so large
outliers have less of an influence. If agglomeration was causing a significant effect on detection,
one could assume that the size and polydispersity of the detected nanocrystals would get larger
due to agglomerated masses being classified as large, singular particles. Tables of the average and
standard deviation of shape features across each shape group were also consistent and independent
of agglomeration (Tables 4.1-4.6), further supporting that agglomeration is non-significant in
detection accuracy for the pipeline.
Note that when the pipeline is run, the color indicative of each shape group is randomly
assigned, and the groups are classified in random order. Since each image that corresponded to a
certain degree of agglomeration was run separately to assess the significance of the agglomeration
on the output, shape groups that correspond to each other across the samples do not necessarily
have the same color associated with them. For example, the light blue shape group in Figure 4.2a
is the same morphology class as the aquamarine shape group in Figure 4.2b. The same is true for
134
the tables of average predictor values and their standard deviations (Tables 4.1-4.6), and the
groups that correspond with each other across the samples are indicated via their shape group
number (groups 1-7).

Figure 4.2. (a) Non-overlapping nanocrystals and their morphology classifications. (b) Semi-
overlapping nanocrystals and their corresponding morphology classifications, with areas of
significant overlap circled in green. (c) Heavily overlapping nanocrystals and their corresponding
morphology classifications, with areas of significant overlap circled in green.

135
Table 4.1. Average values of the predictors (defined in Image processing section) for the seven
shape groups of the non-agglomerated nanocrystals of Figure 4.7a.
Group Color Area Circularity Eccentricity Major Axis Length Minor Axis Length
1 Green 61.2080 0.9531 0.6280 10.2485 7.5154
2 Light blue 208.6257 0.7199 0.8599 24.8063 11.5415
3 Dark blue 112.9653 0.9504 0.5509 13.5294 10.6825
4 Purple 660.2895 0.4518 0.9734 68.3879 13.4089
5 Red 86.0197 0.6062 0.9212 18.1527 6.2290
6 Orange 256.3082 0.4830 0.9690 39.1345 8.9627
7 Aqua/teal 125.5119 0.9681 0.6327 14.3393 10.9673

Table 4.2. Standard deviations of the predictors (defined in Image processing section) for the
seven shape groups of the non-agglomerated nanocrystals of Figure 4.7a.
Group Color Area Circularity Eccentricity Major Axis Length Minor Axis Length
1 Green 22.2829 0.0593 0.1428 2.6350 1.1759
2 Light blue 52.9922 0.0980 0.0863 4.7097 3.0564
3 Dark blue 29.8346 0.0545 0.1686 2.5719 1.3709
4 Purple 144.1394 0.1813 0.0250 15.4307 4.7701
5 Red 56.8560 0.0885 0.0533 6.9591 1.8142
6 Orange 59.9730 0.0817 0.0204 6.1476 1.8443
7 Aqua/teal 43.8154 0.0259 0.0675 2.7395 1.8323

Table 4.3. Average values of the predictors (defined in Image processing section) for the seven
shape groups of the semi-agglomerated nanocrystals of Figure 4.7b.
Group Color Area Circularity Eccentricity Major Axis Length Minor Axis Length
1 Green 43.1816 0.7699 0.7845 9.6443 5.7405
6 Light blue 195.9712 0.4834 0.8782 27.1205 11.5820
5 Dark blue 70.2223 0.5768 0.9319 16.7028 5.7581
3 Purple 85.1721 0.8547 0.5568 11.6847 9.4249
4 Red 136.5281 0.3322 0.9893 37.4135 5.0088
7 Orange 61.0211 0.8117 0.7147 10.9099 7.2687
2 Aqua/teal 122.5420 0.6456 0.7555 16.8678 10.2708

Table 4.4. Standard deviations of the predictors (defined in Image processing section) for the
seven shape groups of the semi-agglomerated nanocrystals of Figure 4.7b.
Group Color Area Circularity Eccentricity Major Axis Length Minor Axis Length
1 Green 20.6468 0.1014 0.0920 2.6485 1.5469
6 Light blue 53.2411 0.1272 0.0934 4.4696 2.9476
5 Dark blue 19.9043 0.0888 0.0318 2.9382 1.1620
3 Purple 24.0885 0.0940 0.1419 2.0817 1.6056
4 Red 28.0284 0.0786 0.0063 7.5423 0.9285
7 Orange 20.8847 0.0776 0.1299 2.0965 1.5655
2 Aqua/teal 27.6564 0.1164 0.1400 2.7732 1.6479

136
Table 4.5. Average values of the predictors (defined in Image processing section) for the seven
shape groups of the heavily agglomerated nanocrystals of Figure 4.7c.
Group Color Area Circularity Eccentricity Major Axis Length Minor Axis Length
5 Green 61.2961 0.7127 0.8435 12.7199 6.5225
7 Light blue 132.8286 0.6015 0.8170 19.0059 10.2960
1 Dark blue 65.7785 0.8722 0.5402 10.0879 8.3658
6 Purple 72.6020 0.4710 0.9629 20.4210 5.1335
4 Red 310.6312 0.3306 0.7818 30.8872 19.2308
3 Orange 78.2315 0.7875 0.7669 12.9831 8.0120
2 Aqua/teal 187.4274 0.4332 0.8970 28.4313 11.1918

Table 4.6. Standard deviations of the predictors (defined in Image processing section) for the
seven shape groups of the heavily agglomerated nanocrystals of Figure 4.7c.
Group Color Area Circularity Eccentricity Major Axis Length Minor Axis Length
5 Green 19.7509 0.1027 0.0690 2.3915 1.4191
7 Light blue 29.7497 0.1236 0.0955 2.6476 1.6164
1 Dark blue 24.7888 0.0855 0.1153 2.0974 1.8149
6 Purple 22.0770 0.0791 0.0192 4.7039 1.2750
4 Red 56.7172 0.0829 0.0373 2.7423 2.7364
3 Orange 21.3618 0.0909 0.0961 2.0502 1.4882
2 Aqua/teal 45.6478 0.1082 0.0704 5.2412 2.5300

4.3.1.2. Creating morphological ground truth
Simulated images of individual nanocrystals were used as ground truth in the training of a
convolutional neural network to directly map a unique morphology to a quantitative description.
Images were generated from a polar function (and the corresponding pre-defined parameter space)
that defines a large class of closed curves with differing degrees of radial symmetry (Figure 4.3a).
This extension of the class of ‘super-ellipses’ provides a natural way of continuously
parameterizing the curvature of the closed curves, yet is the first time it has been applied to
nanocrystal morphologies.
46
When simulating ground truth to train the neural network, parameter
values were randomly drawn from the distributions given in Table 4.7. These values were used to
parameterize and plot nanocrystal morphologies in polar space (radius R as a function of 𝜃 ∈
[0,2π]) and physically correspond to curvature (p-norm, or p), the number of major axes (d), the
ratio between the major and minor axis lengths (Lratio), and a random rotational phase shift (𝜙):
137
𝑅 = O𝐿
EFG!<
cos
H
(
(θ+ϕ)P
5
+sin
H
(
(θ+ϕ)
5

#'/5
(4.1)

This mathematical model (eq. 4.1) is physically motivated since the set of curves of uniform
length in various p-norms is identical to the set of all super-ellipses. It generates plausible
nanocrystal morphologies based on the class of Lamé curves, or the set of functions that satisfy
the p-norm being constant in 2D space.
46
Lamé curves have only two parameters but well-describe
the set of all circles, ellipses, diamonds rectangles, and superellipses (the continuously deformable
curves linking these shapes). The generating function (eq. 4.1) extends this class of functions by
introducing the physically motivated modification of the number of primary axes in the plane, (or
equivalently, a different angular velocity when generating the polar plots) as it corresponds to a
higher degree of symmetry in the nanoparticle. As such, by mathematical design, the model can
represent an extremely large class of geometries with only three parameters. The fact that this
function was chosen by the authors is a particular form of bias, but it is important to separate this
from bias as typically considered in the evaluation of these nanoparticle TEM images. The choice
of any other model would also eliminate the evaluation bias, since a non-human entity is retrieving
the ground truth – distinct from the bias associated with choosing the model to generate that ground
truth. Since it is impossible to eliminate the bias of model choice, we have chosen to make the
model as physically motivated as possible. We believe that this method of producing ground truth
and training a network to retrieve these parameters, for use in later analysis, is the closest effort
yet to an unbiased analysis of nanoparticle TEM images.

138
Table 4.7. Parameter distributions used to generate the ground truth morphologies. All three
distributions are generated mutually independently, making this a true 3D parameter space. The
limits of the distributions are chosen to produce only physically plausible geometries (e.g.,
including the uniform distribution for curvature down to 0.01 is meaningless since it is
indistinguishable from 0.05).
Parameter Distribution
Curvature (p) -1/log2(uniform[0.05,0.99])
Number of Primary Axes (d) Random integer ∈ [1,6]
Major/Minor Axis Ratio (L ratio) 1/(uniform[0.2,1])

Figure 4.3. Overview of outputs from model to generate simulated nanocrystal morphologies to
create a ground truth. (a) Demonstration of how varying individual parameters in the model while
keeping the others constant affects the generated nanocrystal shape: Upper left, p-norm; upper
right, Lratio; bottom, number of major axes d (size adjusted for clarity). (b) Sampling of simulated
nanocrystal TEM images. The labels for each image are (p, d, Lratio).

The ground truth therefore consists of each simulated image stored in a data matrix with a
labels matrix of each image’s corresponding parameter array that was used to generate the curve.
Random sampling helps avoid training the network to ‘recognize’ specific parameter values when
the experimental images can have any parameter value. Introducing a random distribution of
rotational phase shifts, intensity contrasts, pixelation, and Poisson background noise account for
the non-ideal conditions of experimental TEM images. Figure 4.3b demonstrates the
generalizability of this model through the generation of morphologies that are common to colloidal
139
nanocrystals (i.e., qualitative morphology labels such as circles, rectangles, kites, and urchins),
along with abnormal and asymmetric morphologies (Figure 4.4).

Figure 4.4. A sampling of abnormal and asymmetric simulated morphologies.

From this model, 65,000 simulated TEM images depicting distinct nanocrystal
morphologies and their corresponding ground truth parameter values (p, d, and Lratio) were
generated and used to train a convolutional neural network (CNN) in a 70/10/20 train/validate/test
split. This network optimally finds the underlying parameters of nanocrystal shape via regression
from an input TEM image. Choices regarding the model design are detailed in the Methods
section. To evaluate the trained CNN, the validation set was drawn independently from the total
set of simulated images, and the validation root mean squared error (RMSE) was calculated to
indicate the performance of the network over the whole dataset. Regression neural networks
minimize the distance between the ground truth point and the evaluated point in the parameter
space, so it is impossible to optimize the network on classification accuracy, but they have the
advantage of outputting quantitative estimates for the parameters, which is requisite for any
eventual optimization of morphology. Furthermore, we used a learning curve (Figure 4.5a) to
140
compare the training RMSE to the validation RMSE as a function of training data used. This shows
the estimation error was optimally minimized without reaching the region of overfitting, as
validation error nearly identically follows the sample error as it approaches the limit of the
approximation error. The total decrease in RMSE with iteration indicates that the adjusted network
parameters are increasing the accuracy of the predicted model parameters. The validation RMSE
leveling off but not increasing as the training RMSE decreases indicates that a reasonable learning
rate was chosen, and the network is well-trained.
47
Additional standard methods were used to
assess the quality of the network training. The quantile-quantile (Q-Q) plot shown in Figure 4.5b
illustrates a deviation between ground truth (imposed) number of dimensions and the predicted
number of dimensions at a high number of primary axes, indicating that the model slightly
overpredicts for 𝑑 = 1 and underpredicts for 𝑑 = 5,6 (i.e., the network loses its ability to
distinguish between images generated with increasingly large radial symmetry). This is a sensible
interpretation, as even human visual perception struggles to distinguish between shapes with high
degrees of 2D symmetry even without considering the additional pixelation and noise. This plot
also demonstrates that while the ground truth dimension is discrete, the network prediction is
continuous, as discretization would introduce a potential systematic bias. The RMSE plot against
the ground truth parameters in Figure 4.5c shows the model is extremely good for small p, where
the morphology changes the most with small changes in p, and large curvature p comprises most
of the error, indicating that as p increases, there is a small effect on the generated morphologies.
Detailed analyses of the chosen evaluation tools are discussed in the Methods section.

141

Figure 4.5. Diagnostic tools for assessing the quality of the neural network training. (a) Learning
curve plotting the root mean square error (RMSE) for individual nanocrystal images as a function
of the number of images on which the neural network was trained. (b) Q-Q plot for the number of
major axes (d). (c) RMSE-parameter plot for the curvature (p).

After optimizing the network type, number of layers, and network hyper-parameters via
the evaluation tools, the trained CNN was used to evaluate experimental TEM images, pre-
processed as described above. The CNN predicts a point in the parameter space comprised of the
three ground truth parameters (p, d, Lratio) that best corresponds to the morphology depicted in each
cropped image. Using these predicted parameters to ‘re-simulate’ the experimental morphologies
enabled a quantitative comparison to the ground truth via calculating the jaccard similarity
coefficient, which averaged around 0.8 out of 1.0 (see Methods).
41
Re-simulation qualitatively
illustrates that the predicted parameters are accurate renderings of the experimental nanocrystals
(Figure 4.6). While individual ground truth simulations may deviate from their experimental
counterparts, the network still performs well over the distribution of the entire testing set, as the
training and evaluation of a neural network is a statistical process.
142

Figure 4.6. Scatter plots of the total nanocrystal populations with shape group classifications
indicated by color for (a) the hot-injection synthesis and (b) the LARP synthesis of CsPbBr3.
Groups labels include re-simulation of the average predicted CNN parameters compared to the
experimental counterpart that generated the prediction and average size of each group. Average
calculated Jaccard coefficient was 0.8 across every group.

4.3.1.3. Unsupervised Clustering and Classification into Shape Groups
The ground truth predictions for each nanocrystal were concatenated to the 30 morphology
features that were extracted in the image processing. In addition to information about the
distribution of the nanocrystal parameters, morphology feature data was used to obtain spatial
information and pixel scaling data. This aids in removing incorrect segmentations (i.e., near the
scale bar) and leads to a more accurate statistical clustering of the nanocrystals into shape groups
based on both size and shape. Some features such as FilledImage were used for visualizations of
the output. Clustering was unsupervised to further remove human input in the pipeline, which was
achieved by calculating the silhouette value to automatically assess cluster quality of K number of
clusters within each cluster algorithm, and the cophenet correlation coefficient (ccc) to compare
alternative cluster solutions obtained using different algorithms. The latter assesses how well the
clusterings obtained by each algorithm represent the data in a dendogram tree, scoring each
between 0 and 1 (where 1 is the highest quality solution, see Methods).
48
This approach is simpler
143
than others commonly used in literature but equally effective, thus enhancing speed without losing
accuracy.
30,39

The unsupervised clustering method was optimized by testing three different clustering
methods (i.e., K-means, Gaussian mixture model (GMM), and hierarchal linkage tree clustering)
with three versions of the output dataset: (1) solely morphological features (2) solely the three
shape parameters outputted from the CNN, and (3) both (1) and (2). 14,775 different nanocrystals
detected from 50 experimental TEM images of CsPbBr3 nanocrystals, with 10 replicates of each
to help optimize and control covariance matrices, were used in the first optimization. The three
clustering methods were tested with three versions of the output dataset: (1) solely morphological
features, which consisted of four commonly used shape descriptors (i.e., nanocrystal area,
circularity, eccentricity, and aspect ratio) (2) solely the three shape parameters outputted from the
CNN, and (3) both (1) and (2) combined. As seen in Table 4.8, K-means clustering paired with
the dataset 3 resulted in the highest overall performance, with ccc = 0.9483. To ensure
reproducibility of the optimization results, the optimization was repeated with 6,427 nanocrystals
of CsPbBr3 from 31 TEM images of lower resolution. This time, six more feature predictors were
added to gain generalizability: ‘CircularityCalc’, ‘MajorAxisLength’, ‘MinorAxisLength’,
‘ConvexArea’, ‘Perimeter’, and ‘PerimeterOld’. Again, the K-means clustering with the combined
dataset proved to be of the highest quality of all nine combinations, and surprisingly performed
even better than the data set with higher quality images, as seen in Table 4.9. This is likely due to
the addition of more predictors in the data set, and thus was the chosen format of the input data
moving forward. Interestingly, the GMM, which is inherently unsupervised as groupings are based
off of probability and is therefore sometimes used in unsupervised pipelines, failed to converge
the majority of the time, especially for large datasets with high variability in nanocrystal
144
morphology.
16
In summary, K-means clustering outperformed the other two methods with every
dataset, and the dataset of combined morphology features and CNN output parameters
outperformed all other datasets across all clustering methods. Consequently, we used K-means
clustering with a dataset consisting of both the CNN parameters and common morphology
descriptors for the unsupervised clustering in the pipeline (all clustering methods took less than a
minute, so time was not factored into the decision).

Table 4.8. Clustering optimization.
K-means GMM Linkage Tree

Optimal K ccc Optimal K ccc Optimal K ccc
Features only
3 0.6049 3 0.6049 3 0.6049
CNN parameters only
7 0.8725 6 0.8516 4 0.8523
Features & CNN parameters
6 0.9483 5 0.9413 6 0.9094

Table 4.9. Poor resolution clustering optimization.

K-means GMM Linkage Tree

Optimal K ccc Optimal K ccc Optimal K ccc
Features only
2 0.6420 N/A N/A 2 0.5928
CNN parameters only
4 0.8941 5 0.5878 6 0.5424
Features & CNN parameters
4 1.000 N/A N/A 6 0.7174

In comparison to the optimal dataset, clustering results from the dataset of only the CNN
parameters consistently ranked a close second, whereas the morphology features dataset (i.e., only
physical descriptors) consistently performed the worst, by a significant margin (Table 4.8). A
repetition of the optimization with a dataset from lower quality images followed the same patterns
across all combinations, further corroborating this trend (Table 4.9). Adhering to the concept that
descriptor properties relevant to clustering will improve the predictive power of the model,
49
this
result demonstrates that the CNN parameters are significantly more effective in quantitatively
defining nanocrystal morphology than common features extracted from image processing. Thus,
they offer a low-dimensional (3D parameter space) and more effective continuous option for
145
describing morphology in experimental design space mapping, where discrete responses cause
exponential increases in model complexity. Furthermore, although the addition of the physical
features is important for drawing conclusions that are chemically meaningful, they only slightly
increase the ccc value (i.e., the quality of the clustering), suggesting that the CNN parameters are
essential for results that are acceptable to act as ground truth labels. As a result, the final step of
the full pipeline was to use this labeled data for automatic training of a classification algorithm,
creating a built-in practical use of the output data for researchers. We optimized the classification
model via Bayesian optimization, which indicated the implementation of a multiclass support
vector machine (SVM) model (see Methods).

4.3.2. Experimental Applications
4.3.2.1. Training Classification Algorithms
Classification models can be vital tools for elucidating high-level connections between
experimental parameters and product morphologies and morphology-dependent properties, but
their practicality directly correlates to their accuracy.
11,34
To illustrate the effectiveness of the
concepts used in the pipeline to create a ground truth for nanocrystal morphology, we investigated
different classification models trained using various labeled datasets. The first study utilized large
(163 TEM images, N = 42,650 nanocrystals) and small (52 TEM images, N = 13,115 nanocrystals)
datasets, which include the three CNN parameters, the unsupervised clustering results, and 10 of
the most common morphology features used in similar studies (see Methods). The low-
dimensional nature of the morphology feature descriptors and the unsupervised clustering results
aid in bridging the gap between the complex morphology information embodied via the high-
dimensional CNN parameters and their real-world implications about morphology. While the CNN
146
parameters offer a description of morphology that is highly accurate, such a bridge is essential to
connect the relationship between that accurate description and the experimental parameters that
produced it for further applications. Both datasets consisted of several binary and ternary
nanocrystal phases spanning a wide range of morphologies to ensure representation of complex
populations notably absent in the literature.
11,21,29,30,41
The performances of the trained models were
evaluated by computing the prediction accuracy of the test datasets, or the percentage of correctly
classified nanocrystals into shape groups (based on a 2D rendering), from an 80/20 train/test split
with 10-fold cross validation. The accuracy of the model trained using the large dataset was 99.5%,
misclassifying only 42 out of 8,530 nanocrystals from the test dataset (Figure 4.7a). Training a
similar model on the smaller dataset (~30% of the large dataset) did not significantly affect the
model accuracy (99.1%), with only 25 out of the 2,623 nanocrystals in the testing set being
misclassified (Figure 4.7b). This emphasizes that the pipeline is robust for studies where the
number of TEM images is more limited.

Figure 4.7. Confusion matrix of the predicted class vs. the true class from the classification model
trained on 10 feature variables and three CNN parameters for (a) the large data set, consisting of
163 TEM images (corresponding to full_Mdl, test accuracy = 0.9951) and (b) the small data set,
consisting of 52 TEM images (corresponding to full_Mdl2, test accuracy = 0.9907).
147

When the three CNN parameters were removed and the models were trained using only
clustering results predicted from the 10 morphology features as labels, model accuracy of the large
dataset was significantly lower at 74.7%, misclassifying 2,155 of the 8,530 nanocrystals in the
testing set (Figure 4.8a). Model accuracy of the smaller dataset was even worse, with a test
accuracy of 55.7%, misclassifying 1,162 of the 2,623 nanocrystals in the testing set (Figure 4.8c).
This demonstrates that the addition of the CNN parameters drastically increases both model
training efficiency and accuracy, further supporting their importance in accurate morphology
classification. To ensure that the superiority of the models trained using the CNN parameters was
not solely due to the addition of three additional variables, three of the 10 morphology features
were randomly removed from the large and small datasets, and the classification algorithm was re-
trained with 10 variables: seven morphology features and the three CNN parameters. Model
accuracy remained significantly higher than without the inclusion of the CNN parameters, with a
test accuracy of 99.2% for the larger dataset and 99.0% for the smaller dataset (Figure 4.9). To
further corroborate this result, the classification algorithm was retrained using the large dataset,
but this time only including the CNN parameters and four morphology-feature predictors (Figure
4.9b). The test accuracy remained exceptionally high at 98.5%, which was within a standard
deviation of the others. This illustrates that implementation of a simulated ground truth via deep
learning results in nearly perfect classification models for classifying nanocrystal morphology,
while simultaneously highlighting the ineffectiveness of a features-only approach.

148

Figure 4.8. Confusion matrix of the predicted class vs. the true class (a) from the classification
model ‘feat_Mdl’, test accuracy = 0.7474, trained on the large data set of 163 TEM images, using
only the 10 morphology features, (b) from the classification model corresponding to ‘feat_Mdl2’,
test accuracy = 0.7401, trained on the large data set of 163 TEM images, using only four of the 10
morphology features, and (c) from the classification model corresponding to ‘feat_mdl3’, test
accuracy = 0.5571, trained on the small data set of 52 TEM images, using only the 10 morphology
features.

Figure 4.9. Confusion matrix of the predicted class vs. the true class (a) from the classification
model corresponding to ‘part_Mdl’, test accuracy = 0.9918, trained on the large data set of 163
TEM images, using the original training variables but with three random feature variables
removed. (b) From the classification model corresponding to ‘part_Mdl2’, test accuracy = 0.9949,
trained on the large data set of 163 TEM images, using the original training variables but with six
random feature variables removed. (c) From the classification model corresponding to
‘part_Mdl3’, test accuracy = 0.9899, trained on the small data set of 52 TEM images, using the
original training variables but with three random feature variables removed.

149
4.3.2.2. Morphological Differentiation of CsPbBr3 Nanocrystals from Different Syntheses
With the optimal route for morphology classification determined, the full pipeline was
implemented to compare two different synthetic methods for the preparation of colloidal CsPbBr3
nanocrystals that are nominally reported to give cuboidal morphologies; that is, the hot-injection
and ligand assisted reprecipitation (LARP) methods.
50,51
While 0-D CsPbBr3 nanocrystals are
commonly described as cuboids (and 2D projections of the cuboids in TEM images appear as
squircles—quantitatively super-ellipses with p-norms roughly between 5 and 20), recent studies
on the superball form factor of cuboidal edge roundness suggest that a more precise quantification
is necessary for accurate differentiation between nanocrystal shape, as such differences may have
consequences in terms of nanocrystal optoelectronic properties and/or reactivity (e.g., ion
exchange).
52
Here, 7,082 distinct nanocrystals synthesized via the hot-injection method (without
experimental size selection) were detected from 20 inputted TEM micrographs and analyzed. As
determined by the unsupervised cluster evaluation algorithm, the nanocrystals were subsequently
classified into four shape groups: small cuboids (group 1), large irregular nanocrystals (group 2),
larger cuboids (group 3), and small platelets (group 4) (Figure 4.10). The general trends in feature
values for each group and their location within the 3D parameter space are illustrated in Figure
4.11. The average and standard deviation of the characteristics of each group were calculated
(Tables 4.10 and 4.11) and visualized via histograms of the observations for each predictor
variable (Figure 4.10d and 4.12). Each shape group in the histograms are plotted in a different
color, enabling the calculation of the normal distribution that corresponds to each group.
150

Figure 4.10. (a) Classification of the CsPbBr3 nanocrystals resulting from the hot-injection
synthesis differentiated by color and overlaid onto original TEM images. Nanocrystals connected
to edges are removed. (b) Overlaid shape groupings (left) compared to the original bright field
TEM image (right). (c) A subset of 210 nanocrystals from each of the four shape groups. (d) Three
exemplary histograms and corresponding normal distributions of feature descriptors (clusters are
color coded).

151

Figure 4.11. Unsupervised clustering of the observed CsPbBr3 nanocrystals made via hot-injection
synthesis. (a) Dendrogram of the nanocrystal similarities. (b) Parallel coordinate plots of each
observed nanocrystal with group assignment coded by color. (c) Pareto chart of the six principal
components. (d) Scatter plots of each observed nanocrystal with the group assignment coded by
color in the 3D parameter space of the top three principal components. ccc = 0.9432.

Table 4.10. Average values of predictors (defined in Methods) for the four shape groups of
CsPbBr3 nanocrystals synthesized via hot injection.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 61.6158 0.9195 0.5401 9.6716 7.9768 2567.8 180.4105 189.0457 1.2266 0.0018 20.0054 2.0216 1.7125
2 276.3667 0.6625 0.7254 24.1715 15.5214 12,604 456.9070 478.2140 1.6245 0.0013 21.7260 2.4402 1.7336
3 124.5744 0.8739 0.5211 13.9089 11.5403 5249.9 266.8571 279.4506 1.2109 0.0017 21.2399 2.3072 1.7061
4 73.0234 0.7277 0.7966 12.8134 7.3783 3222.4 220.2988 230.5707 1.7715 0.0014 21.3070 2.2738 1.7028

152
Table 4.11. Standard deviations of predictors (defined in Methods) for each shape group of
CsPbBr3 nanocrystals synthesized via hot injection.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 21.2854 0.0612 0.1394 1.8532 1.7153 876.48 35.5529 37.3873 0.1407 1.2158e-04 6.5089 0.8345 0.4259
2 109.2246 0.1158 0.1587 4.4907 3.8949 5136.9 100.5615 105.2326 0.4038 2.3008e-04 6.6315 0.8233 0.4286
3 30.4012 0.0665 0.1431 1.9445 1.4277 1294.5 34.8289 36.3075 0.1450 1.3210e-04 5.9851 0.8779 0.4489
4 32.9983 0.0939 0.0877 3.3494 1.7898 1441.8 53.7282 56.3086 0.3947 1.8658e-04 6.5748 0.9056 0.4303

Figure 4.12. Histograms and their corresponding normal distributions of the input variables used
in shape classification for the CsPbBr3 nanocrystals from the hot-injection synthesis, with shape
groups indicated by color.

Analysis of these output results along with the population sizes of each group (Figure 4.13)
indicated that ~70% of the ensemble consisted of small cuboids (group 1) with an average 2D area
of 62 nm
2
and a high circularity of 0.92 (i.e., more rounded edges), and larger cuboids (group 3)
with an average 2D area of 125 nm
2
and sharper corners (circularity = 0.87). About 20% of the
ensemble consisted of the more plate-like nanocrystals (group 4) with a significantly higher aspect
ratio of 1.8 (as compared to the average aspect ratio of the cuboids falling around 1.2). The
153
remaining 10% of the ensemble was comprised of much larger nanocrystals with irregular
morphologies (group 2). This suggests a relatively monodisperse population, which is further
supported by the fact that the two cuboidal groups (1 and 3) that describe most of the population
are rather monodisperse themselves. Calculation of the size distributions yielded 𝜎/𝑑
̅ = 20% and
14% for groups 1 and 3, respectively, compared to 𝜎/𝑑
̅ = 33% for the entire population (Table
4.16). Furthermore, using these data to train a classification model resulted in an accuracy of
99.6%, only misclassifying six nanocrystals (Figure 4.14), suggesting acceptable ground truth
simulations (validated in Figure 4.6).

Figure 4.13. Total classified CsPbBr3 nanocrystals from the hot-injection synthesis with shape
groups indicated by color, showing the relative make-up of each group.

154

Figure 4.14. Confusion matrix of the predicted class vs. the true class from the classification model
trained on 7,082 CsPbBr3 nanocrystals (5,666 training, 1,416 testing) from 20 TEM images of the
hot injection synthetic product. Test accuracy = 0.9958.

The 20 TEM images were comprised of two different synthetic batches of CsPbBr3
nanocrystals, identically prepared, and thus were analyzed separately to gain insight on synthetic
reproducibility of the hot-injection method. Both nanocrystal populations (N = 4,064 and 3,018)
were equally clustered into four shape groups with morphologies qualitatively like those described
above (see Methods). Statistical analysis of each shape group indicated comparable morphology
statistics to each other and to the total product (Tables 4.10-4.16). Furthermore, the average sizes
(defined as the average major axis length) of the total populations were also nearly equivalent
(batch 1 = 12.5 nm, batch 2 = 12.1 nm, and total = 12.4 nm). Both batches also followed the same
trends in terms of the relative population fractions of nanocrystals belonging to each group. Most
nanocrystals belonged to the monodisperse cuboids of groups 1 and 3 (although batch 2 had
slightly more of the sharper cornered cuboids), followed by the more oblong platelets of group 4.
Each batch had a relatively small population of the larger irregular shapes of group 2 (Figures
4.15 and 4.16). These data indicate good morphological reproducibility for the hot-injection
synthesis of CsPbBr3 nanocrystals and provide a proof of concept for the pipeline to quantify
synthetic reproducibility in general.
155

Table 4.12. Average values of predictors (defined in Methods) for the four shape groups of
CsPbBr3 nanocrystals synthesized in hot injection batch 1.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 76.4384 0.9180 0.5174 10.7171 8.9826 3183.9 202.0016 211.6519 1.2034 0.0018 20.7088 2.1056 1.7212
2 345.8179 0.7105 0.6892 26.0462 17.6106 15,209 494.4572 517.3618 1.5208 0.0014 20.5325 2.3378 1.7883
3 145.9646 0.8380 0.5790 15.6066 12.1545 6192.2 295.4319 309.2274 1.3045 0.0017 21.2775 2.4034 1.6666
4 64.5972 0.7372 0.7950 12.1335 6.9068 2836.3 206.8333 216.4916 1.7832 0.0015 21.7497 2.2004 1.6917

Table 4.13. Standard deviations of predictors (defined in Methods) for each shape group of
CsPbBr3 nanocrystals synthesized in hot injection batch 1.

Figure 4.15. Total classified CsPbBr3 nanocrystals from batch 1 of the hot-injection synthesis
showing the relative make-up of each group, with shape groups indicated by color along with 210
example nanocrystals from each shape group.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 23.1040 0.0558 0.1409 1.8493 1.7008 954.2286 35.5095 37.2488 0.1282 1.1083e-04 6.0045 0.8608 0.4522
2 105.6043 0.1084 0.1618 4.6618 3.3526 4677.8 83.7588 87.7768 0.3587 2.1535e-04 5.9047 0.8775 0.3957
3 33.0039 0.0794 0.1622 2.1935 1.7062 1393.6 35.5163 37.1098 0.2474 1.5778e-04 5.4323 0.8614 0.4342
4 26.9583 0.0989 0.0955 3.2644 1.5222 1177.9 49.2420 51.6850 0.4290 1.9638e-04 6.4965 0.8785 0.4187
156

Figure 4.16. Total classified CsPbBr3 nanocrystals from batch 2 of the hot-injection synthesis
showing the relative make-up of each group, with shape groups indicated by color along with 210
example nanocrystals from each shape group.

Table 4.14. Average values of the predictors (defined in Methods) for the four shape groups of
CsPbBr3 nanocrystals synthesized in hot injection batch 2.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 49.2334 0.9242 0.5653 8.7366 7.0488 2051.3 160.6821 168.3308 1.2537 0.0018 18.8453 1.9485 1.6471
2 254.8304 0.6393 0.7538 23.8432 14.7501 11,885 447.9757 468.9509 1.6886 0.0013 22.0026 2.3797 1.7248
3 107.7135 0.8907 0.4932 12.7367 10.8735 4529.2 245.5172 257.3102 1.1734 0.0018 20.7913 2.2481 1.7560
4 74.3754 0.7266 0.7748 12.5627 7.6750 3312.4 221.6624 231.8906 1.6785 0.0014 20.7958 2.2877 1.7315

Table 4.15. Standard deviations of the predictors (defined in Methods) for each shape group of
CsPbBr3 nanocrystals synthesized in hot injection batch 2.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 18.4319 0.0612 0.1361 1.7545 1.5644 758.9290 33.0486 34.8025 0.1515 1.2150e-04 6.7512 0.8509 0.4196
2 105.5354 0.1090 0.1413 4.3325 3.9752 5255.6 108.6159 113.6254 0.4062 2.1647e-04 6.6007 0.8556 0.4116
3 28.0500 0.0651 0.1242 1.7922 1.3808 1199.7 34.3184 35.7568 0.1002 1.2923e-04 6.3010 0.9019 0.4325
4 35.7661 0.0872 0.0878 3.2038 2.0406 1585.3 56.4780 59.0434 0.3424 1.7329e-04 6.6597 0.9103 0.4315
157

Table 4.16. Batch-to-batch morphological comparison of CsPbBr3 nanocrystals synthesized by
the hot injection method.
Batch 1 Batch 2 Total
Shape group 1 2 3 4 1 2 3 4 1 2 3 4
Average size
a
(nm) 10.7 26.1 15.6 12.1 8.7 23.8 12.7 12.6 9.7 24.2 13.9 12.8
Size distribution (𝜎/𝑑
̅ ) 0.17 0.18 0.14 0.27 0.21 0.18 0.14 0.25 0.20 0.19 0.14 0.27
Total average size (nm) 12.5±3.7 12.1 ±4.6 12.4 ± 4.1
Total size distribution (𝜎/𝑑
̅ ) 0.30 0.38 0.33
a
Average major axis length across all shape groups.

Similarly, 6,427 distinct nanocrystals were detected from 31 TEM micrographs depicting
populations that were synthesized by the LARP method, to compare the products to those produced
via hot injection. This dataset was clustered into six shape groups, whose general shapes are
illustrated in Figure 4.17c, although relative sizes shown between the groups (defined in Table
4.19) are not to scale. Shape groups 1-6 had average sizes of 8.9, 32.2, 15.4, 6.7, 13.1, and 21.9
nm, respectively. Similarly, each had varied size distributions (𝜎/𝑑
̅ ), or polydispersities, of 24%,
15%, 15%, 25%, 28% and 30%, respectively. More detailed characteristics of each group were
calculated, visualized, and analyzed in an identical manner to that of the hot-injection synthesis
(see Methods). Analysis of the results indicated that each group had approximately equal
representation in the ensemble (~20%), apart from the very large irregular quasi-cuboids of group
2 that made up < 5% of the ensemble (Figure 4.19). This, along with the wide range of average
sizes across the groups, suggests a more polydisperse product, which aligns with the polydispersity
calculated for the total dataset of 50% (Table 4.17). The LARP dataset was used to train a
classification algorithm, resulting in a test accuracy of 99.3% (Figure 4.20), which corroborated
the simulated ground truth for further applications (Figure 4.6).

158

Figure 4.17. (a) Classification of the CsPbBr3 nanocrystals resulting from the LARP synthesis
differentiated by color and overlaid onto original bright field TEM images. Nanocrystals connected
to edges have been removed. (b) Overlaid shape groupings (left) compared to the original TEM
image (right). (c) A subset of ≤ 210 nanocrystals from each of the six shape groups (d) Three
exemplary histograms and corresponding normal distributions of feature descriptors (clusters are
color coded).

159

Figure 4.18. Unsupervised clustering of the observed CsPbBr3 nanocrystals from the LARP
synthesis. (a) Dendrogram of the nanocrystal similarities. (b) Parallel coordinate plots of each
observed nanocrystal with group assignment coded by color. (c) Pareto chart of the six principal
components. (d) Scatter plots of each observed nanocrystal with the group assignment coded by
color in the 3D parameter space of the top three principal components. ccc = 0.4586.

160

Figure 4.19. Total CsPbBr3 nanocrystals classified from the LARP synthesis, showing the
relative percentages of nanocrystals in each group.

Figure 4.20. Confusion matrix of the predicted class vs. the true class from the classification model
trained on 6,427 CsPbBr3 nanocrystals from the 31 TEM images of the LARP synthesis (5,124
training, 1,285 testing). Test accuracy = 0.9930.

Comparing the two synthetic methods, LARP yielded significant CsPbBr3 morphological
differences from hot injection. The nanocrystals produced via hot injection were more well-
defined, exemplified both quantitatively by a higher classification accuracy and qualitatively by
161
the nanocrystal segmentations in Figures 4.10c and 4.17c. The average morphologies of each
group within the parameter space defined by the CNN parameters in Figure 4.6 show that the hot-
injection synthesis generally produced more uniform cube-like nanocrystals across the ensemble,
whereas the nanocrystal morphologies resulting from the LARP synthesis are more irregular and
varied. Additionally, the hot-injection synthesis gave averages sizes ranging from 9.7-24.2 nm
across the groups (Figure 4.6a), nearly half of the 6.7-32.2 nm size range seen for the LARP
synthesis (Figure 4.6b), suggesting greater monodispersity. This is confirmed by the nanocrystals
resulting from the hot-injection synthesis being classified into fewer shape groups (4 vs. 6), as
fewer well-defined normal distributions in the predictor variable histograms indicate more
morphologically similar populations (i.e., fewer statistically different morphologies) (Figures 4.12
and 4.21 and Tables 4.10, 4.11, 4.17, and 4.18). The previously mentioned polydispersities
calculated for the entire ensembles only strengthen this claim (33% for hot injection vs. 50% for
LARP). Qualitatively this makes sense, as the CsPbBr3 nanocrystals resulting from the hot-
injection syntheses generally self-assemble upon drop casting onto the TEM grid.
53
Furthermore,
a deeper analysis of the groups that characterize the upper bounds of the resultant morphologies
(i.e., the outlier groups – group 2 for both syntheses) revealed that group 2 from the LARP
synthesis had an average 2D area that was well over two standard deviations greater than that of
the hot-injection synthesis (573.7 nm
2
vs. 276.4 nm
2
, respectively), indicating a drastic difference
between the morphological parameter spaces of the two products (Tables 4.10, 4.11, 4.17, and
4.18). Although it is generally accepted in the literature that the hot-injection method offers a
superior morphological product due to more controlled instantaneous nucleation,
53
the pipeline
output provides the statistically significant data to support that assertion, and quantifies the
morphological differences. Using the CNN to define shape across an entire nanocrystal population
162
therefore helps establish more accurate quantifiable metrics for typically qualitative conclusions
about the general trends in nanocrystal syntheses.

Figure 4.21. Histograms and their corresponding normal distributions of the input variables used
in shape classification of the CsPbBr3 nanocrystals made via LARP, with shape groups indicated
by color.

Table 4.17. Average values of the predictors (defined in Methods) for the four shape groups of
CsPbBr3 nanocrystals synthesized via LARP.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 39.1757 0.8160 0.7703 8.9117 5.5804 1690.9 151.8459 158.8663 1.6203 0.0016 21.2623 2.1419 1.6692
2 573.7375 0.6519 0.6353 32.1486 23.8103 25,999 666.8564 698.1349 1.3787 0.0013 18.8130 2.3375 1.6989
3 145.9738 0.8263 0.5707 15.4068 12.3162 6251.6 295.6497 309.4471 1.2626 0.0016 20.0357 2.3325 1.7046
4 29.4968 0.9430 0.5549 6.6573 5.4216 1232.1 122.0920 127.7180 1.2362 0.0019 18.5172 1.9627 1.7497
5 62.0060 0.6202 0.8622 13.1355 6.4211 2938.8 218.6326 228.8509 2.1326 0.0012 19.8174 2.3673 1.6944
6 272.4243 0.7399 0.6162 21.9325 16.5869 12,021 429.7902 449.7485 1.3472 0.0015 19.4476 2.3061 1.7080

163
Table 4.18. Standard deviations of the predictors (defined in Methods) for each shape group of
CsPbBr3 nanocrystals synthesized via LARP.
Grp Area Circularity Eccentricity
Major
Axis
Length
Minor
Axis
Length
Convex
Area Perimeter
Perimeter
Old
Aspect
Ratio
Circularity
Calc CNN1 CNN2 CNN3
1 19.3420 0.0647 0.0713 2.0955 1.4432 834.11 37.4474 39.2414 0.2199 1.2845e-04 6.6875 0.8289 0.4155
2 184.4651 0.1161 0.1497 4.9543 4.4617 8450.5 121.5881 127.6297 0.2653 2.3064e-04 4.8796 0.7742 0.4011
3 41.9474 0.0790 0.1378 2.2833 2.0001 1743.9 42.4147 44.4252 0.1602 1.5696e-04 5.4171 0.8363 0.4190
4 15.4427 0.0676 0.1350 1.6993 1.4659 644.72 32.7729 34.3678 0.1246 1.3428e-04 6.4273 0.8134 0.4152
5 35.2468 0.0844 0.0677 3.5077 2.0823 1687.5 64.7533 67.8194 0.4730 1.6770e-04 6.2033 0.8437 0.4105
6 64.7995 0.1102 0.1511 2.7104 2.4885 2643.2 48.9139 51.1370 0.2430 2.1887e-04 5.3137 0.8157 0.3920

Notably, the histograms of all three CNN predictor variables for nanocrystals from both
synthetic methods consist of nearly perfectly overlapping normal distributions for each group,
whereas the feature descriptors show histograms with much more statistically different
distributions for each cluster (Figures 4.12 and 4.21). This is counterintuitive for clustering data,
as separated normal distributions for a specific feature generally indicate a unique cluster, but the
data from a single feature is just one dimension of a complex, high-dimensional definition of
morphology. If one were to combine the data from several feature histograms into a single
histogram, the separated distributions will start to fade if you consider, for example, that a
morphology cluster with a high value of circularity would have a low value of a feature like
eccentricity. After enough features are combined, the separation between clusters would not exist
at all on such a histogram plot. Such is the nature of the CNN parameters. The overlapping
distributions on a histogram support the concept of the CNN parameter combination as a high-
dimensional quantitative definition of shape for each nanocrystal, which encompasses morphology
information equivalent to an entire array of morphology features. While they are poor tools for
extracting specific characteristics, they hold a complete picture of high-dimensional data in a low-
dimensional form (much like what multidimensional scaling or principal component analysis do
164
for high-dimensional data visualization). This reaffirms their ability to be used as ground truth
labels in subsequent training.
The average nanocrystal sizes for both syntheses were also measured by hand by two
experienced researchers for comparison. Due to the average size and size distributions of each
nanocrystal ensemble being automatically output by the pipeline, thousands of nanocrystals are
accurately measured in just minutes. In contrast, the common practice of manually measuring the
major axis length of N ³ 300 nanocrystals per sample not only takes considerably longer but is
subject to inaccurate population sampling due to human bias and error.
12,13
For the hot-injection
method, the average nanocrystal size measurements from the researchers were 9.7 ± 2.1 nm with
a size distribution of 22% (N = 311) and 12.6 ± 4.4 nm with a size distribution of 35% (N = 310),
respectively. For the LARP method, the average size measurements were 10.6 ± 2.2 nm with a
size distribution of 21% (N = 300) and 11.9 ± 4.4 nm with a size distribution of 37% (N = 310).
The manually measured average sizes were comparable to those outputted by the pipeline, as all
were within a single standard deviation of each other (Table 4.19). However, the manual
measurements varied significantly when it came to the size distributions. This is due in part to the
inconsistency of manual measuring, which is exacerbated by the small population statistics (~300
each). This problem is eliminated when using the pipeline. Our pipeline completed both analyses
within 10 min, with analysis of the nanocrystals from the LARP method (N = 6,427 nanocrystals)
only taking about 3 min. The manual measurements took an average of 29 min to analyze the
images of the hot-injection products and 33 min to analyze the images of the LARP products. The
manual analyses yielded an average analysis throughput of ca. 10 nanocrystals/min, compared to
the pipeline throughput of 6,270 nanocrystals/min. This signifies over a 600-fold increase in
measurement efficiency using the pipeline.
165

Table 4.19. Efficiency comparison of analysis methods.
Automated Pipelines By-Hand
Ref.
30
This Work Researcher 1 Researcher 2
Synthesis method Hot
injection
LARP Hot
injection
LARP Hot
injection
LARP Hot
injection
LARP
Shape groups 2 2 4 6 — — — —
Average size
a
(nm) — — 12.4 12.5 12.6 11.9 9.7 10.6
Size distribution (𝜎/𝑑
̅)
— — 0.33 0.50 0.35 0.37 0.22 0.21
Nanocrystal count (N) 331 489 7,082 6,427 310 310 311 300
Run time
b
(min:sec) 16:38 7:37 8:39 5:05 35:14 42:18 23:15 24:39
a
Average major axis length across all shape groups.
b
All code is run on a 32 GB 6-core Intel® i7 @ 2.6 GHz. All code (including ref.
28
) was run in its native
OS environment.

The two datasets were then inputted into the previously published Autodetect-mNP
pipeline to directly compare the accuracy and sensitivity of the two pipelines.
30
In comparison to
the thousands of nanocrystals and several shape groups detected in our pipeline, only a few
hundred nanocrystals from each dataset made it through the selection process in the Autodetect-
mNP pipeline for analysis. The nanocrystals from both synthesis methods were also only grouped
into two shape groups: one group that resembled cuboids and one group that resembled platelets,
with nanocrystal edges that were rather jagged (Table 4.19 and Figures 4.22 and 4.23). In addition
to losing analytical details through an oversimplification of morphology, quantitative information
about each shape group was not accessible, nor realizable in 3D space. This demonstrates that not
only is our pipeline capable of morphological differentiation of the same CsPbBr3 nanocrystals
prepared using two different synthesis methods, but it is also more accurate, precise, and efficient
than current shape classification algorithms in the literature. While training a neural network is
computationally taxing, the evaluation of a trained neural network is efficient, so we expect this
pipeline to be faster than other methods.
166

Figure 4.22. Output of the Autodetect-mNP pipeline when analyzing CsPbBr3 nanocrystals from
the hot-injection synthesis. (a) Total nanocrystals detected (blue) vs. total nanocrystals that passed
the filtering technique used (white). (b) Total classified nanocrystals with shape group indicated
by color, showing the relative make-up of each group. (c) Histograms of the input variables for
shape classification with shape groups indicated by color.

Figure 4.23. Output of the Autodetect-mNP pipeline when analyzing CsPbBr3 nanocrystals from
the LARP synthesis. (a) Total nanocrystals detected (blue) vs. total nanocrystals that passed the
filtering technique used (white). (b) Total classified nanocrystals with shape group indicated by
color, showing the relative make-up of each group. (c) Histograms of the input variables for shape
classification with shape groups indicated by color.

4.3.2.3. Applicability to Morphologically Diverse Ensembles
Fifty-two TEM images of colloidal nickel sulfide nanocrystals encompassing the complex
phase space of that binary system (e.g., Ni3S4, NiS, Ni9S8, and Ni3S2) were simultaneously inputted
into the pipeline because of the morphological diversity amongst the various products (N = 13,132
nanocrystals). This allowed the pipeline to be tested across both phase and morphology for many
167
nanocrystals. The output yielded 10 statistically significant shape groups with ccc = 0.9369
(Figures 4.24-4.26). Notably, each distinct phase of nickel sulfide, as assessed by powder X-ray
diffraction, consisted of a unique combination of shape groups, illustrating the ability of the
pipeline to differentiate between crystal phases with distinct nanocrystal morphologies. Phase-pure
Ni3S2 nanocrystals were assigned to shape groups 7-10 with a primary contribution from groups 9
and 10 (Figure 4.24a), demonstrating accurate identification of irregular, concave shapes, which
as previously mentioned, has been an ongoing problem in shape classification in pipelines that can
handle nanocrystal overlap.
19,29,30,35,45
Ni3S4 nanocrystals were assigned to shape groups 3-8
(Figure 4.24b) and nicely display the classification of several shape groups in a single phase-pure
sample. Ensembles of colloidal Ni9S8 nanocrystals were assigned to shape groups 1-8 (Figure
4.24c), demonstrating an assessment of temporal reaction trajectories from quasi-spherical
nanocrystals at early reaction times, to elongated rod-like structures at longer reaction times. NiS
nanocrystals were assigned to shape groups 1-7 (Figure 4.24d), showing successful classification
of nanocrystals imaged at different magnifications. A classification algorithm trained with the data
had an accuracy of 99.5% (Figure 4.7). Additionally, the simultaneous classification of a large
number of shape groups has not yet been demonstrated in the literature, and is particularly more
difficult when done in an unsupervised nature due to the high level of precision in nanocrystal
detection and analysis that is necessary for unassisted classification.
11,16,30

168

Figure 4.24. Classification of a variety of nickel sulfide nanocrystals, illustrating the ability of the
pipeline to detect and quantify more morphologically diverse samples. Nanocrystals connected to
edges have been removed. (a) Irregular, concave morphologies of Ni3S2. (b) Morphological
diversity observed in phase pure Ni3S4. (c) Temporal reaction trajectory of morphology changes
in Ni9S8. (d) Magnification differences in TEM images of NiS. (1-10) Colored segregation of
≤ 210 representative nickel sulfide nanocrystals into the 10 distinct shape groups.

169

Figure 4.25. Unsupervised clustering of the nickel sulfide nanocrystals. (a) Dendrogram of the
nanocrystal similarities. (b) Parallel coordinate plots of each nickel sulfide nanocrystal with group
assignment coded by color. (c) Pareto chart of the six principal components. (d) Scatter plots of
each nickel sulfide nanocrystal with the group assignment coded by color in the 3D parameter
space of the top three principal components. ccc = 0.9369.

170
Figure 4.26. Histograms and their corresponding normal distributions of the input variables used
in shape classification for the nickel sulfide nanocrystals with shape groups indicated by color.

4.4. Methods
4.4.1. Nanocrystal Synthesis
4.4.1.1. Syntheses of CsPbBr3
50,51

Ligand assisted reprecipitation method. Cs2CO3 (99.9%), PbO (99.9%), and oleic acid (90%)
were purchased from Sigma-Aldrich. Tetraoctylammonium bromide (TOAB) was purchased from
Beantown Chemical. In a typical procedure, a 10 mM CsPb-precursor was prepared by dissolving
65.16 mg (0.2 mmol) of Cs2CO3 and 89.28 mg (0.4 mmol) PbO in 4 mL of oleic acid under vacuum
at 120 °C for 30 min. Upon cooling the mixture to room temperature, the solution was added to 40
mL of toluene. Separately, a 40 mM Br-precursor solution was prepared by dissolving 1.2 mmol
of TOAB in 2.4 mL of oleic acid and 30 mL of toluene by stirring at 1000 rpm at room temperature.
Upon complete dissolution of both precursor solutions, 5.7 mL of the CsPb-precursor solution was
added to a flask with 4.3 mL of the Br-precursor solution. The reaction was allowed to stir for 10 s
171
before being split evenly between two 50-mL centrifuge tubes (5 mL of product in each). The
product in each centrifuge tube was precipitated with 4 mL of isopropyl alcohol (IPA) and briefly
vortex mixed, followed by centrifugation (6,000 rpm, 5 min). The supernatant had a bright yellow
tint and was discarded, leaving the yellow/green solid to be redispersed in exactly 1 mL of hexanes.
Each tube was then briefly vortex mixed and bath sonicated before undergoing centrifugation
(6000 rpm, 5 min). The bright yellow/green supernatant was used for TEM characterization.
Hot-injection method. Cs2CO3 (99.9%), octadecene (90%), oleylamine (70%), and oleic acid
(90%) were purchased from Sigma-Aldrich. PbBr2 (99+%) was purchased from Alfa Aesar. A
Cs(oleate) precursor was prepared by drying Cs2CO3 (101.8 mg, 0.312 mmol), 0.5 mL oleic acid,
and 5 mL octadecene under vacuum at 120 ˚C for 2 h. Then, the Cs(oleate) solution was placed
under a nitrogen atmosphere and cooled to 100 ˚C for injection. PbBr2 (138 mg, 0.376 mmol), 1
mL oleic acid, 1 mL oleylamine, and 7.5 mL octadecene were dried at 120 °C for 2 h. The lead
solution was then ramped to 140 °C under a nitrogen atmosphere and allowed to equilibrate for 10
min. The solution of Cs(oleate) (0.8 mL) heated to 100 °C was quickly injected into the lead
solution; the reaction was allowed to stir for 10 s, and then the flask was immersed into an ice bath
to quench the reaction. The product was centrifuged at 6000 rpm for 5 min. The supernatant was
discarded, and the precipitate was redispersed in 4 mL hexanes and sonicated for 2 min. The
product was precipitated with 4 mL of IPA in a centrifuge tube and briefly vortex mixed, followed
by centrifugation (6000 rpm, 5 min). The supernatant had a slight green-yellow tint and was
discarded, leaving the yellow/green solid to be redispersed in exactly 2.5 mL of hexanes and briefly
vortexed mixed and bath sonicated before undergoing centrifugation (6000 rpm, 5 min). The bright
green supernatant was used for TEM characterization.

172
4.4.1.2. Synthesis of Nickel Sulfides
24

Nickel(II) iodide (NiI
2
, Alfa Aesar, 9 9.5%), N,N′-diphenyl thiourea (Alfa Aesar, 98%), N,N′-
dibutyl thiourea (Alfa Aesar, 98%), oleylamine (cis-9-octadecenylamine, Sigma Aldrich, 70%),
dodecylamine (1-aminododecane, Sigma Aldrich, 98%), dibenzylamine (Alfa Aesar, 98%) and 1-
dodecanethiol (Sigma Aldrich, 98%) were all purchased and used without further purification.
Nanocrystal syntheses were conducted under N
2 using Schleck techniques in the absence of water
and oxygen.
Ni3S2 nanocrystal synthesis. NiI
2 (0.19 mmol, 0.06 g) and degassed oleylamine (15.2 mmol, 5.0
mL) were added to a three-neck flask fitted with a reflux condenser and rubber septa. The solution
was heated to 120 °C and degassed for 30 min under vacuum. N,N′-Dibutyl thiourea (1.9 mmol,
0.36 g) was dissolved in dibenzylamine (15.6 mmol, 3.0 mL) and the solution was sparged by
bubbling nitrogen through it for 15 min. The solution of NiI
2 in oleylamine was heated to 180 °C
and then the solution of N,N′-dibutyl thiourea was quickly injected into the flask under flowing
N
2
. After 5 min, 1-dodecanethiol (12.5 mmol, 3.0 mL) was also injected into the reaction mixture
and the reaction was allowed to proceed for 1 h with stirring under flowing N
2
. The reaction was
quenched by placing it in a water bath and allowing it to cool to room temperature.
Ni9S8
nanocrystal synthesis. In a typical synthesis, NiI
2

(0.38 mmol, 0.12 g) and degassed
oleylamine (15.2 mmol, 5.0 mL) were added to a three-neck flask fitted with a reflux condenser
and rubber septa. The solution was heated to 120 °C and degassed for 30 min under vacuum. N,N′-
Diphenyl thiourea (1.14 mmol, 0.26 g) was dissolved in dibenzylamine (10.4 mmol, 2.0 mL) and
the solution was sparged by bubbling N
2

through it for 15 min. The solution of NiI
2 in oleylamine
was heated to 180 °C under flowing N
2 and then the N,N′-diphenyl thiourea solution in
dibenzylamine was quickly injected into the reaction flask. After 5 min, 1-dodecanethiol (12.5
173
mmol, 3.0 mL) was subsequently injected into the reaction mixture and then the reaction was
allowed to proceed at 180 °C for 4 h with stirring under flowing N
2
. After 4 h, the reaction was
quenched by placing it in a water bath and allowing it to cool to room temperature.
Ni3S4
nanocrystal synthesis. NiI
2

(0.38 mmol, 0.12 g) and degassed oleylamine (15.2 mmol, 5.0
mL) were added to a three-neck flask fitted with a reflux condenser and rubber septa. The solution
was heated to 120 °C and degassed for 30 min under vacuum. N,N′-Diphenyl thiourea (0.57 mmol,
0.13 g) was dissolved in dibenzylamine (7.8 mmol, 1.5 mL) and the solution was sparged by
bubbling N
2

through it for 15 min. The solution of NiI
2 in oleylamine was heated to 180 °C, and
then the N,N′-diphenyl thiourea solution in dibenzylamine was quickly injected into the reaction
flask and allowed to react for 4 h with stirring under flowing N
2
. The reaction was quenched by
placing it in a water bath and allowing it to cool to room temperature.
NiS nanocrystal synthesis. NiI
2 (0.38 mmol, 0.12 g) and degassed dodecylamine (21.7 mmol, 5.0
mL) were added to a three-neck flask fitted with a reflux condenser and rubber septa. The solution
was cycled between vacuum and N2 several times at room temperature. N,N′-Diphenyl thiourea
(1.9 mmol, 0.43 g) was dissolved in dibenzylamine (15.6 mmol, 3.0 mL) and the solution was
sparged by bubbling N2 through it for 15 min. The solution of NiI
2 in dodecylamine was heated to
180 °C, and then the N,N′-diphenyl thiourea solution was quickly injected into the reaction flask
and allowed to react for 5 min with stirring under flowing N
2
. The reaction was quenched by
placing it in a water bath and allowing it to cool to room temperature.
Nickel sulfide nanocrystal purification. Nanocrystals were purified by precipitation in 25 mL of
ethanol followed by centrifugation at 6000 rpm for 10 min. The supernatant was discarded. The
precipitated nanocrystals were redispersed in 20 mL of hexanes and centrifuged again at 6000 rpm
for 3 min, causing the larger particulates to settle. The precipitate was discarded, and the
174
nanocrystals suspended in hexanes were reprecipitated again by addition of 20 mL of ethanol. The
precipitated nanocrystals were finally dispersed in hexanes. All workup procedures were carried
out in air.

4.4.2. Characterization
Transmission electron microscopy (TEM). Transmission electron microscopy (TEM) was
performed at an operating voltage of 200 kV on a JEOL JEM-2100, equipped with a Gatan Orius
charge-coupled device camera. Samples for TEM were prepared by drop-casting colloidal
dispersions of the nanocrystals in hexanes onto carbon-coated copper grids (carbon type-B, 200
mesh, Ted Pella). For by-hand analysis, the TEM micrographs were processed in ImageJ pixel
counting software to analyze nanocrystal size and polydispersity statistics for comparison to the
machine learning pipeline. The average edge lengths of the CsPbBr3 cubic nanocrystals (size) were
derived by manually measuring the edge length of each nanocrystal, from which the standard
deviation (size distributions) was obtained. A minimum of N = 300 individual nanocrystals were
measured per sample and averaged over multiple images.
12

4.4.3. TEM Processing Pipeline
4.4.3.1. Image Processing
Unprocessed TEM images in .tiff format were loaded into the MATLAB workspace via
imread function. The images were grayscaled and subject to adaptive thresholding via im2gray
imadjust, and adapthisteq to normalize the contrast. The images were then subject to filtering via
the medfilt2 function to smooth noise while retaining edge sharpness. The imbinarize function then
assigned individual pixels to two groups: foreground and background. The binary images were
175
then subjected to a series of openings, closings, filling, and border clearing to fill the holes, sharpen
the edges, and remove misidentified foreground and truncated or incomplete nanocrystals
connected to the edges of the image, rendering the most accurate binary representation of the
nanocrystals (see Scheme 4.2 and Code). Post-processing of the individual nanocrystal
segmentations then removed all nanocrystals with 8-connected pixels above a threshold of 0.4
(overlapping nanocrystals). Next, the watershed function was used to separate overlapping
nanocrystals with a separation distance of 2 pixels, followed by the implementation of bwdist and
regionprops functions to calculate 30 property measurements for each nanocrystal.
The 30 morphology properties output from the image processing algorithm for each
indicted region (i.e., nanocrystal) are as follows: (1) ‘Area’, the total area, in nm
2
, of the region
encompassed by the edge of the detected nanocrystal; (2) ‘BoundingBox’, the position and size of
the smallest box containing the region; (3) ‘Centroid’, the center of mass of the region; (4)
‘Circularity’, the roundness of objects, computed as (4×‘Area’×p)/(‘Perimeter’
2
); (5)
‘ConvexHull’, the smallest convex polygon that can contain the region; (6) ‘ConvexImage’, the
image that specifies the convex hull, with all pixels within the hull filled in; (7) ‘ConvexArea’, the
number of pixels in ‘ConvexImage’; (8) ‘Eccentricity’, the eccentricity of the ellipse that has the
same second-moments as the region, calculated as the ratio of the distance between the foci of the
ellipse and its major axis length; (9) ‘EquivDiameter’ the diameter of a circle with the same area
as the region; (10) ‘EulerNumber’, the number of objects in the region minus the number of holes
in those objects; (11) ‘Extent’, the ratio of pixels in the region to pixels in the total bounding box;
(12) ‘Extrema’, the extrema points in the region; (13) ‘FilledImage’, the image the same size as
the bounding box of the region, returned as a binary (logical) array where the ‘on’ pixels
correspond to the nanocrystal region with any holes filled in; (14) ‘FilledArea’, the number of ‘on’
176
pixels in ‘FilledImage’; (15) ‘Image’, the image the same size as the bounding box of the region;
(16) ‘MaxFeretDiameter’, the maximum distance between any two boundary points on the
antipodal vertices of convex hull that enclose the object; (17) ‘MaxFeretAngle’, the angle of the
maximum Feret diameter with respect to horizontal axis of the image; (18) ‘MaxFeretCoordinates’,
the endpoint coordinates of the maximum Feret diameter; (19) ‘MinFeretDiameter’, the minimum
distance between any two boundary points on the antipodal vertices of convex hull that enclose
the object; (20) ‘MinFeretAngle’, the angle of the minimum Feret diameter with respect to
horizontal axis of the image; (21) ‘MinFeretCoordinates’, the endpoint coordinates of the
minimum Feret diameter; (22) ‘MajorAxisLength’, the length (in pixels) of the major axis of the
ellipse that has the same normalized second central moments as the region; (23)
‘MinorAxisLength’, the length (in pixels) of the minor axis of the ellipse that has the same
normalized second central moments as the region; (24) ‘Orientation’, the angle between the x-axis
and the major axis of the ellipse that has the same second-moments as the region; (25) ‘Perimeter’,
the distance around the boundary of the region; (26) ‘PixelIdxList’, the linear indices of the pixels
in the region; (27) ‘PerimeterOld’, the algorithm used to calculate perimeter in older versions of
MATLAB, using weights of 1 for horizontal or vertical steps, and weights of √2 for diagonal steps;
(28)‘PixelList’, the locations of pixels in the region returned as a p-by-Q matrix, where each row
of the matrix has the form [x y z ...] and specifies the coordinates of one pixel in the region;
(29) ‘Solidity’, the proportion of the pixels in the convex hull that are also in the region, computed
as ‘Area’/’ConvexArea’; and (30) ‘SubarrayIdx’, the elements of L inside the object bounding box,
returned as a cell array that contains indices such that L(idx{:}) extracts the elements. More
detailed definitions are available in the MATLAB documentation.
48
We then manually calculated
177
circularity as the variable ‘CircularityCalc’ by dividing the ‘Area” by the ‘Perimeter’ squared, as
well as ‘AspectRatio’ by dividing the ‘MajorAxisLength’ by the ‘MinorAxisLength’ (see Code).

Scheme 4.2. Step-by-step workflow of the image processing.

4.4.3.2. Ground Truth Creation
The generation of simulated TEM images from their ground-truth parameters was done in
MATLAB using code built on the patch and fplot functions for the function generation, imresize
for pixelation, and poissrnd for noise. These images were then randomly partitioned into training,
validation, and testing sets as 4D-arrays before using MATLAB’s Deep Learning Toolbox with a
custom network design, hyper-parameters, and learning rate to train and evaluate the network.
These training-design properties were iteratively adjusted considering the training curve, the Q-Q
plots, and the RMSE plots (Figure 4.5).
178
Parameter Distributions. The chosen parameter distributions are illustrated in Table 4.7. The
distribution for curvature is chosen such that the spacing from the origin to the corners is a linear
spacing, which ensures that the distribution represents the nonlinear density of how quickly the
features change with small changes in p. Similarly, Lratio does not map linearly onto physical space,
but instead represents inverse length. Choosing these distributions guarantees a sample set that is
the most representative of the physical features of the generated morphologies. Each point in this
parameter space maps onto a closed curve, but it is not a 1:1 mapping (e.g., 𝑝 = 2 and various
positive integers, d). The random rotations and color values are used to help ensure the network is
applicable to the largest class of physically plausible nanocrystal TEM images. Since the training,
validation, and ground-truth test sets are all generated with this random phase factor that is not
being retrieved by the neural network, it is not necessary to account for different possible rotational
orientations of the nanoparticle image before training or evaluating the neural network.
Neural Network Design. The use of a neural network was chosen over analytical regression
because of the pixelated nature of the experimental images, as well as the complex and stochastic
relationship that exists between the mathematical model and the generated and experimental
images. A regression neural network is used over a classification neural network due to the
predicted parameters being able to be combined with non-neural network analysis methods. Here
we consider three types of neural networks, and it is important to pick an appropriate one for the
fitting task to include enough complexity to learn the patterns without having so many parameters
that every nuance in the training data is learned, to the detriment of generalizing to the validation,
testing, and experimental images: overfitting. A Fully Connected (FC) network consists only of
weighted bypass connections and most closely resembles standard regression techniques and
perceptron methods. A Convolutional Neural Network (CNN) adds additional convolution layers
179
that ‘blur’ local pixels, which is particularly effective for identifying features in images. A
Residual Neural Network (RNN) combines the ability of a CNN to identify large features with a
FC network’s direct pixel values by introducing bypass layers. Figure 4.27 shows some of the
diagnostic tools when training with (a) a FC network and (b) an RNN. The CNN showed superior
performance (diagnostic tools shown in Figure 4.5) over the other two types of neural networks
considered for this regression task (fully connected or residual) since it decreases the training and
validation RMSE as it learns and avoids overfitting. The optimal network being a CNN is standard
for this type of pixelated retrieval task, as they are commonly used for extracting meaningful
information from images, with convolution filters that enable broad features to be identified rather
than emphasizing the contribution of individual pixels like fully connected neural networks.
42
The
RNN can provide a better retrieval of the parameters due to the additional pass information
available (e.g., both the output of the convolution layers and the specific pixel values from the
bypass layers), but it is not used here since it overfits to the training data, even at low learning
rates. The results we present in Figure 4.27 are representative cases of each network type; for
specific details regarding the network architectures (number/size of layers and ordering of
normalization/ pooling/ linear response/bypass layers), learning rates, and hyperparameters, please
refer to the Github link for the full implementation of this pipeline.

180

Figure 4.27. Diagnostic tools for alternate neural network constructions. (a) Fully Connected (FC)
network. Left: training curve showing underfitting (insignificant learning), even at learning rates
increased by multiple orders of magnitude. Right: Q-Q plot for d, showing significant bias in the
prediction and an inability to distinguish between images of all number of major axes. (b) Residual
Neural Network (RNN). Left: training curve showing significant overfitting as the validation
RMSE no longer improves past half the training data, while the training RMSE continues to
decrease. Right: Q-Q plot for d, showing the adverse effects of overfitting when evaluating the test
set.

Neural Network Evaluation. It is standard practice in neural network evaluation and
hyperparameter optimization to use the approximation and estimation errors as a proxy for using
the learning curve; that is, a plot of the sample and validation errors as a function of the size of the
training set.
47
When evaluating a neural network structure and iterating the hyperparameters of
number of layers and learning rate, validation root mean squared error (RMSE) is an indication of
how well the network performs, but it is essential to contextualize this with a comparison to the
training RMSE as a function of training time and a decomposition of the network error over the
parameter space. Another tool for diagnosing neural network performance for regression is the
181
quantile-quantile (Q-Q) plot, which plots the network evaluation (i.e., the predicted parameters) of
the allocated test dataset randomly partitioned from the simulated images prior to training against
each of the ground truth parameters used to simulate each image. Patterns in the Q-Q plot,
particularly systemic deviations from the lines 𝛽
i,pred
= 𝛽
i,ground
, can be used to assess the
performance and predictive capabilities of the neural network as large-scale patterns between the
ground truth (imposed) number of dimensions and the predicted number of dimensions become
visible (particularly when comparing against the true-fit line). RMSE plots against each of the
ground truth parameters show where the training error is particularly high for each of the parameter
dimensions, or where the parameter-space distance between the ground truth point and the
predicted point is the greatest. This plot also determines the upper bounds of parameters such that
the network training is not entirely dominated by the error minimization of the fringes of the
parameter space. The type of network, number of layers, and network hyper-parameters were all
adjusted using these tools.
The FC network is unable to significantly reduce the RMSE over many iterations, even
with high learning rates. This makes sense since the distinction between the parameters does not
depend on the absolute pixel intensities but rather on local patterns in the pixel intensities. This
holds true even upon adding as many as 30 FC layers and going to learning rates as high as 0.01
(Figure 4.27a). In sharp contrast to that, the RNN has too many parameters such that it very
quickly achieves a point where additional training does not improve the RMSE, where overfitting
begins. While the overall trend in d is correctly predicted since each distribution at different ground
truth d roughly follow the line of slope 1 in the Q-Q plot, the variance of each distribution is
particularly high compared to the CNN trained network (Figure 4.27b). Of note is the negative
predicted values for d = 1. This, combined with the fact that the validation RMSE for the CNN
182
and the RNN do not significantly differ (3.6243 for the CNN, 3.8736 for the RNN), indicates that
there is no benefit to using the more complex network. It is good practice to use the simplest model
that fully minimizes the approximation error, and it is advantageous to use the entirety of the
training data, so the use of the CNN is most appropriate. The final CNN had the number of layers,
learning rate, and drop rate adjusted considering the diagnostic tools shown in Figure 4.5, resulting
in a network with four blocks of convolutional layers.
Re-simulating images from the predicted neural network parameters for the experimental
images was used to assess the accuracy of the ground truth. The binary mask of the re-simulated
nanocrystal image was compared to the binary mask of the experimental nanocrystal image using
the jaccard function, which computes the intersection of binary images BW1 and BW2 divided by
the union of BW1 and BW2, also known as the Jaccard index.
48
Results of the Jaccard similarity
coefficient calculations are visualized in Figure 4.6.
There are certain methods that can be used to efficiently optimize network
hyperparameters, but these methods are often themselves time-consuming and computationally
intensive.
54
As such, for this training set the hyperparameters were adjusted independently with
linear spacing until an optimum for each hyperparameter was achieved. It is certainly possible that
nonlinear optimization methods would yield a better-trained network, but the practical
performance gain for the final classification accuracy is expected to be negligible.

4.4.3.3. Unsupervised Clustering Optimization
Three types of clustering methods were investigated for the unsupervised clustering
optimization: K-means, Gaussian mixture model (GMM), and hierarchical linkage tree clustering.
K-means clustering divides observations into K groups by determining locations of K-group
183
centers and assigning each observation to the nearest group center. Centers are determined
iteratively starting with random centers and reallocating as each observation is assigned until the
centers converge to a fixed center. Centers occasionally converge to suboptimal locations, so 10
random starting positions were assessed by indicating 10 replicates in the hyperparameters, and
the solution with the minimum total distance from observation to center was chosen. GMM fits K
n-dimensional normal distributions to the data and uses those distributions to assign each
observation to a cluster. Initial estimates of distance parameters are iteratively returned, and
observations are then grouped based on which of the K Gaussians they are most likely to come
from. The probabilities used to determine the clusters were set as an output variable (p) and can
provide insight into the quality of the clustering, as well clustered data will clearly belong to just
one of the K Gaussians. Again, 10 replicates were indicated to help optimize and control variance.
Hierarchical clustering is inherently unsupervised by building a map of the full structure of the
data by recursively linking pairs of groups into larger groups in order of the distance between them,
called a dendrogram tree. Here we use single linkage, which is the default linkage. Initially, each
observation is its own group, and each linkage combines observations into larger groups until all
the data has been linked into a single group (Figure 4.28 and Figure 4.32). Potential groupings
can be determined via large vertical distance between groups on the dendrogram tree. These
groupings are evaluated and optimized via calculating the cophenet correlation coefficient (ccc)
via eq. 4.4,
𝑐𝑐𝑐 = 𝑐𝑜𝑝ℎ𝑒𝑛𝑒𝑡(𝑍,𝑌) (4.4)
which computes the ccc for the hierarchical cluster tree represented by Z. Z is the output of the
linkage function. Y contains the distances or dissimilarities used to construct Z, as output by the
pdist function. Z is a matrix of size (m–1)-by-3, with distance information in the third column. Y
184
is a vector of size m*(m–1)/2. The cophenetic correlation for a cluster tree is defined as the linear
correlation coefficient between the cophenetic distances obtained from the tree, and the original
distances (or dissimilarities) used to construct the tree. Thus, it is a measure of how faithfully the
tree represents the dissimilarities among observations. The cophenetic distance between two
observations is represented in a dendrogram by the height of the link at which those two
observations are first joined. That height is the distance between the two subclusters that are
merged by that link. The output value is the ccc. The magnitude of this value should be very close
to 1 for a high-quality solution. The cophenetic correlation between Z(:,3) and Y is defined as
𝑐 =
∑ (K
!'
#L)(N
!'
#O)
!('
P∑ (K
!'
#L)
$
∑ (Q
!'
#O)
$
!(' !('
(4.5)
where Yij is the distance between objects i and j in Y. Zij is the cophenetic distance between objects
i and j, from Z(:,3), and y and z are the average of Y and Z(:,3), respectively. This quantifies how
accurately the tree linkages represents the distance between observations. The number of clusters,
K, are therefore determined by a horizontal line at some value Y that cross K vertical linkages of
the dendrogram tree. Eq. 4.5 can then also be used to compare alternative cluster solutions obtained
using different algorithms, seen later.
All data were normalized using the normalize function. All clusterings were made
unsupervised via automated cluster evaluation (evalclusters function) based on the silhouette plots
of the groupings as the evaluation criteria, which judges the quality of the predicted number of
clusters by calculating a silhouette criterion value for each point. The silhouette value Si for the i
th
point is defined as
185
𝑆𝑖 =
(R
!
#F
!
)
SFT(F
!
,R
!
)
(4.6)
where ai is the average Euclidean distance from the i
th
point to the other points in the same cluster
as i, and bi is the minimum average distance from the i
th
point to points in a different cluster,
minimized over clusters. The silhouette value is a measure of how similar one point is to points in
its own cluster, when compared to points in other clusters. The silhouette value ranges from -1 to
1. A high silhouette value indicates that an observation (i) is well matched to its own cluster, and
poorly matched to other clusters. If most points have a high silhouette value, then the clustering
solution is appropriate. If many points have a low or negative silhouette value, then the clustering
solution might have too many or too few clusters. The number of clusters that maximized the
collective silhouette value was chosen as the optimal number, with an upper limit of 12 clusters to
avoid overfitting. This is a much simpler but equally effective approach than others seen in
literature, enhancing speed without losing accuracy.
30,39
This number was evaluated by calculating
the ccc, discussed previously.
48
Additionally, the results of each clustering were visualized using
the parallelcoords plotting function, which plots each individual observation, or nanocrystal, as a
horizontal line colored by their assigned group, where the x-axis consists of each grouping variable
and the y-axis represents the value of each observation at each specific variable of x (Figure 4.29
and Figure 4.33). Each dataset parameter space was visualized using a 3D scatter plot, where the
axes represent the grouping variables and each point represents a single observation, color coded
by its assigned group (Figure 4.30 and Figure 4.34). For the two datasets that had more than three
grouping variables, classical multidimensional scaling (CMD) was used to visualize datasets in 3D
space. CMD transforms an n-dimensional space into the smallest space necessary to preserve
pairwise distances of any distance metric. If Euclidean distances are used, the result is the same as
that of principal component analysis (PCA). Herein, Euclidean distances were used, so the results
186
of the dimension reduction represented the principal components in order of how much variance
in the data they explained, demonstrated via a Pareto chart (Figure 4.31 and Figure 4.35).

Figure 4.28. Dendrograms of (a) the features only data set, (b) the CNN parameters only data set,
and (c) both data sets combined.

Figure 4.29. Parallel coordinate plots of each observed nanocrystal with group assignment coded
by color for (a) K-means, (b) GMM, and (c) Linkage Tree clustering for the features only data set;
(d) K-means, (e) GMM, and (f) Linkage Tree clustering for the CNN parameters only data set; and
(g) K-means, (h) GMM, and (i) Linkage Tree clustering for the combined data set.
187

Figure 4.30. Scatter plots of each observed nanocrystal with the group assignment coded by color
in the 3D parameter space of (a) the features only data set, (b) the CNN parameters only data set,
and (c) the combined data set.

Figure 4.31. Pareto charts for (a) the features only data set, (b) the CNN parameters only data set,
and (c) the combined data set.
188
Figure 4.32. Dendrograms of (a) the features only data set, (b) the CNN parameters only data set,
and (c) both data sets combined.

Figure 4.33. Parallel coordinate plots of each observed nanocrystal with group assignment coded
by color for (a) K-means, (b) GMM, and (c) Linkage Tree clustering for the features only data set;
(d) K-means, (e) GMM, and (f) Linkage Tree clustering for the CNN parameters only data set; and
(g) K-means, (h) GMM, and (i) Linkage Tree clustering for the combined data set.

189
Figure 4.34. Scatter plots of each observed nanocrystal, with the group assignment coded by color
in the 3D parameter space of (a) the features only data set, (b) the CNN parameters only data set,
and (c) the combined data set.

Figure 4.35. Pareto charts for (a) the features only data set, (b) the CNN parameters only data set,
and (c) the combined data set.

190
4.4.3.4. Optimization of a Classification Algorithm
To optimize the classification model, the fitcauto function was used on the optimal data set
indicated by the unsupervised clustering with three CNN parameters and 10 morphology features:
‘Area’, ‘Circularity’, ‘Eccentricity’, “AspectRatio’, ‘CircularityCalc’, ‘MajorAxisLength’,
‘MinorAxisLength’, ‘ConvexArea’, ‘Perimeter’, and ‘PerimeterOld’. Given predictor and
response data, fitcauto automatically tries a selection of classification model types with different
learner algorithms, and hyperparameter values. The function uses Bayesian optimization to select
the best learner and hyperparameter values for the model by computing the cross-validation
classification error for each model variation. The data was partitioned into training and testing sets
using the cvpartition function with an 80/20 train/test split. By default, 150 iterations are trained
and analyzed for the hyperparameters of five different learner types: ensemble, k–nearest
neighbors (KNN), naïve Bayes (NB), support vector machine (SVM) and decision tree. Validation
loss is computed for the learner and hyperparameter values at each iteration. Fitcauto computes
the cross-validation classification error by default. Observed minimum validation loss and
estimated minimum validation loss are also computed at each iteration. The observed value
corresponds to the smallest validation loss value computed so far in the optimization process. For
the estimated value, fitcauto updates an objective function model maintained by the Bayesian
optimization process and uses this model to estimate the minimum validation loss (Figure 4.36).
One of the following evaluation results is output for each iteration: (1) Best — The learner and
hyperparameter values at this iteration give the minimum observed validation loss computed so
far. That is, the validation loss value is the smallest computed so far. (2) Accept — The learner
and hyperparameter values at this iteration give meaningful (for example, non-NaN) observed and
estimated validation loss values. (3) Error — The learner and hyperparameter values at this
191
iteration result in an error (for example, a validation loss value of NaN). The final display also
includes a description of two models: (1) Best observed learner with the listed learner type and
hyperparameter values – this model yields the final observed minimum validation loss. (2) Best
estimated learner with the listed learner type and hyperparameter values – this model yields the
final estimated minimum validation loss. Once the optimal model and hyperparameters expected
to best classify new data are indicated, fitcauto retrains the model on the entire training dataset and
returns it as the Mdl output. The performance of the model is then evaluated using the test dataset,
with test accuracy defined as 1 minus the loss of the model for the test set, or the percentage of
correctly classified observations. This is visualized using a confusion matrix, exemplified in
Figures 4.7-4.9.

Figure 4.36. Bayesian optimization of classification algorithm and hyperparameters.
Learner types to explore: ensemble, knn, nb, svm, tree Total iterations
(MaxObjectiveEvaluations): 150
Total time (MaxTime): Inf
192
________________________________________________________
Optimization completed.
Total iterations: 150
Total elapsed time: 75720.4142 seconds
Total time for training and validation: 75050.5914 seconds
Best observed learner is a multiclass svm model with:
Coding (ECOC): onevsone
BoxConstraint: 579.18
KernelScale: 2.0812
Observed validation loss: 0.025162
Time for training and validation: 442.6376 seconds
Best estimated learner (returned model) is a multiclass svm model with:
Coding (ECOC): onevsone
BoxConstraint: 877.19
KernelScale: 1.9516
Estimated validation loss: 0.03133
Estimated time for training and validation: 394.9173 seconds
Observed validation loss (or training loss) of 0.025162 and estimated validation loss of
0.03133 signify a good result considering an indication of a good model is a training loss that is
slightly less, but similar to the validation loss.
55

4.4.3.5. Classification Algorithms for Experimental Applications
Materials in the model datasets include Ni3S4, 𝑎-NiS, 𝑏-NiS, Ni9S8, Ni3S2, CoNi2S4, and CsPbBr3.
Full Training Data.
i. Large Data Set
full_Mdl = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'MinorAxisLength'
'ConvexArea' 'Perimeter' 'PerimeterOld' 'AspectRatio' 'CircularityCalc' 'cnnPredmat1'
'cnnPredmat2' 'cnnPredmat3’}
ResponseName: 'Grp’
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9]
ScoreTransform: 'none’
BinaryLearners: {36×1 cell}
CodingName: 'onevsone'
193

testAccuracy = single 0.9951
Total: 42,650 nanocrystals
Training: 34,120 nanocrystals
Testing: 8,530 nanocrystals
Misclassified: 42 nanocrystals

ii. Small Data Set – Full Data

full_Mdl2 = CompactClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'MinorAxisLength'
'ConvexArea' 'Perimeter' 'PerimeterOld' 'AspectRatio' 'CircularityCalc' 'cnnPredmat1'
'cnnPredmat2' 'cnnPredmat3'}
ResponseName: 'Grp'
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9 10]
ScoreTransform: 'none'
BinaryLearners: {45×1 cell}
CodingMatrix: [10×45 double]

testAccuracy = 0.9907
Total: 13,115 nanocrystals
Training: 10,492 nanocrystals
Testing: 2,623 nanocrystals
Misclassified: 25 nanocrystals

Feature-Predictors Only.

i. Large Data Set

feat_Mdl = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'MinorAxisLength'
'ConvexArea' 'Perimeter' 'PerimeterOld' 'AspectRatio' 'CircularityCalc’}
ResponseName: 'Grp’
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9]
ScoreTransform: 'none’
BinaryLearners: {36×1 cell}
CodingName: 'onevsone'

testAccuracy = 0.7474
Total: 42,650 nanocrystals
Training: 34,120 nanocrystals
Testing: 8,530 nanocrystals
Misclassified: 2,155 nanocrystals

ii. Large Data Set – 4 Morphology-Feature Predictors

feat_Mdl2 = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength'}
ResponseName: 'Grp'
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9]
ScoreTransform: 'none’
BinaryLearners: {36×1 cell}
CodingName: 'onevsone'
194

testAccuracy = 0.7401
Total: 42,650 nanocrystals
Training: 34,120 nanocrystals
Testing: 8,530 nanocrystals
Misclassified: 2,217 nanocrystals

iii. Small Data Set

feat_Mdl3 = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'MinorAxisLength'
'ConvexArea' 'Perimeter' 'PerimeterOld' 'AspectRatio' 'CircularityCalc’}
ResponseName: 'Grp'
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9 10]
ScoreTransform: 'none'
BinaryLearners: {45×1 cell}
CodingName: 'onevsone'

testAccuracy = 0.5571
Total: 13,115 nanocrystals
Training: 10,492 nanocrystals
Testing: 2,623 nanocrystals
Misclassified: 1,162 nanocrystals

Random Feature-Predictor Removal.

i. Large Data Set – 7 Morphology-Feature Predictors

Part_Mdl = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'MinorAxisLength'
'ConvexArea' 'Perimeter' 'cnnPredmat1' 'cnnPredmat2' 'cnnPredmat3'}
ResponseName: 'Grp'
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9]
ScoreTransform: 'none'
BinaryLearners: {36×1 cell}
CodingName: 'onevsone'

testAccuracy = 0.9918
Total: 42,650 nanocrystals
Training: 34,120 nanocrystals
Testing: 8,530 nanocrystals
Misclassified: 67 nanocrystals

ii. Large Data Set – 4 Morphology-Feature Predictors

Part_Mdl2 = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'cnnPredmat1'
'cnnPredmat2' 'cnnPredmat3’}
ResponseName: 'Grp’
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9]
ScoreTransform: 'none’
BinaryLearners: {36×1 cell}
CodingName: 'onevsone'
195

testAccuracy = 0.9849
Total: 42,650 nanocrystals
Training: 34,120 nanocrystals
Testing: 8,530 nanocrystals
Misclassified: 129 nanocrystals

iii. Small Data Set – 7 Morphology-Feature Predictors

part_Mdl3 = ClassificationECOC

PredictorNames: {'Area' 'Circularity' 'Eccentricity' 'MajorAxisLength' 'MinorAxisLength'
'ConvexArea' 'Perimeter' 'cnnPredmat1' 'cnnPredmat2' 'cnnPredmat3'}
ResponseName: 'Grp'
CategoricalPredictors: []
ClassNames: [1 2 3 4 5 6 7 8 9 10]
ScoreTransform: 'none'
BinaryLearners: {45×1 cell}
CodingName: 'onevsone'

testAccuracy = 0.9899
Total: 13,115 nanocrystals
Training: 10,492 nanocrystals
Testing: 2,623 nanocrystals
Misclassified: 27 nanocrystals

4.5. Conclusions
We successfully created an unbiased ground truth for nanocrystal morphology analysis by
stochastically simulating individual 2D nanocrystal TEM images from a 3D parameter space to
train a convolutional neural network. We then used improved computer vision techniques and the
trained network to predict the morphological parameters for each experimental image. This labeled
data offers a significant improvement in accuracy for both unsupervised clustering and the training
of a morphology classification algorithm. We demonstrated this by coding a fully automated
pipeline to identify, label, and classify nanocrystals from experimental TEM images. The neural
network portion of the pipeline outputs each nanocrystal image labeled with a quantitative
morphology definition comprised of three parameters from the CNN and the values of 30
morphology features. Once shape groups have been assigned via unsupervised clustering and
appended to the dataset, the pipeline outputs the average value and standard deviation of all 30
196
features for each distinct group, histograms and their corresponding normal distributions for each
predictor variable across each group, a linkage tree dendrogram of the population, a parallel
coordinate plot, a Pareto chart of the principal components, a 3D scatter plot of the parameter space
with groups indicated by color, visualizations of the groupings overlaid onto the original TEM
images, exemplary nanocrystal morphologies of each group in a montage, a montage of the total
classified nanocrystals to show relative population sizes of each group, the average size and
standard deviation of the input nanocrystal ensemble as a whole, and the confusion matrix and
classification accuracy from the classification algorithm training.
This simple, time-efficient pipeline will facilitate more accurate mapping between
nanocrystal morphology and applications in synthetic optimizations, mechanistic studies involving
size and shape, and morphology-function relationships. We demonstrate this by successfully
quantifying morphological reproducibility, differentiating morphological characteristics between
CsPbBr3 nanocrystals prepared using two different synthetic methods, and quantifying several
different morphologically diverse crystal phases of binary nickel sulfide nanocrystals. Not only is
the pipeline sensitive enough to identify nuances in nanocrystal populations that are limited when
using manual assessments, but these studies explicitly demonstrate improved generalizability over
current morphology classification algorithms found in the literature. This offers a route to
accurately automate and normalize TEM analysis for a wide range of nanocrystal sizes and shapes
across images of differing quality, contrast, magnification, and resolution, while concurrently
demonstrating more than a 600-fold improvement in efficiency over the currently accepted by-
hand measurement.
Looking forward, there is room for improvement via 3D reconstruction using tilt-series of
TEM images to validate the method. However, the generalizability and high-throughput nature of
197
the pipeline could enable widespread implementation of the output shape parameters as
quantitative definitions of individual 2D morphologies, affording standardization of nanocrystal
morphology reporting across the literature. This would facilitate cohesion of experimental findings
and consequentially aid in the implementation of data-driven learning (via the compilation of large
datasets). Additionally, the continuous, yet low dimensional nature of nanocrystal morphologies
described as CNN triplets creates a theoretically infinite, yet mappable morphological parameter
space in 3D (i.e., visualizable), as compared to the high dimensionality needed to utilize
continuous physical features. Targeted points can therefore be readily determined and utilized for
optimizing syntheses based on morphology using simple tools like regression, which is something
that the discrete nature of qualitative labels make extremely complex and impractical. Such
applications pave a pathway towards further investigations of morphology-dependent process-
structure-property relationships using this pipeline, which we are currently working on with
different materials.
4.6. Code
The code used in the pipeline as well as a how-to guide for implementation is available at:
https://github.com/EmilyWill330/TEMPipeline.git

4.7. References
(1) X. Peng, L. Manna, W. Yang, J. Wickham, E. Scher, A. Kadavanich and A. P. Alivisatos,
Nature, 2000, 404, 59–61.

(2) S. Link and M. A. El-Sayed, Int. Rev. Phys. Chem., 2000, 19, 409–453.

(3) M. Kruszynska, H. Borchert, A. Bachmatiuk, M. H. Rümmeli, B. Büchner, J. Parisi and J.
Kolny-Olesiak, ACS Nano, 2012, 6, 5889–5896.

(4) A. Leonardi and M. Engel, ACS Nano, 2018, 12, 9186–9195.
198

(5) J. Billet, W. Dujardin, K. De Keukeleere, K. De Buysser, J. De Roo and I. Van Driessche,
Chem. Mater., 2018, 30, 4298–4306.

(6) Q. Liu, A. Díaz, A. Prosvirin, Z. Luo and J. D. Batteas, Nanoscale, 2014, 6, 8935–8942.

(7) Y. Fu, H. Zhu, J. Chen, M. P. Hautzinger, X.-Y. Zhu and S. Jin, Nat. Rev. Mater., 2019, 4,
169–188.

(8) R. Narayanan and M. A. El-Sayed, Nano Lett., 2004, 4, 1343–1348.

(9) M. Subhramannia and V. K. Pillai, J. Mater. Chem., 2008, 18, 5858.

(10) J. Shi, K. Lei, W. Sun, F. Li, F. Cheng and J. Chen, Nano Res., 2017, 10, 3836–3847.

(11) L. Boselli, H. Lopez, W. Zhang, Q. Cai, V. A. Giannone, J. Li, A. Moura, J. M. de Araujo,
J. Cookman, V. Castagnola, Y. Yan and K. A. Dawson, Commun. Mater., 2020, 1, 35.

(12) C. J. Murphy and J. M. Buriak, Chem. Mater., 2015, 27, 4911–4913.

(13) W. D. Pyrz and D. J. Buttrey, Langmuir, 2008, 24, 11350–11360.

(14) A. Kreshuk, U. Koethe, E. Pax, D. D. Bock and F. A. Hamprecht, PLoS ONE, 2014, 9,
e87351.

(15) J. M. Sosa, D. E. Huber, B. Welk and H. L. Fraser, Integrating Mater. Manuf. Innov., 2014,
3, 123–140.

(16) H. Wen, X. Xu, S. Cheong, S.-C. Lo, J.-H. Chen, S. L. Y. Chang and C. Dwyer, Nanoscale
Adv., 2021, 3, 6956–6964.

(17) M. Schorb, I. Haberbosch, W. J. H. Hagen, Y. Schwab and D. N. Mastronarde, Nat.
Methods, 2019, 16, 471–477.

(18) Y. Qian, J. Z. Huang, X. Li and Y. Ding, IEEE Trans. Image Process., 2016, 25, 5713–
5726.

(19) Chiwoo Park, J. Z. Huang, J. X. Ji, and Yu Ding, IEEE Trans. Pattern Anal. Mach. Intell.,
2013, 35, 1–1.

(20) L. Cervera Gontard, D. Ozkaya and R. E. Dunin-Borkowski, Ultramicroscopy, 2011, 111,
101–106.

(21) B. Lee, S. Yoon, J. W. Lee, Y. Kim, J. Chang, J. Yun, J. C. Ro, J.-S. Lee and J. H. Lee, ACS
Nano, 2020, 14, 17125–17133.
199

(22) X. Hu, F. Li, F. Xia, X. Guo, N. Wang, L. Liang, B. Yang, K. Fan, X. Yan and D. Ling, J.
Am. Chem. Soc., 2020, 142, 1636–1644.

(23) B. Lim, P. H. C. Camargo and Y. Xia, Langmuir, 2008, 24, 10437–10442.

(24) G. Barim, S. R. Smock, P. D. Antunez, D. Glaser and R. L. Brutchey, Nanoscale, 2018, 10,
16298–16306.

(25) M. Calvaresi, Nat. Nanotechnol., 2020, 15, 512–513.

(26) K. A. Brown, S. Brittman, N. Maccaferri, D. Jariwala and U. Celano, Nano Lett., 2020, 20,
2-10.

(27) S. Sacanna, D. J. Pine and G.-R. Yi, Soft Matter, 2013, 9, 8096.

(28) E. Kim, K. Huang, A. Saunders, A. McCallum, G. Ceder and E. Olivetti, Chem. Mater.,
2017, 29, 9436–9444.

(29) C. R. Laramy, K. A. Brown, M. N. O’Brien and Chad. A. Mirkin, ACS Nano, 2015, 9,
12488–12495.

(30) X. Wang, J. Li, H. D. Ha, J. C. Dahl, J. C. Ondry, I. Moreno-Hernandez, T. Head-Gordon
and A. P. Alivisatos, JACS Au, 2021, 1, 316–327.

(31) L. Bezinge, R. M. Maceiczyk, I. Lignos, M. V. Kovalenko and A. J. DeMello, ACS Appl
Mater Interfaces, 2018, 10, 18869-18878.

(32) O. Wetzel, O. Prymak, K. Loza, N. Gumbiowski, M. Heggen, P. Bayer, C. Beuck, C.
Weidenthaler and M. Epple, Inorg. Chem., 2022, 61, 5133–5147.

(33) H. Tao, T. Wu, M. Aldeghi, T. C. Wu, A. Aspuru-Guzik and E. Kumacheva, Nat. Rev.
Mater., 2021, 6, 701-716.

(34) E. J. Braham, R. D. Davidson, M. Al-Hashimi, R. Arróyave and S. Banerjee, Dalton Trans.,
2020, 49, 11480-11488.

(35) H. Kim, J. Han and T. Y.-J. Han, Nanoscale, 2020, 12, 19461-19469.

(36) C. Park and Y. Ding, MRS Commun., 2019, 9, 545–555.

(37) L. Mill, D. Wolff, N. Gerrits, P. Philipp, L. Kling, F. Vollnhals, A. Ignatenko, C. Jaremenko,
Y. Huang, O. De Castro, J. Audinot, I. Nelissen, T. Wirtz, A. Maier and S. Christiansen,
Small Methods, 2021, 2100223.

200
(38) C. K. Groschner, C. Choi and M. C. Scott, Microsc. Microanal., 2021, 27, 549–556.

(39) H. Wen, J. M. Luna-Romera, J. C. Riquelme, C. Dwyer and S. L. Y. Chang, Nanomaterials,
2021, 11, 2706.

(40) S. R. Spurgeon, C. Ophus, L. Jones, A. Petford-Long, S. V. Kalinin, M. J. Olszta, R. E.
Dunin-Borkowski, N. Salmon, K. Hattar, W.-C. D. Yang, R. Sharma, Y. Du, A.
Chiaramonti, H. Zheng, E. C. Buck, L. Kovarik, R. L. Penn, D. Li, X. Zhang, M. Murayama
and M. L. Taheri, Nat. Mater., 2021, 20, 274–279.

(41) L. Yao, Z. Ou, B. Luo, C. Xu and Q. Chen, ACS Cent. Sci., 2020, 6, 1421–1430.

(42) N. O’Mahony, S. Campbell, A. Carvalho, S. Harapanahalli, G. V. Hernandez, L. Krpalkova,
D. Riordan and J. Walsh, in Advances in Computer Vision, eds. K. Arai and S. Kapoor,
Springer International Publishing, Cham, 2020, vol. 943, pp. 128–144.

(43) N. Artrith, K. T. Butler, F.-X. Coudert, S. Han, O. Isayev, A. Jain and A. Walsh, Nat. Chem.,
2021, 13, 505–508.

(44) B. Lin, N. Emami, D. A. Santos, Y. Luo, S. Banerjee and B.-X. Xu, Npj Comput. Mater.,
2022, 8, 88.

(45) C. Park, J. Z. Huang, D. Huitink, S. Kundu, B. K. Mallick, H. Liang and Y. Ding, IIE Trans.,
2012, 44, 507–522.

(46) L. Fox, Math. Gaz., 1969, 53, 222–224.

(47) S. Shalev-Shwartz and S. Ben-David, Understanding machine learning: from theory to
algorithms, Cambridge University Press, New York, NY, USA, 2014.

(48) MathWorks Statistics and Machine Learning Toolbox: Documentation, MathWorks, 2022.
(49) G. Yamankurt, E. J. Berns, A. Xue, A. Lee, N. Bagheri, M. Mrksich and C. A. Mirkin, Nat.
Biomed. Eng., 2019, 3, 318–327.

(50) S. Wei, Y. Yang, X. Kang, L. Wang, L. Huang and D. Pan, Chem. Commun., 2016, 52,
7265–7268.

(51) L. Protesescu, S. Yakunin, M. I. Bodnarchuk, F. Krieg, R. Caputo, C. H. Hendon, R. X.
Yang, A. Walsh and M. V. Kovalenko, Nano Lett., 2015, 15, 3692–3696.

(52) D. Dresen, A. Qdemat, S. Ulusoy, F. Mees, D. Zákutná, E. Wetterskog, E. Kentzinger, G.
Salazar-Alvarez and S. Disch, J. Phys. Chem. C, 2021, 125, 23356–23363.

(53) C. B. Murray, D. J. Norris and M. G. Bawendi, J. Am. Chem. Soc., 1993, 115, 8706–8715.

201
(54) M. Feurer and F. Hutter, in Automated Machine Learning, eds. F. Hutter, L. Kotthoff and J.
Vanschoren, Springer International Publishing, Cham, 2019, pp. 3–33.

(55) M. Stone, Biometrika, 1974, 61, 509–515.

202
Chapter 5. Predictive Synthesis of Copper Selenides through the Construction of a
Multidimensional Phase Map using a Data-Driven Classifier*

*Manuscript in preparation.

5.1. Abstract
Copper selenides are an important family of technological materials with applications in
catalysis, plasmonics, photovoltaics, and thermoelectrics. Despite being a simple binary material,
the Cu–Se phase diagram is complex and contains multiple crystal structures, in addition to several
metastable structures that are not found on thermodynamic phase diagrams. Consequently, being
able to rationally navigate this complex phase space by simply tuning experimental parameters
poses a significant challenge. We demonstrate that data-driven learning can successfully map this
phase space in a minimal number of experiments. We combine chimie douce synthetic methods
with multivariate analyses via classification techniques to enable predictive phase determination.
A surrogate model was constructed with experimental data derived from a design matrix for four
experimental variables: diselenide precursor C–Se bond strength, time, temperature, and solvent
composition. The reactions in the surrogate model resulted in 11 distinct phase combinations of
copper selenide. This data was used to train a classification model that predicts phase with 95.7%
accuracy. Analysis of the resultant decision tree enabled detailed conclusions to be drawn about
how the experimental variables affect phase and provided prescriptive synthetic conditions for
phase isolation. This guided the accelerated phase targeting of klockmannite CuSe, which could
not be isolated as a phase pure material in the reactions used to construct the surrogate model. The
reaction conditions the model predicted to synthesize klockmannite CuSe were experimentally
validated, highlighting the utility of this approach.

203
5.2. Introduction
The Materials Genome Initiative has accelerated materials discovery through efforts such
as the Materials Project, which combines supercomputing and density functional theory (DFT) to
theoretically predict new materials and their properties before they are made.
1,2
While this
“materials by design” approach has successfully identified vast numbers of materials with a wide
range of targeted properties, a significant bottleneck now exists at the next step of the process: the
Edisonian and by-hand nature of materials synthesis. When attempting to synthesize materials,
there is no robust predictive framework that can help map the reaction coordinate from precursors
to the final crystalline solid throughout nucleation and growth. Moreover, the compositions and
structure types of crystalline inorganic solids are so disparate that it is exceptionally challenging
to apply the lessons learned from one system to another. In fact, even “simple” binary systems can
possess complex thermodynamic phase diagrams that are synthetically difficult to navigate. For
example, we invested multiple years of research to achieve phase control for binary nickel sulfide
nanocrystals, where the key synthetic handle ended up being a nonintuitive, second-order
interaction between the sulfur precursor and the surface ligand that distinguished between the
Ni3S4, 𝛼-NiS, 𝛽-NiS, Ni9S8 and Ni3S2 phases.
3
Additionally, Brock and co-workers studied the
effects of “synthetic levers” on the synthesis of binary nickel phosphides, where intricate webs of
numerous reaction pathways had to be decoupled in order to obtain the desired phase pure products
of Ni2P, Ni5P4, NiP2 and Ni12P5.
4,5

Binary copper selenides are an important family of materials with applications in catalysis,
plasmonics, photovoltaics, and thermoelectrics.
6–10
These wide-ranging functional properties are
a direct result of the rich and complex Cu–Se phase diagram that encompasses compositions
spanning from Cu2Se to CuSe2 with multiple distinct crystal structures (Table 5.1).
11
The most
204
common polymorphs reported in the literature are berzelianite Cu2−xSe (cubic, space group
𝐹𝑚3
,
𝑚), umangite Cu3Se2 (tetragonal, space group 𝑃4
,
2
'
𝑚), klockmannite CuSe (𝛾-CuSe –
hexagonal, space group 𝑃6
)
/𝑚𝑚𝑐, transforms at lower temperatures into b-CuSe – orthorhombic,
space group 𝐶𝑚𝑐𝑚), pyritic krutaite CuSe2 (cubic, space group 𝑃𝑎3
,
), and marcasitic krutaite
CuSe2 (orthorhombic, space group 𝑃𝑛𝑛𝑚). In recent years, several additional metastable copper
selenide polymorphs that do not exist on thermodynamic phase diagrams have been reported,
including weissite-like Cu2–xSe (trigonal, space group 𝑃3
,
𝑚1) and wurtzite-like Cu2–xSe
(hexagonal, space group 𝑃63𝑚𝑐).
12,13
In addition to their polymorphic diversity eliciting an array
of unique physicochemical properties, copper selenides have been found to function as essential
binary intermediates in the synthesis of higher-order, multinary copper selenides that adopt
topologically related anion sublattices.
9
For example, umangite Cu3Se2, which has a nearly
hexagonal Se
2–
sublattice, is an important intermediate in the synthesis of the metastable, wurtzite-
like phases of CuInSe2, Cu2ZnSnSe4, and Cu2FeSnSe4.
14–16

Table 5.1. Polymorphs and crystallographic information of binary copper selenides.
Phase Structure
Space
Group
Select XRD
Peaks 2θ/°
Temperature Range on
Bulk Phase Diagram
α-Cu2-xSe berzelianite Anti-fluorite (cubic) 𝐹𝑚3
)
𝑚 44, 27, 52 <105°C
Cu2Se Cubic 𝐹4
)
3𝑚
Cu2Se Cubic 𝐹 2 3
β-Cu2-xSe (Cu1.95Se) Rhombohedral (trigonal) 𝑅3
)
𝑚𝐻 44, 13, 27, 40, 52 105~1148°C
Cu2Se Monoclinic 𝑃121/𝑐1
Cu2Se Monoclinic 𝐶1𝑚1
Cu2Se Orthorhombic 𝐼𝑚𝑚2
Cu2Se Orthorhombic 𝑃𝑚𝑛2_1
Cu2Se Tetragonal 𝑃42/𝑛
“Weissite-like” Cu2-xSe Trigonal 𝑃3
)
𝑚1 45, 48, 26, 29 metastable
“Wurtzite-like” Cu2-xSe Hexagonal 𝑃63𝑚𝑐 metastable
Umangite Cu3Se2 Tetragonal 𝑃4
)
2
)
𝑚 25, 50, 52, 40 <113°C
klockmannite
α-Cu1-xSe Hexagonal 𝑃6
*
/𝑚 28, 46, 31 <54°C
β-Cu1-xSe Orthorhombic 𝐶𝑚𝑐𝑚 45, 28, 31 54~137°C
γ-Cu1-xSe Hexagonal CuS 𝑃6
*
/𝑚𝑚𝑐 28, 46, 31, 50, 56 137~380°C
CuSe Tetragonal 𝑃4/𝑛𝑚𝑚1
Marcasitic Krutaite CuSe2-I Orthorhombic 𝑃𝑛𝑛𝑚 29, 33, 34, 46 <332°C, low pressure phase
Pyritic Krutaite CuSe2-II Cubic 𝑃𝑎3
)
33, 36, 29, 49 <332°C, high pressure phase
205

Traditional high-temperature solid-state techniques generally do not provide sufficient
synthetic control to trap metastable products at ambient temperature and pressure, which makes
navigating the complex phase diagrams for systems like Cu–Se difficult.
17,18
This is because the
solid-state synthesis of copper selenides relies on supplying the reaction vessel with an excess of
energy to overcome the kinetic barriers of solid-state diffusion, which typically drives the
formation of the most thermodynamically stable products, with only a few notable exceptions.
19–
21
Alternatively, chimie douce (or soft chemistry) methods increase the probability of isolating
phases across the entire phase diagram because of the possibility of exerting kinetic control.
22
The
power of chimie douce lies in the expansiveness of its experimental variable space; however, this
presents a significant challenge in and of itself. That is, the targeted synthesis of a single material
can be difficult because of the large number of variables that must be controlled to obtain phase
purity. Consequently, chimie douce materials syntheses are developed via fragmented empirical
knowledge of the underlying ramifications of experimental variables.
23
For this reason, achieving
phase control in such syntheses can be extremely tedious and laborious.
Optimizing the synthesis of a phase pure material is traditionally done using the one-
variable-at-a-time (OVAT) method, where only one variable is changed at a time while all the
other variables are held constant. In an experimental domain where n experimental variables create
an n-dimensional design space, this one-dimensional approach is not only time and labor intensive,
but inefficient in revealing potential higher-order interactions between experimental variables and
their effects on synthetic outcomes, such as phase. The inability to map a more complete picture
of how the experimental variable space correlates to phase determination is limiting. One solution
is to utilize data-driven learning to render synthetic phase maps that allow for rational targeting of
206
materials within the high-dimensional variable space. Because phase is a categorical (or discrete)
outcome, the usual regression-based multivariate techniques like design of experiments (DoE) and
response surface methodology (RSM) cannot be used because they require continuous outcomes.
24

Deep learning techniques like convolutional neural networks can map the multidimensional
variable space, but they require large datasets that are not feasible when novel chemistry is being
employed and/or done in a low-throughput manner. On the other hand, training a classification
algorithm can handle both smaller data sets and categorical variables, making it a tractable solution
to this problem.
Herein, we combine the chimie douce synthesis of copper selenides with multivariate
analyses via data-driven classification to accelerate predictive phase determination. After training
and testing a classification algorithm on reaction data, a synthetic phase map was constructed that
encompasses a wide experimental variable space beyond the usual thermodynamic variables of
composition and temperature. The resulting multidimensional phase map allows an outcome to be
predicted for a given set of experimental variables. We also show that the phase map can guide the
synthesis of phase pure klockmannite CuSe, which was not accessible under any set of synthetic
conditions used in the training dataset. This is the first example of constructing a multidimensional
phase map using data-driven classifiers, which streamlines the synthesis of distinct phases of
copper selenide, including two metastable phases not found on the thermodynamic Cu–Se phase
diagram.

5.3. Results and Discussion
Our solution-phase synthesis of copper selenides is based on the relatively low-temperature
reaction of Cu(oleate)2 with diorganyl diselenide precursors. Diorganyl diselenides (R-Se-Se-R,
207
where R = alkyl, allyl, benzyl, or phenyl) have emerged over the past 15 years as versatile
precursors for the kinetically controlled synthesis of metal selenides.
25–29
The utility of these
precursors stems from their programmable reactivity as a function of C–E bond strength, which
depends upon the identity of the functional group, as first proposed by Vela and co-workers.
30

However, it has been shown by Macdonald and co-workers that this reactivity, along with the
mechanisms of precursor conversion, concurrently depend upon the presence of Cu(oleate)2 and
oleylamine;
31
a C18 primary amine that acts as both a high-temperature solvent and reducing
agent.
32,33
This illustrates the high-dimensional nature of this chemistry, which complicates
rational phase determination. We use a classification algorithm that identifies patterns in the
reaction data to subsequently predict the copper selenide phase, or combination of phases, for a
given set of experimental variables. To accomplish this, a surrogate model was created that
consists of reaction data sampled throughout the experimental space.
34,35

5.3.1. Construction of the Surrogate Model
A surrogate model provides the classification algorithm with the training and testing data
required to map the patterns of the experimental variables and identify which variables are the
most important for phase determination. DoE screening and optimization matrices were utilized to
construct the surrogate model. Despite the inability of the data to be modeled via regression
because of its discrete nature, DoE provides an orthogonally balanced experimental sampling of
the design space. This prevents the over- and under-sampling of any one region in the n-
dimensional domain, which can occur when other techniques like random sampling are used.
24

First, the variables to be investigated must be chosen and the synthetic bounds of the experimental
space must be set. The variables initially chosen for this investigation were: (1) C–Se precursor
208
bond strength, (2) volumetric ratio of oleylamine to 1-octadecene (ODE), (3) temperature, and (4)
time. To experimentally vary the C–Se bond strength, we chose two different diselenide precursors
with C-Se bonds that differ by 22 kcal mol
–1
.
14
The less reactive Ph2Se2 precursor has a stronger
C–Se bond (BDE = 65 kcal mol
–1
), while the more reactive Bn2Se2 precursor has a weaker C–Se
bond (BDE = 43 kcal mol
–1
). By limiting the diselenide precursors to two disparate C–Se bond
strengths, we minimize the number of categorical variables in our screening, since categorical
variables cause an exponential increase in the number of experiments required to satisfy a full
screening design.
The volumetric ratio of oleylamine to ODE was chosen because of the known influence of
oleylamine on the decomposition mechanism of the diselenide precursors, in addition to its ability
to act as a reducing agent for copper.
31,33
This allows the amount of oleylamine to be varied
continuously while keeping the overall reaction concentration constant. Time and temperature
were chosen due to their direct influence on the kinetics and thermodynamics of the reaction. The
surrogate model was limited to these variables since the cost of increasing the number of variables
and the corresponding number of required experiments outweighed the insight that would be
gained. The bounds of the experimental space are provided in Table 5.2. The ranges of the
continuous variables were chosen because values above or below these bounds rendered the
reaction either unsuccessful (no product), too time consuming, or unfeasible due to synthetic
limitations (e.g., temperatures significantly past solvent boiling points).

Table 5.2. The bounds of the binary Cu-Se phase space investigated for the surrogate model.
Bound C–Se Bond Strength (kcal mol
–1
) Oleylamine:ODE (v/v) Temperature (˚C) Time (min)
High Ph 2Se 2 1:0 320 120
Low Bn 2Se 2 1:20 170 1

209
The reactions performed to construct the surrogate model were defined by (1) two full
factorial screening matrices consisting of 16 experiments for each diselenide precursor, plus two
center points (i.e., two 2
3
factorials, 34 experiments, Table 5.3), (2) two Doehlert optimization
matrices for three variables; one 13-experiment matrix for each diselenide precursor (26
experiments, Table 5.4), and (3) 20 additional experiments from previous experimental data
accrued in our laboratory, including replicates to assess statistical significance (Table 5.5).

Table 5.3. Screening reactions and corresponding responses for the surrogate model.
Rxn. Precursor Time Temp. OAm:ODE Time Temp. OAm Phase
coded coded coded min ˚C % vol
1 Ph 2Se 2 -1 -0.6 1 1 200 100 Wurtzite-like Cu 2-xSe/umangite Cu 3Se 2
2 Ph 2Se 2 -1 -0.6 -1 1 200 5 Berzelianite Cu 2-xSe
3 Bn 2Se 2 -1 -0.56 -1 1 202.9 5 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
4 Bn 2Se 2 -0.513 -0.6 1 30 200 100 Berzelianite Cu 2-xSe/klockmannite Cu 1-
xSe 5 Ph 2Se 2 -0.513 -0.573 -1 30 202 5 Weissite-like Cu 2-xSe/umangite Cu 3Se 2
6 Bn 2Se 2 -1 -0.556 1 1 203.3 100 Berzelianite Cu 2-xSe
7 Ph 2Se 2 1 1 1 120 320 100 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
8 Bn 2Se 2 1 1 1 120 320 100 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
9 Ph 2Se 2 1 -0.59 1 120 200.7 100 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
10 Ph 2Se 2 -1 1 1 1 320 100 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
11 Ph 2Se 2 -1 1 -1 1 320 5 Berzelianite Cu 2-xSe
12 Ph 2Se 2 1 -0.59 -1 120 200.6 5 Berzelianite Cu 2-xSe
13 Ph 2Se 2 1 1 -1 120 320 5 Berzelianite Cu 2-xSe
14 Bn 2Se 2 -1 1 1 1 320 100 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
15 Bn 2Se 2 -1 1 -1 1 320 5 Berzelianite Cu 2-xSe/klockmannite CuSe
16 Bn 2Se 2 1 -0.6 -1 120 200 5 Berzelianite Cu 2-xSe/klockmannite CuSe
17 Bn 2Se 2 1 1 -1 120 320 5 Berzelianite Cu 2-xSe/klockmannite CuSe
18 Bn 2Se 2 1 -0.6 1 120 200 100 Berzelianite Cu 2-xSe/klockmannite CuSe
19 Bn 2Se 2 -0.008 0.2 -0.053 60 260 50 Berzelianite Cu 2-xSe
20 Ph 2Se 2 -0.008 0.205 -0.053 60 260 50 Berzelianite Cu 2-xSe
21 Ph 2Se 2 -0.513 -0.947 1 30 174 100 Weissite-like Cu 2-xSe/wurtzite-like Cu 2-
xSe 22 Ph 2Se 2 -0.513 0.4 1 30 275 100 Berzelianite Cu 2-xSe/umangite Cu 3Se 2
23 Ph 2Se 2 -0.513 -0.567 -1 30 202.5 5 Berzelianite Cu 2-xSe
24 Ph 2Se 2 -0.933 -0.707 1 5 192 100 Weissite-like Cu 2-xSe/wurtzite-like Cu 2-
xSe 25 Ph 2Se 2 -0.513 -0.691 1 30 193.2 100 Weissite-like Cu 2-xSe
26 Ph 2Se 2 -0.513 -0.597 -0.684 30 200.2 20 Berzelianite Cu 2-xSe
27 Bn 2Se 2 -0.513 -0.588 1 30 200.9 100 Berzelianite Cu 2-xSe/klockmannite CuSe
28 Bn 2Se 2 1 -0.519 1 120 206.1 100 Berzelianite Cu 2-xSe/klockmannite CuSe
29 Ph 2Se 2 1 -0.597 1 120 200.2 100 Berzelianite Cu 2-xSe
30 Bn 2Se 2 -1 -0.548 1 1 203.9 100 Berzelianite Cu 2-xSe/klockmannite CuSe
31 Bn 2Se 2 -0.513 0.525 1 30 284.4 100 Berzelianite Cu 2-xSe/klockmannite CuSe
32 Bn 2Se 2 -0.513 -0.441 1 30 211.9 100 Berzelianite Cu 2-xSe/klockmannite CuSe
33 Bn 2Se 2 -0.513 -0.553 -1 30 203.5 5 Klockmannite CuSe/umangite Cu 3Se 2
34 Ph 2Se 2 -1 -0.653 1 1 196 100 Weissite-like Cu 2-xSe
210
Table 5.4. Full Doehlert optimization design and corresponding responses for the surrogate model.

Table 5.5. Additional Copper Selenide experiments from previous studies.
Rxn. Precursor Time Temp. OAm:ODE Time Temp. OAm Phase
coded coded coded min ˚C % vol
1 Ph2Se2 -0.513 -0.573 1 30 202 100 Weissite-like Cu2-xSe/umangite Cu3Se2
2 Ph2Se2 1 -0.657 1 120 196.2 100 Berzelianite Cu2-xSe/weissite-like Cu2-xSe
3 Ph2Se2 -0.513 -0.46 -0.684 30 210.2 20 Umangite Cu3Se2
4 Ph2Se2 -0.932 -0.38 1 5 216.5 100 Umangite Cu3Se2
5 Ph2Se2 -0.933 -0.573 1 5 202 100 Umangite Cu3Se2
6 Ph2Se2 -1 -0.356 1 1 218.3 100 Weissite-like Cu2-xSe/umangite Cu3Se2
7 Ph2Se2 -1 -0.54 1 1 204.5 100 Umangite Cu3Se2
8 Ph2Se2 -1 -0.6 1 1 200 100 Weissite-like Cu2-xSe
9 Bn2Se2 -0.25 -0.56 -1 45.63 203 5 Berzelianite Cu2-xSe/klockmannite CuSe
10 Bn2Se2 1 -0.89 -0.312 120 178.3 37.7 Berzelianite Cu2-xSe
11 Bn2Se2 -0.03 -0.9 -1 58.73 177.5 5 Berzelianite Cu2-xSe/klockmannite CuSe
12 Bn2Se2 -1 -1 -1 1 170 5 Berzelianite Cu2-xSe/umangite Cu3Se2
13 Bn2Se2 -0.77 -1 -1 15 170 5 Berzelianite Cu2-xSe
14 Bn2Se2 -0.51 -1 -1 30 170 5 Berzelianite Cu2-xSe
15 Bn2Se2 0.245 -1 -1 75 170 5 Berzelianite Cu2-xSe/klockmannite CuSe
16 Bn2Se2 0 -1 -1 60.5 170 5 Berzelianite Cu2-xSe/klockmannite CuSe
17 Bn2Se2 -0.26 -1 -1 45 170 5 Berzelianite Cu2-xSe/klockmannite CuSe
18 Bn2Se2 -1 -0.8 -1 1 185 5 Berzelianite Cu2-xSe
19 Bn2Se2 -0.77 -0.8 -1 15 185 5 Berzelianite Cu2-xSe/klockmannite CuSe
20 Bn2Se2 -0.51 -0.8 -1 30 185 5 Berzelianite Cu2-xSe/klockmannite CuSe
Rxn. Precursor Time Temp. OAm:ODE Time Temp. OAm Phase
coded coded coded min ˚C % vol
1 Ph2Se2 -1 0 0 1 245 52.2 Weissite-like Cu2-xSe/umangite Cu3Se2
2 Bn2Se2 -1 0 0 1 245 52.5 Berzelianite Cu2-xSe/klockmannite CuSe
3 Bn2Se2 1 0 0 120 245 52.5 Berzelianite Cu2-xSe
4 Ph2Se2 1 0 0 120 245 52.5 Berzelianite Cu2-xSe/umangite Cu3Se2
5 Ph2Se2 0 0 0 60.5 245 52.5 Berzelianite Cu2-xSe/umangite Cu3Se2
6 Bn2Se2 0 0 0 60.5 245 52.5 Berzelianite Cu2-xSe
7 Ph2Se2 -0.5 -0.866 0 30.75 180.1 52.5 Berzelianite Cu2-xSe
8 Ph2Se2 0.5 0.866 0 90.25 309.9 52.5 Berzelianite Cu2-xSe/umangite Cu3Se2
9 Bn2Se2 0.5 0.289 0.817 90.25 266.7 91.3 Berzelianite Cu2-xSe
10 Ph2Se2 0.5 0.289 0.817 90.25 266.7 91.3 Wurtzite-like Cu2-xSe
11 Ph2Se2 -0.5 -0.866 0 30.75 180.1 52.5 Wurtzite-like Cu2-xSe/umangite Cu3Se2
12 Ph2Se2 -0.5 -0.289 -0.817 30.75 223.3 13.7 Berzelianite Cu2-xSe/umangite Cu3Se2
13 Ph2Se2 0.5 -0.866 0 90.25 180.1 52.5 Weissite-like Cu2-xSe
14 Ph2Se2 0.5 -0.289 -0.817 90.25 223.3 13.7 Berzelianite Cu2-xSe
15 Ph2Se2 -0.5 0.866 0 30.75 309.9 52.5 Berzelianite Cu2-xSe/umangite Cu3Se2
16 Ph2Se2 0 0.577 -0.817 60.5 288.3 13.7 Berzelianite Cu2-xSe
17 Ph2Se2 0.5 0.289 0.817 90.25 266.7 91.3 Wurtzite-like Cu2-xSe
18 Ph2Se2 0 -0.577 0.817 60.5 201.7 91.3 Weissite-like Cu2-xSe/berzelianite Cu2-xSe
19 Bn2Se2 -0.5 -0.866 0 30.75 180.1 52.5 Berzelianite Cu2-xSe/klockmannite CuSe
20 Bn2Se2 -0.5 -0.289 -0.817 30.75 223.3 13.7 Berzelianite Cu2-xSe/klockmannite CuSe
21 Bn2Se2 0.5 -0.866 0 90.25 180.1 52.5 Berzelianite Cu2-xSe/klockmannite CuSe
22 Bn2Se2 0.5 -0.289 -0.817 90.25 223.3 13.7 Berzelianite Cu2-xSe/klockmannite CuSe
23 Bn2Se2 -0.5 0.866 0 30.75 309.9 52.5 Berzelianite Cu2-xSe/umangite Cu3Se2
24 Bn2Se2 0 0.577 -0.817 60.5 288.3 13.7 Berzelianite Cu2-xSe
25 Bn2Se2 0.5 0.289 0.817 90.25 266.7 91.3 Berzelianite Cu2-xSe/umangite Cu3Se2
26 Bn2Se2 0 -0.577 0.817 60.5 201.7 91.3 Berzelianite Cu2-xSe/klockmannite CuSe
211

This resulted in 80 total reactions for the surrogate model, which yielded 11 unique phase
combinations of copper selenides: (A) berzelianite Cu2–xSe, (B) berzelianite Cu2–xSe/klockmannite
CuSe, (C) umangite Cu3Se2/klockmannite CuSe, (D) berzelianite Cu2–xSe/umangite Cu3Se2, (E)
umangite Cu3Se2, (F) wurtzite-like Cu2–xSe /umangite Cu3Se2, (G) wurtzite-like Cu2–xSe, (H)
weissite-like Cu2–xSe/wurtzite-like Cu2–xSe, (I) weissite-like Cu2–xSe, (J) weissite-like Cu2–
xSe/umangite Cu3Se2, (K) berzelianite Cu2–-xSe/weissite-like Cu2–xSe. For simplicity, the unique
phase combinations will be referenced throughout the rest of the text by their letter and color codes,
indicated in Figure 5.1a, and the relative amounts of each phase present in a particular
combination will not be considered at this stage of the study. Powder X-ray diffraction (XRD)
patterns characterizing the four phase pure copper selenides (i.e., A, E, G, I) are given in Figure
5.1b. The conditions for each reaction and their resulting phase combination are proved in Tables
5.3-5.

Figure 5.1. (a) Coded and color identifiers for each of the 11 unique phase combinations of copper
selenide observed during construction of the surrogate model. (b) Powder XRD patterns of four
resulting phase pure copper selenides: berzelianite Cu2–xSe (A), umangite Cu3Se2 (E), wurtzite-
like Cu2–xSe (G), and weissite-like Cu2–xSe (I).

212
5.3.2. Mapping the Cu-Se Phase Space via Classification
Due to the discrete nature of categorical outcomes like phase (i.e., there is a fixed integer
number of possible responses) typical regression techniques cannot be used to optimize the
outcome of phase determination. This is because regression requires a continuous array of values.
Classification algorithms can deal with the complexity of a categorical outcome. If prediction
accuracy is statistically significant (typically, validation loss £ 0.05),
35
then these algorithms can
map the effects of several variables on a list of categorical outcomes, which, in this case, are the
unique phase combinations of binary copper selenides synthesized in the study.
The classification model was chosen by performing a Bayesian optimization of the
hyperparameters on the surrogate model data via the fitcauto function in MATLAB. After 120
iterations the best observed and estimated learner was an ensemble using the bag method over 254
learning cycles, with minimum leaf size equal to 21 and the maximum number of splits equal to
70. Specifically, bootstrap aggregation, or bagging with random predictor selections at each split
(random forest), was used as it was predicted to be best for the multiclass nature of the data. The
classification bagged ensemble model was trained with these optimized hyperparameters (as
specified by the Bayesian optimization) and leave-one-out cross-validation. This evaluation
method was chosen as it is ideal for smaller datasets, providing a much less biased measure of test
error compared to using a single test set because we repeatedly fit a model to a dataset that
contains n-1 observations.
36
Classification accuracy was 95.7 % with a validation loss = 0.043,
misclassifying only 3 reactions (Figure 5.2). Re-substitution loss was 0.0380, which equates to
misclassification rate. The closer the model predictions are to the observations, the smaller the
misclassification error will be, and error ≤ 0.05 is considered acceptable, rendering our model
statistically significant.
213

Figure 5.2. Confusion chart of the classification model predictions, showing correct predictions
in blue and incorrect predictions in orange.

The effects of the experimental variables on phase were analyzed, and importance scores
were calculated for the algorithm predictions. Univariate feature ranking for classification (fsschi2)
ranks features (predictors) using chi-square tests. The predictor variables and the response variable
(phase) from the training data were provided to a function that returns the indices of predictors
ordered by predictor importance. This means the first predictor returned is the most important
predictor. These results indicated that C–Se bond strength was the most significant variable in
determining phase, followed by temperature, volumetric ratio of oleylamine to ODE, and time,
respectively (Figure 5.3a). Interestingly, when the data sets were separate by C–Se bond strength
(or diselenide precursor), the variables had different rankings of importance (Figure 5.3b,c).
Although temperature remains the most important variable for both, the volumetric ratio of
oleylamine to ODE has a significantly greater effect on reactions that use the Bn2Se2 precursor.
214
This reinforces the notion that the mechanism of precursor conversion may differ between Bn2Se2
and Ph2Se2; for this reason, the two precursors will be assessed separately moving forward.

Figure 5.3. Relative importance scores of the experimental variables for (a) the entire surrogate
model and separated by (b) Ph2Se2 and (c) Bn2Se2 precursors.

After analysis of the experimental results, the Ph2Se2 precursor, which has the stronger C–
Se bond strength, resulted in nine distinct phase combinations of copper selenide: (A) berzelianite
Cu2–xSe, (D) berzelianite Cu2–xSe/umangite Cu3Se2, (E) umangite Cu3Se2, (F) wurtzite-like Cu2–
xSe /umangite Cu3Se2, (G) wurtzite-like Cu2–xSe, (H) weissite-like Cu2–xSe /wurtzite-like Cu2–xSe,
(I) weissite-like Cu2–xSe, (J) weissite-like Cu2–xSe/umangite Cu3Se2, and (K) berzelianite
Cu2–-xSe/weissite-like Cu2–xSe (Figure 5.4a). Copper selenide phase combinations E-K were
unique to the Ph2Se2 precursor. The Bn2Se2 precursor, with a weaker C–Se bond strength, resulted
in four distinct phase combinations of copper selenide: (A) berzelianite Cu2-xSe, (B) berzelianite
Cu2–xSe/klockmannite CuSe, (C) umangite Cu3Se2/klockmannite CuSe, and (D) berzelianite Cu2–
xSe/umangite Cu3Se2 (Figure 5.4b). Phase combinations B and C were unique to the Bn2Se2
precursor, revealing that only a precursor with a lower C–Se bond strength can form the
klockmannite CuSe phase, although not phase pure. Similarly, only a precursor with a greater C–
Se bond strength can form the metastable weissite-like and wurtzite-like Cu2–xSe phases, which
were both made phase pure. Both precursors can make phase pure berzelianite Cu2–xSe. This stands
215
in contrast with umangite Cu3Se2, the only copper selenide that we could isolate phase pure in our
previous synthetic explorations with these same diselenide precursors.
14

Figure 5.4. Each reaction in the surrogate model with their respective coded variable values for
(a) diphenyl diselenide precursor and (b) dibenzyl diselenide precursor. The resultant phase
combination for each reaction is indicated by color.

Extrapolating the reaction outcomes plotted within the phase space in Figures 5.5a,c using
a nearest neighbor likelihood algorithm yielded a prediction interpolant (i.e., a function that can
be evaluated at query points) of the 3-dimensional phase maps for each diselenide precursor, as
illustrated in Figures 5.5b,d. ScatteredInterpolant was used to perform the interpolation on the 3-
D dataset of scattered surrogate model data. This returns the interpolant, 𝐹, for the experimental
dataset. 𝐹 can be evaluated at a set of query points, such as (𝑥𝑞,𝑦𝑞,𝑧𝑞) in 3-D, to produce
predicted interpolated values 𝑣𝑞 = 𝐹(𝑥𝑞,𝑦𝑞,𝑣𝑞).

216

Figure 5.5. Visualization of the Cu–Se phase maps for the (a,b) Ph2Se2 and (c,d,) Bn2Se2
precursors. The data points in (a) and (c) represent experiments ran in the experimental space for
each respective precursor and are color coded to the phase outcome shown in the legend.

Looking specifically at the Ph2Se2 precursor, the stronger C–Se bond strength leads to a
higher energy barrier for precursor conversion, which offers a route to kinetic trapping. Such
kinetic trapping facilitates a much richer phase space, which is exemplified by the nine phase
combinations achieved using this precursor, versus the four phase combinations achieved with
Bn2Se2 precursor. With Ph2Se2, berzelianite Cu2−xSe forms under oleylamine-poor reaction
conditions, whereas umangite Cu3Se2 forms under more oleylamine-rich reaction conditions
(Figure 5.5b). This is counterintuitive since the more reducing conditions introduced with greater
217
volumetric ratios of oleylamine result in Cu3Se2, which formally contains Cu
2+
,
37
thus highlighting
value behind using the phase map.
The phase map in Figure 5.5b pinpoints the locations of the metastable weissite-like and
wurtzite-like Cu2–xSe phases, which indicate regions that border between thermodynamic and
kinetic stability. These regions are bound by the areas of lower temperatures and higher volumetric
ratios of oleylamine. The change in conditions that separate three phases, umangite Cu3Se2,
weissite-like Cu2−xSe, and wurtzite-like Cu2−xSe (E, G, and I, respectively), can be seen in greater
detail in the sub-region of the experimental space shown in Figure 5.6. The wurtzite-like
polymorph forms at higher temperatures and high volumetric ratios of oleylamine, which
transitions into the umangite Cu3Se2 phase within in a small window of slightly lower temperatures
and shorter reaction times. The phase map eventually transitions into the weissite-like Cu2–xSe
phase at longer reaction times, with a very small temperature difference of 2.5 ˚C separating
umangite Cu3Se2 and weissite-like Cu2-xSe. These results are consistent with umangite Cu3Se2
being a low-temperature phase on the thermodynamic phase diagram;
11
however, both weissite-
like and wurtzite-like Cu2–xSe are metastable phases that are not present on the thermodynamic
phase diagram and had not been observed in any of our previous exploratory chemistry with these
diselenide precursors. This illustrates the power of this approach; that is, we were able to isolate
two phase pure metastable materials within a complex experimental space where a reaction
temperature difference of only a few degrees can separate it from umangite Cu3Se2.

218

Figure 5.6. (a) Powder XRD patterns of phase combinations that result in the (b) sub-region of the
phase map that is bound by the area of lower temperatures and higher volumetric ratios of
oleylamine with the Ph2Se2 precursor.

That these three phases ((E) umangite Cu3Se2, (G) weissite-like Cu2−xSe, and (I) wurtzite-
like Cu2−xSe) all exist within this sub-region of the experimental space is perhaps not surprising
given the fact that both umangite Cu3Se2 and weissite-like Cu2–xSe can be viewed as slight
distortions of the wurtzite-like anionic sublattice.
12,38
The anion sublattice of the umangite Cu3Se2
structure maintains a quasi-planar hexagonal framework of Se
2−
that stack in an alternating ABAB
fashion. The interplanar distance between these anion layers is 3.2 Å, which is similar to the
anisotropic hexagonal wurtzite-like structure with the hexagonally close-packed layers stacked
along the c-axis with a d-spacing of 3.4 Å.
13,39
Similarly, in the weissite-like structure, Cu
+

occupies trigonal and tetrahedral sites in a slightly distorted hexagonal Se
2−
sublattice with a d-
spacing of 3.4 Å.
12
This structure has Cu-rich and Cu-deficient layers sandwiching a distorted
hexagonal layer of Se
2−
.
For the Bn2Se2 precursor, most of the experimental space returns mixtures of phases
(Figure 5.5c,d). As opposed to Ph2Se2, phase pure berzelianite Cu2−xSe forms under more
oleylamine-rich reaction conditions with Bn2Se2, illustrating that the relationship between the
219
diselenide precursor and oleylamine is counter-correlated between the two precursors and
therefore in accordance with the hypothesis that the two systems might be undergoing different
precursor conversion mechanisms. While most of this experimental space returns phase
combinations with berzelianite Cu2−xSe, the Bn2Se2 precursor yields phase combinations
containing klockmannite CuSe, whereas Ph2Se2 does not. Klockmannite possesses some degree of
Se–Se bonding in the structure.
40
Thus, klockmannite CuSe is observed with Bn2Se2 because its
Se–Se bond (BDE = 53 kcal mol
–1
) is 11 kcal mol
–1
stronger than the Se–Se bond in Ph2Se2;
therefore, some degree of precursor conversion may result in Se2
2–
.

Figure 5.7. Simplified routes of the Cu-Se phase map predicted by the classification model, with
Bn2Se2 precursor pathways to phase-pure products indicated by orange arrows and Ph2Se2
precursor pathways to phase-pure products indicated by blue arrows.

The trained model could then be visualized, which output a phase map in the form of a
decision tree. This shows specific variable values and the prescriptive synthetic routes to all pure
and mixed phase combinations. A simplified version of this decision tree illustrating the routes to
synthesize the phase pure copper selenides from this stage of the study is illustrated in Figure 5.7.
220
Unsurprisingly, precursor type was the first node on the full decision tree, so the full tree was
separated by precursor for simplicity in Figures 5.8,9.

Figure 5.8. Classification tree/phase map for the diphenyl dieselenide precursor.
221

Figure 5.9. Classification tree/phase map for the dibenzyl diselenide precursor.

5.3.3. Predictive Phase Determination of Klockmannite CuSe via Classification Model
To illustrate the power of classification as a tool for predictive phase determination, we
next address the inability to isolate phase pure klockmannite CuSe within the bounds of the
experimental space. Although klockmannite is observed within the Bn2Se2 portion of the phase
map, it is only found in combination with berzelianite Cu2–xSe or umangite Cu3Se2. Thus, the
model was used to predict a synthetic route to klockmannite CuSe using the Bn2Se2 precursor.
222
Like the bounded region of the metastable phases discussed for Ph2Se2 (vide supra), the target
phase was predicted to lie somewhere in the region near the boundary of klockmannite combined
with umangite Cu3Se2 (C) and klockmannite combined with berzelianite Cu2–xSe (B).

Figure 5.10. (a) Cu–Se phase map for the Bn2Se2 precursor, with the sub-region of interest circled
in black. The initial experimental conditions are identified by a black circle (C) and the target
conditions for synthesizing phase pure klockmannite CuSe are indicated by a star (M). (b) Le Bail
refinement of the XRD pattern collected on the mixture of klockmannite and umangite Cu3Se2 (C)
before phase targeting. (c) Le Bail refinement of the XRD pattern collected on the phase pure
klockmannite CuSe (M) after phase targeting. (d-f) Response surfaces giving the predicted relative
phase purity of klockmannite CuSe throughout the experimental space, with the initial
experimental conditions and target conditions shown by the black circle (C) and star (M).
223

This sub-region of the experimental space is characterized by short to medium reaction
times, low temperatures, and low volumetric ratios of oleylamine to ODE, as indicated by the
circled area in Figure 5.10a. The classification model predicted that this region of phase pure
klockmannite CuSe, coded M, can be specifically defined by the following conditions: 203.5 ˚C
≤ M <234.3 ˚C, M < 9.3 vol % of oleylamine in ODE, and 22.5 min ≤ M < 30.4 min (Figure
5.9). Interestingly, this set of conditions falls in the undefined region of the simplified decision
tree seen in Figure 5.7 that represents the four current phase pure products.
Using these synthetic guidelines, three reactions were conducted at 205, 215, and 225 ˚C
with aliquots taken at 1, 15, 30, 45, 60, and 120 min to better sample the phase outcome over time
in this region, since time was predicted to be the least significant variable. Since high volumetric
ratios of oleylamine to ODE were projected to hinder phase purity, it was kept at the minimum (5
vol %). The relative amounts of klockmannite CuSe synthesized in each reaction was qualitatively
estimated from peak intensities, with the reaction at 205 ˚C containing ca. 40% klockmannite CuSe
and 60% berzelianite Cu2–xSe, the reaction at 215 ˚C containing ca. 50% klockmannite and 50%
berzelianite, and the reaction at 225 ˚C containing ca. 80% klockmannite and 20% berzelianite
(Figure 5.11). The model predictions were experimentally validated by these initial aliquots,
which showed maximum fractions of klockmannite between 15-30 min for all three temperatures.
After 45 minutes, klockmannite CuSe growth plateaued at the lower two temperatures and the 225
˚C reaction began to produce umangite Cu3Se2. The mixture of klockmannite CuSe, umangite
Cu3Se2, and berzelianite Cu2–xSe from the reaction at 225 ˚C represents a twelfth unique phase
combination (coded L). This insight prompted two more reactions at 220˚C and 230 ˚C, where
aliquots were taken at 1, 5, 10, 15, 20, and 25 min. These reactions resulted in ca. 70%
klockmannite and 60% klockmannite, respectively (Figure 5.12). Reaction times ranging between
224
5-25 min seemed to have negligible effects on phase determination in this temperature range,
further validating the model’s prediction that time was the least important variable in determining
phase for this system.

Figure 5.11. Initial temperature screening of klockmannite phase synthesized via Bn2Se2 over time
at (a) 205 ˚C and (b) 215 ˚C and (c) 225 ˚C. Klockmannite stick pattern is indicated in purple.

Figure 5.12. Second iteration of temperature screening of klockmannite phase synthesized via
Bn2Se2 over time at (a) 220 ˚C and (b) 230 ˚C. Klockmannite stick pattern is indicated in purple.
225

These data were combined with the surrogate model experiments that yielded a phase
combination that included klockmannite. Response surface methodology was used to fit the data
to a polynomial model, resulting in the response surfaces shown in Figure 5.10d-f. After
extrapolating the data, the optimal reaction conditions were predicted to be a reaction time of 24.3
min, a reaction temperature of 223.5 ˚C, and 4.7 vol% oleylamine in ODE, corroborating the initial
prediction made by the classification model. A reaction was run under these predicted conditions
in triplicate to validate the model, which in each case identically resulted in phase pure
klockmannite CuSe. This represents the thirteenth unique phase or phase combination in this
experimental space (coded M). A refinement of the XRD pattern and crystal structure are given in
Figure 5.10c, which had a 𝜒
2
= 1.31. We compared this to a refinement of the XRD the highest
fraction of klockmannite CuSe from the original surrogate model (Figure 5.10b) from the reaction
at 203.5 ˚C, 30 min, and 5 vol % oleylamine in ODE, which had a 𝜒
2
= 5.70 and shows a
combination of umangite Cu3Se2 and klockmannite (C). These results demonstrate how the
classification algorithm allowed us to isolate the klockmannite phase in only six additional
experiments past the construction of the surrogate model.
A subsequent aliquot study at 223.5 ˚C (i.e., the predicted optimal temperature) was
conducted to assess if the predicted time of 24.3 min was significant, considering that time had
seemed to have negligible effects on phase only 3.5 ˚C lower in the study at 220˚C. The results
showed slight impurities at both 20 min and 30 min (Figure 5.13), indicating that phase pure
klockmannite inhabits an extremely minute region of the phase space and requires a very precise
set of reaction conditions. The small change in experimental conditions that shifts the synthetic
outcome from a binary phase mixture to phase pure klockmannite CuSe highlights the utility of
226
this approach to target a desired phase that was previously inaccessible in a small sub-region of a
large, high-dimensional experimental space.

Figure 5.13. Aliquot study of klockmannite phase synthesized via Bn2Se2 over time at 223.5 ˚C.
Klockmannite stick pattern is indicated in purple.

5.4. Conclusions
The large experimental space to synthesize binary copper selenides was mapped for four
variables: C–Se precursor bond strength (Ph2Se or Bn2Se), volumetric ratio of oleylamine to ODE,
reaction time, and temperature. Patterns in the data were analyzed using a data-driven classification
algorithm. After performing experiments dictated by orthogonal screening and optimization design
matrices, a surrogate model was created to provide experimental data for training and testing of a
classification model. Calculation of variable importance scores and multivariate, high-dimensional
phase maps created via the resulting classification tree and likelihood algorithms enabled detailed
conclusions to be drawn about the relationships between experimental variables and phase. The
type of diselenide precursor was shown to be the most important factor for phase determination,
followed by temperature, with certain phases lying in very narrow temperature ranges within the
phase map. The importance of the volumetric ratio of oleylamine to ODE and time depended on
227
the precursor type, suggesting that the resulting phase is dictated by different precursor conversion
mechanisms. The precursor with a higher C–Se bond strength (Ph2Se2) lead to a richer phase map
with more unique phase combinations, allowing the isolation of three distinct phases, including
two metastable phases.
The phase maps and insights from data-driven classification acted as a guide for the
accelerated isolation of phase pure klockmannite CuSe in just six additional experiments. The
isolation of this phase is significant because of the presence of Se–Se bonding within this structure,
as it is accessed by the Bn2Se2 precursor with the stronger Se–Se bond. This is the first example
of high-dimensional mapping of a multiphase, multivariate domain with mixed categorical and
discrete variables, and the first example of data-driven classification techniques being employed
to target a previously inaccessible phase within an experimental space. The resulting phase maps
not only streamline phase determination in the complex binary Cu–Se system studied here but will
be broadly applicable to the targeted chimie douce synthesis of other materials classes as well.

5.5. Experimental Procedures
5.5.1. Materials and General Procedures
Copper(II) dichloride dihydrate (CuCl2·2H2O, 99%, Sigma-Aldrich), sodium oleate
(>97%, TCI America), diphenyl diselenide (Ph2Se2, 98%, Sigma-Aldrich), dibenzyl diselenide
(Bn2Se2, 98%, Alfa Aesar), 1-octadecence (90%, Sigma-Aldrich), and oleylamine (70%, Sigma-
Aldrich) were obtained as indicated. Oleylamine and 1-octadecene were degassed under vacuum
at 120 °C for 4 h and then overnight at room temperature prior to use. Reactions were conducted
under a nitrogen atmosphere by using standard Schlenk techniques. All reactions employed J-KEM
228
temperature controllers with in-situ thermocouples to control and monitor the temperature of the
reaction vessel.

5.5.2. Synthesis of Copper(Oleate)2
An adapted literature approach was used.
14
Sodium oleate (3.0 g, 9.9 mmol) and
CuCl2·2H2O (0.84 g, 4.9 mmol) were placed in a round-bottom flask. A solution containing 10
mL of ethanol, 8 mL of DI water, and 17 mL of hexanes was added to the flask, and the reaction
mixture was heated to 70 °C. After 25 min, an additional 10 mL of hexanes was added to the
solution, and the flask was kept at 70 °C for 4 h. The product was collected in the hexanes layer,
separated, and washed three times with 30 mL of DI water in a separatory funnel. The hexanes
layer was collected, and all volatiles were removed to produce a blue-green Cu(oleate)2 product.

5.5.3. Synthesis of Copper Selenide
Cu(oleate)2 (0.16 g, 0.25 mmol) and R2Se2 (0.25 mmol, 0.0850 g R = Bn or 0.0785 g R =
Ph) were placed in a three-neck round-bottom flask and dissolved in 12 mL of varying volumetric
ratios of oleylamine and 1-octadecene under flowing nitrogen. The flask was then heated to 70 °C
and degassed for 30 min under vacuum. The temperature was raised to 140 °C and the flask was
degassed for an additional 30 min. At this point the solution is clear and has a teal color. The
reaction temperature was ramped to the indicated set point under flowing nitrogen at 5-6 °C·min
−1

and held at that temperature for the duration of the reaction. The reaction solution was then
thermally quenched by quickly removing it from the heating mantle and immediately placing it in
a room-temperature water bath. Hexanes (< 5 mL) were added to the reaction suspension, which
was then removed from the round-bottom flask and split equally between two 50 mL centrifuge
229
tubes that were filled to 40 mL with ethanol, sonicated for 10 min, and centrifuged at 6000 rpm
for 4.5 min. This washing procedure was repeated twice with 5 mL of hexanes and 40 mL of
ethanol. After the final centrifugation step, the copper selenide was isolated and dried under
flowing nitrogen at room temperature to give a powder for X-ray diffraction.

5.5.4. Characterization
Powder X-ray diffraction (XRD) measurements were made from 10-70˚ 2𝜃 with a step size
of 1˚ min
–1
on a Rigaku Ultima IV powder X-ray diffractometer using Cu Kα radiation (λ = 1.54
Å). Powder samples were prepared on a zero-diffraction silicon substrate.

5.6. References
(1) Jain, A.; Ong, S. P.; Hautier, G.; Chen, W.; Richards, W. D.; Dacek, S.; Cholia, S.; Gunter,
D.; Skinner, D.; Ceder, G.; Persson, K. A. Commentary: The Materials Project: A Materials
Genome Approach to Accelerating Materials Innovation. APL Mater. 2013, 1, 011002.

(2) Nosengo, N. Can Artificial Intelligence Create the next Wonder Material? Nature 2016, 533,
22–25.

(3) Barim, G.; Smock, S. R.; Antunez, P. D.; Glaser, D.; Brutchey, R. L. Phase Control in the
Colloidal Synthesis of Well-Defined Nickel Sulfide Nanocrystals. Nanoscale 2018, 10,
16298–16306.

(4) Li, D.; Senevirathne, K.; Aquilina, L.; Brock, S. L. Effect of Synthetic Levers on Nickel
Phosphide Nanoparticle Formation: Ni5P4 and NiP2. Inorg. Chem. 2015, 54, 7968–7975.

(5) Muthuswamy, E.; Savithra, G. H. L.; Brock, S. L. Synthetic Levers Enabling Independent
Control of Phase, Size, and Morphology in Nickel Phosphide Nanoparticles. ACS Nano 2011,
5, 2402–2411.

(6) Hessel, C. M.; Pattani, V. P.; Rasch, M.; Panthani, M. G.; Koo, B.; Tunnell, J. W.; Korgel,
B. A. Copper Selenide Nanocrystals for Photothermal Therapy. Nano Lett. 2011, 11, 2560–
2566.

230
(7) Lie, S. Q.; Wang, D. M.; Gao, M. X.; Huang, C. Z. Controllable Copper Deficiency in
Cu2−xSe Nanocrystals with Tunable Localized Surface Plasmon Resonance and Enhanced
Chemiluminescence. Nanoscale 2014, 6, 10289–10296.

(8) Dorfs, D.; Härtling, T.; Miszta, K.; Bigall, N. C.; Kim, M. R.; Genovese, A.; Falqui, A.;
Povia, M.; Manna, L. Reversible Tunability of the Near-Infrared Valence Band Plasmon
Resonance in Cu2–xSe Nanocrystals. J. Am. Chem. Soc. 2011, 133, 11175–11180.

(9) Tappan, B. A.; Brutchey, R. L. Polymorphic Metastability in Colloidal Semiconductor
Nanocrystals. ChemNanoMat 2020, 6, 1567–1588.

(10) Anantharaj, S.; Ede, S. R.; Sakthikumar, K.; Karthick, K.; Mishra, S.; Kundu, S. Recent
Trends and Perspectives in Electrochemical Water Splitting with an Emphasis on Sulfide,
Selenide, and Phosphide Catalysts of Fe, Co, and Ni: A Review. ACS Catal. 2016, 6, 8069–
8097.

(11) Glazov, V. M.; Pashinkin, A. S.; Fedorov, V. A. Phase Equilibria in the Cu-Se System. Inorg.
Mater. 2000, 36, 641–652.

(12) Lord, R. W.; Fanghanel, J.; Holder, C. F.; Dabo, I.; Schaak, R. E. Colloidal Nanoparticles of
a Metastable Copper Selenide Phase with Near-Infrared Plasmon Resonance. Chem. Mater.
2020, 32, 10227–10234.

(13) Gariano, G.; Lesnyak, V.; Brescia, R.; Bertoni, G.; Dang, Z.; Gaspari, R.; De Trizio, L.;
Manna, L. Role of the Crystal Structure in Cation Exchange Reactions Involving Colloidal
Cu2Se Nanocrystals. J. Am. Chem. Soc. 2017, 139, 9583–9590.

(14) Tappan, B. A.; Barim, G.; Kwok, J. C.; Brutchey, R. L. Utilizing Diselenide Precursors
toward Rationally Controlled Synthesis of Metastable CuInSe2 Nanocrystals. Chem. Mater.
2018, 30, 5704–5713.

(15) Tappan, B. A.; Chu, W.; Mecklenburg, M.; Prezhdo, O. V.; Brutchey, R. L. Discovery of a
Wurtzite-like Cu2FeSnSe4 Semiconductor Nanocrystal Polymorph and Implications for
Related CuFeSe2 Materials. ACS Nano 2021, 15, 13463–13474.

(16) Tappan, B. A.; Crans, K. D.; Brutchey, R. L. Formation Pathway of Wurtzite-like
Cu2ZnSnSe4 Nanocrystals. Inorg. Chem. 2021, 60, 17178–17185.

(17) Martinolich, A. J.; Neilson, J. R. Toward Reaction-by-Design: Achieving Kinetic Control of
Solid State Chemistry with Metathesis. Chem. Mater. 2017, 29, 479–489.

(18) Stein, A.; Keller, S. W.; Mallouk, T. E. Turning Down the Heat: Design and Mechanism in
Solid-State Synthesis. Science 1993, 259, 1558–1564.

231
(19) Martinolich, A. J.; Kurzman, J. A.; Neilson, J. R. Polymorph Selectivity of Superconducting
CuSe2 Through Kinetic Control of Solid-State Metathesis. J. Am. Chem. Soc. 2015, 137,
3827–3833.

(20) Thompson, J. O.; Anderson, M. D.; Ngai, T.; Allen, T.; Johnson, D. C. Nucleation and
Growth Kinetics of Co-Deposited Copper and Selenium Precursors to Form Metastable
Copper Selenides. J. Alloys Compd. 2011, 509, 9631–9637.

(21) Nozaki, H.; Shibata, K.; Onoda, M.; Yukino, K.; Ishii, M. Phase Transition of Copper
Selenide Studied by Powder X-Ray Diffractometry. Mater. Res. Bull. 1994, 29, 203–211.

(22) Sanchez, C.; Rozes, L.; Ribot, F.; Laberty-Robert, C.; Grosso, D.; Sassoye, C.; Boissiere, C.;
Nicole, L. “Chimie Douce”: A Land of Opportunities for the Designed Construction of
Functional Inorganic and Hybrid Organic-Inorganic Nanomaterials. Comptes Rendus Chim.
2010, 13, 3–39.

(23) Braham, E. J.; Davidson, R. D.; Al-Hashimi, M.; Arróyave, R.; Banerjee, S. Navigating the
Design Space of Inorganic Materials Synthesis Using Statistical Methods and Machine
Learning. Dalton Trans. 2020, 49, 11480-11488.

(24) Williamson, E. M.; Sun, Z.; Mora-Tamez, L.; Brutchey, R. L. Design of Experiments for
Nanocrystal Syntheses: A How-To Guide for Proper Implementation. Chem. Mater. 2022,
34, 9823–9835.

(25) Brutchey, R. L. Diorganyl Dichalcogenides as Useful Synthons for Colloidal Semiconductor
Nanocrystals. Acc. Chem. Res. 2015, 48, 2918–2926.

(26) Norako, M. E.; Greaney, M. J.; Brutchey, R. L. Synthesis and Characterization of Wurtzite-
Phase Copper Tin Selenide Nanocrystals. J. Am. Chem. Soc. 2012, 134, 23–26.

(27) Franzman, M. A.; Schlenker, C. W.; Thompson, M. E.; Brutchey, R. L. Solution-Phase
Synthesis of SnSe Nanocrystals for Use in Solar Cells. J. Am. Chem. Soc. 2010, 132, 4060–
4061.

(28) Norako, M. E.; Brutchey, R. L. Synthesis of Metastable Wurtzite CuInSe2 Nanocrystals.
Chem. Mater. 2010, 22, 1613–1615.

(29) Geisenhoff, J. Q.; Tamura, A. K.; Schimpf, A. M. Manipulation of Precursor Reactivity for
the Facile Synthesis of Heterostructured and Hollow Metal Selenide Nanocrystals. Chem.
Mater. 2020, 32, 2304–2312.

(30) Guo, Y.; Alvarado, S. R.; Barclay, J. D.; Vela, J. Shape-Programmed Nanofabrication:
Understanding the Reactivity of Dichalcogenide Precursors. ACS Nano 2013, 7, 3616–3626.

232
(31) Koziel, A. C.; Goldfarb, R. B.; Endres, E. J.; Macdonald, J. E. Molecular Decomposition
Routes of Diaryl Diselenide Precursors in Relation to the Phase Determination of Copper
Selenides. Inorg. Chem. 2022, 61, 14673–14683.

(32) Mourdikoudis, S.; Liz-Marzán, L. M. Oleylamine in Nanoparticle Synthesis. Chem. Mater.
2013, 25, 1465–1476.

(33) Uk Son, S.; Kyu Park, I.; Park, J.; Hyeon, T. Synthesis of Cu2O Coated Cu Nanoparticles and
Their Successful Applications to Ullmann-Type Amination Coupling Reactions of Aryl
Chlorides. Chem. Commun. 2004, 7, 778.

(34) Barnard, A.; Opletal, G. Selecting Machine Learning Models for Metallic Nanoparticles.
Nano Futures. 2020, 4, 035003.

(35) Artrith, N.; Butler, K. T.; Coudert, F.-X.; Han, S.; Isayev, O.; Jain, A.; Walsh, A. Best
Practices in Machine Learning for Chemistry. Nat. Chem. 2021, 13, 505–508.

(36) Wong, T.-T. Performance Evaluation of Classification Algorithms by K-Fold and Leave-
One-out Cross Validation. Pattern Recognit. 2015, 48, 2839–2846.

(37) Flores-Rojas, E.; Samaniego-Benítez, J. E.; Serrato, R.; García-García, A.; Ramírez-Bon, R.;
Ramírez-Aparicio, J. Transformation of Nanostructures Cu2O to Cu3Se2 through Different
Routes and the Effect on Photocatalytic Properties. ACS Omega 2020, 5, 20335–20342.

(38) Morimoto, N.; Koto, K. Crystal Structure of Umangite, Cu3Se2. Science 1966, 152, 345–345.

(39) Hernández-Pagán, E. A.; Robinson, E. H.; La Croix, A. D.; Macdonald, J. E. Direct Synthesis
of Novel Cu2– xSe Wurtzite Phase. Chem. Mater. 2019, 31, 4619–4624.

(40) Milman, V. Klockmannite, CuSe: Structure, Properties and Phase Stability from Ab Initio
Modeling. Acta Crystallogr. B 2002, 58, 437–447.

233
Appendix A.*

*Supplementary Information for Chapter 1. Design of Experiments for Nanocrystal Syntheses: A
How-To Guide for Proper Implementation, Published in Chem. Mater. 2022,

A.1. Coding Variable Values
An easy way to convert between coded and real values in the design matrices is by using
Excel. Eq. A.1 from the main text can be entered to solve for either the real values based on the
coded values or vice versa and can be applied to an entire reaction matrix at once.
𝑥
!
=
"
!
#"
!
"
$"
!
× 𝛼 (A.1)
where 𝑋
!
%
is the real value at the center of the experimental domain for factor i (or the value mid-
way between the lower and upper bound, coded as 0), Δ𝑋
!
is the step variation of the real value (or
the difference between the center value (0) and the value of the upper or lower bound), and a is
equal to the coded value limit for each variable, which in this case is one. Tables A.1, A.2, and
A.3 below can be used to practice these conversions. An example Excel spreadsheet and the
entered equation to turn the coded values of factor one into the corresponding real values are given
in Figure A.1.

Table A.1. Example reaction bounds for five factors.
Variable Low level High level Coded low level Coded high level
Precursor ratio 1:1 1:4 -1 +1
Metal:ligand ratio 1:2 1:16 -1 +1
Temperature 170 ˚C 320 ˚C -1 +1
Time 1 min 5 h -1 +1
Solvent volume
(concentration)
4 mL 12 mL -1 +1

234
Table A.2. Coded Doehlert optimization matrix for five factors.
Run Precursor Ratio Metal:ligand ratio Temperature Time Solvent Volume
1 0 0 0 0 0
2 1 0 0 0 0
3 0.5 0.866 0 0 0
4 0.5 0.289 0.817 0 0
5 0.5 0.289 0.204 0.791 0
6 0.5 0.289 0.204 0.158 0.775
7 −1 0 0 0 0
8 −0.5 −0.866 0 0 0
9 −0.5 −0.289 −0.817 0 0
10 −0.5 −0.289 −0.204 −0.791 0
11 −0.5 −0.866 −0.204 −0.158 −0.775
12 0.5 −0.289 0 0 0
13 0.5 −0.289 −0.817 0 0
14 0.5 −0.289 −0.204 −0.791 0
15 0.5 −0.289 −0.204 −0.158 −0.775
16 −0.5 0.866 0 0 0
17 0 0.577 −0.817 0 0
18 0 0.577 −0.204 −0.791 0
19 0 0.577 −0.204 −0.158 −0.775
20 −0.5 0.289 0.817 0 0
21 0 −0.577 0.817 0 0
22 0 0 0.613 −0.791 0
23 0 0 0.613 −0.158 −0.775
24 −0.5 0.289 0.204 0.791 0
25 0 −0.577 0.204 0.791 0
26 0 0 −0.613 0.791 0
27 0 0 0 0.633 −0.775
28 −0.5 0.289 0.204 0.158 0.775
29 0 −0.577 0.204 0.158 0.775
30 0 0 −0.613 0.158 0.775
31 0 0 0 −0.633 0.775

235
Table A.3. Real values of a Doehlert optimization matrix for five factors based on the reaction
bounds in Table A.1.
Run Precursor Ratio Metal:ligand ratio Temperature Time Solvent Volume
1 2.5 9 245 150.5 8
2 4 9 245 150.5 8
3 3.25 15.062 245 150.5 8
4 3.25 11.023 306.275 150.5 8
5 3.25 11.023 260.3 268.7545 8
6 3.25 11.023 260.3 174.121 11.1
7 1 9 245 150.5 8
8 1.75 2.938 245 150.5 8
9 1.75 6.977 183.725 150.5 8
10 1.75 6.977 229.7 32.2455 8
11 1.75 2.938 229.7 126.879 4.9
12 3.25 6.977 245 150.5 8
13 3.25 6.977 183.725 150.5 8
14 3.25 6.977 229.7 32.2455 8
15 3.25 6.977 229.7 126.879 4.9
16 1.75 15.062 245 150.5 8
17 2.5 13.039 183.725 150.5 8
18 2.5 13.039 229.7 32.2455 8
19 2.5 13.039 229.7 126.879 4.9
20 1.75 11.023 306.275 150.5 8
21 2.5 4.961 306.275 150.5 8
22 2.5 9 290.975 32.2455 8
23 2.5 9 290.975 126.879 4.9
24 1.75 11.023 260.3 268.7545 8
25 2.5 4.961 260.3 268.7545 8
26 2.5 9 199.025 268.7545 8
27 2.5 9 245 245.1335 4.9
28 3.25 11.023 260.3 174.121 11.1
29 2.5 4.961 260.3 174.121 11.1
30 2.5 9 199.025 174.121 11.1
31 2.5 9 245 55.8665 11.1

236

Figure A.1. Excel spreadsheet to convert the coded values in Table A.2 into the real values given
in Table A.3.

A.2. Screening Designs
A.2.1. Full Factorial Design
The method of ensuring orthogonality can be visualized by indicating the -1’s and +1’s by
colors in Tables A.4, A.5, and A.6. Note the similar pattern of the colors in each example.

Table A.4. Example 2-level design for three factors (2
3
factorial).
Run Factor A Factor B Factor C
1 -1 -1 -1
2 +1 -1 -1
3 -1 +1 -1
4 +1 +1 -1
5 -1 -1 +1
6 +1 -1 +1
7 -1 +1 +1
8 +1 +1 +1

237
Table A.5. Example 2-level design for four factors (2
4
).
Run Factor A Factor B Factor C Factor D
1 -1 -1 -1 -1
2 +1 -1 -1 -1
3 -1 +1 -1 -1
4 +1 +1 -1 -1
5 -1 -1 +1 -1
6 +1 -1 +1 -1
7 -1 +1 +1 -1
8 +1 +1 +1 -1
9 -1 -1 -1 +1
10 +1 -1 -1 +1
11 -1 +1 -1 +1
12 +1 +1 -1 +1
13 -1 -1 +1 +1
14 +1 -1 +1 +1
15 -1 +1 +1 +1
16 +1 +1 +1 +1

Table A.6. Example 2-level factorial design for five factors (2
5
).
Run Factor A Factor B Factor C Factor D Factor E
1 -1 -1 -1 -1 -1
2 +1 -1 -1 -1 -1
3 -1 +1 -1 -1 -1
4 +1 +1 -1 -1 -1
5 -1 -1 +1 -1 -1
6 +1 -1 +1 -1 -1
7 -1 +1 +1 -1 -1
8 +1 +1 +1 -1 -1
9 -1 -1 -1 +1 -1
10 +1 -1 -1 +1 -1
11 -1 +1 -1 +1 -1
12 +1 +1 -1 +1 -1
13 -1 -1 +1 +1 -1
14 +1 -1 +1 +1 -1
15 -1 +1 +1 +1 -1
16 +1 +1 +1 +1 -1
17 -1 -1 -1 -1 +1
18 +1 -1 -1 -1 +1
19 -1 +1 -1 -1 +1
20 +1 +1 -1 -1 +1
21 -1 -1 +1 -1 +1
22 +1 -1 +1 -1 +1
23 -1 +1 +1 -1 +1
24 +1 +1 +1 -1 +1
25 -1 -1 -1 +1 +1
26 +1 -1 -1 +1 +1
27 -1 +1 -1 +1 +1
28 +1 +1 -1 +1 +1
29 -1 -1 +1 +1 +1
30 +1 -1 +1 +1 +1
31 -1 +1 +1 +1 +1
32 +1 +1 +1 +1 +1
238

A.2.2. Blocked Designs
When denoting blocked designs in a study, an asterisk is shown next to the design
resolution to indicate that the stated resolution assumes that blocking factors do not interact with
experimental factors. This is the standard assumption when the analysis is performed.

Table A.7. A 2
3
full factorial design for three variables and their contrast coefficients.
Variable Mean A B C AB AC BC ABC Avg. Response
Run 1 + − − − + + + − 𝑦 ,
'

Run 2 + + − − − − + + 𝑦 ,
(

Run 3 + − + − − + − + 𝑦 ,
)

Run 4 + + + − + − − − 𝑦 ,
*

Run 5 + − − + + − − + 𝑦 ,
+

Run 6 + + − + − + − − 𝑦 ,
,

Run 7 + − + + − − + − 𝑦 ,
-

Run 8 + + + + + + + + 𝑦 ,
.

Divisor 8 4 4 4 4 4 4 4

Randomized Complete Block Design (RCBD) – This is the standard blocked design described in
the main text. Splitting the design matrix of the 2
3
factorial example from Tables A.4 and A.7 into
two blocks is as follows:

Splitting a full 2
3
factorial into four blocks is as follows:

239

Latin Square Design – This design allows for two nuisance factors to be blocked at the
same time with one primary factor. It gets its name from the fact that we can write it as a square
with Latin letters to correspond to the treatments. The treatment factor levels are the Latin letters
in the Latin square design. The number of rows and columns must correspond to the number of
treatment levels. So, if we have four treatments then we would need to have four rows and four
columns to create a Latin square. This gives us a design where we have each of the treatments and
in each row and in each column. The rows represent one blocking factor like batch of raw material
and the columns represent another blocking factor like the experimenter performing the reaction.
A, B, C, and D represent the treatment of the primary factor.

240

You can make any size square you want, for any number of treatments. Each treatment just
must occur only once in each row and once in each column. The model is represented by eq. A.2:
𝑦
!74
= 𝜇+𝜌
!
+𝛽
7
+𝜏
4
+𝑒
!74
(A.2)
where t is the number of treatments, the nuisance factor blocked by rows, 𝑖 = 1,...,𝑡, the
nuisance factor blocked by columns, 𝑗 = 1,...,𝑡, and the type of treatment [𝑘 = 1,...,𝑡] where
𝑘 = 𝑑(𝑖,𝑗) and the total number of observations 𝑁 = 𝑡
(
(the number of rows times the number
of columns). Note that a Latin Square is an incomplete design, which means that it does not include
observations for all possible combinations of i, j and k. Therefore, we use the notation 𝑘 = 𝑑(𝑖,𝑗).
Once we know the row and column of the design, then the treatment is specified. In other words,
if we know i and j, then k is specified by the Latin Square design. This property has an impact on
how we calculate means and sums of squares, and for this reason, we cannot use the balanced
ANOVA even though it looks perfectly balanced. We will see later that although it has the property
of orthogonality, it is not complete. An assumption made when using a Latin Square design is that
the three factors (treatments, and two nuisance factors) do not interact. If this assumption is
241
violated, the Latin Square design error term will be inflated. The randomization procedure for
assigning treatments that you would like to use when you apply a Latin Square is somewhat
restricted to preserve the structure of the Latin Square. The ideal randomization would be to select
a square from the set of all possible Latin Squares of the specified size. However, a more practical
randomization scheme would be to select a standardized Latin Square at random (these are
tabulated) and then:
1. randomly permute the columns,
2. randomly permute the rows, and
3. assign the treatments to the Latin letters in a random fashion.

A.2.3. Fractional Factorials and Design Resolution
1

Resolution – An indication of the confounding pattern of the design. Designs are classified as
having one of the following resolutions:

Resolution III: Designs that confound the estimates of the main effects with two-factor
interactions. Such designs can be safely interpreted only if all two-factor interactions are small or
non-existent.

Resolution IV: Designs that can obtain clear estimates of all main effects. However, some or all
the two-factor interactions are confounded with other two-factor interactions or block effects. The
alias structure, as described below, indicates where the confounding occurs.

242
Resolution V: Designs that can obtain clear estimates of all main effects and all two-factor
interactions. Higher-order interactions, however, are confounded with these effects. In most cases,
this is not a problem since third order and higher effects are usually assumed to be small or non-
existent. Resolution V designs are typically excellent selections.

Resolution V+: The design has resolution greater than 5, allowing for the estimation of 3-factor or
higher-order interactions if desired.

Nodal: Nodal designs are designs in which the highest number of factors are investigated in a given
number of runs for a specific resolution. This can also be thought of as the “saturated” design for
that resolution. For example, the 2IV
4-1
design in Table A.8, and the 2III
7-4
design in Table A.9 are
the nodal designs for resolution IV and resolutions III in eight runs, respectively.

Table A.8. Example 2IV
4-1
nodal fractional factorial design (½ fraction of the full factorial in Table
A.5).
contrast a b c abc
Run Factor A Factor B Factor C Factor D
1 -1 -1 -1 -1
2 +1 -1 -1 +1
3 -1 +1 -1 +1
4 +1 +1 -1 -1
5 -1 -1 +1 +1
6 +1 -1 +1 -1
7 -1 +1 +1 -1
8 +1 +1 +1 +1

To calculate the resolution of any fractional factorial design, use the following method for
the 2IV
4-1
design illustrated in Table A.8. As explained for this fractional factorial in the main text,
we set D = ABC to create a half fraction of a 4-factor factorial (exemplified in Table A.5). In total,
we obtained the half fraction by using the letters of the 2
3
factorial in Table A.7 of the main text;
243
that is, A = a, B = b, C = c, and D = abc. Thus, D = ABC is the generating relation of the design.
If you multiply the signs of the elements in any column by those of the same column, you get a
column of plus (+) signs identified by the symbol I, called the identity. Thus A×A = A
2
= I, B×B
= B
2
= I, and so forth. Multiplying both sides of the design generator ABC by D gives D×D = D
2

= ABCD, that is, I = ABCD. The design is thus a resolution IV because the generating relation
contains four letters. This expression is a somewhat more convenient form of the generating
relation, as it enables you to obtain all the alias relationships. For example, multiplying both sides
of I = ABCD by A gives A = A
2
BCD = IBCD = BCD, with the identity I acting as 1. Thus, the
estimates of A and BCD are confounded, and A and BCD are aliases. For the entire design:

I = ABCD, A = BCD, B = ACD, C = ABD, D = ABC, AB = CD, AC = BD, AD = BC

Assuming ternary interaction effects and higher are insignificant, the abbreviated alias
structure becomes AB = CD, AC = BD, and AD = BC. All main effects are unconfounded, which
defines a resolution of IV. In general, the resolution of any half fraction design equals the number
of letters in its generating relation. If you had instead used a binary interaction column for factor
D, say, D = AB, you would then have a generating relation of I = ABD and thus A = BD, B = AD,
C = ABCD, D = AB, AC = BCD, BC = ACD, and CD = ABC. Since the generating relation I =
ABD contains three letters this design is a resolution III. Three of the main effects are aliased with
2-factor interactions, and therefore this is not the most desirable alias structure.

244
Table A.9. Example 2III
7-4
nodal fractional factorial design (
1
8
# fraction of the 128 runs in the full
2
7
factorial).
contrast a b c ab ac bc abc
Run Factor A Factor B Factor C Factor D Factor E Factor F Factor G
1 -1 -1 -1 +1 +1 +1 -1
2 +1 -1 -1 -1 -1 +1 +1
3 -1 +1 -1 -1 +1 -1 +1
4 +1 +1 -1 +1 -1 -1 -1
5 -1 -1 +1 +1 -1 -1 +1
6 +1 -1 +1 -1 +1 -1 -1
7 -1 +1 +1 -1 -1 +1 -1
8 +1 +1 +1 +1 +1 +1 +1

To find the alias structure and the resolution of the design in Table A.9, the generators
would be D = AB, E = AC, F = BC, G = ABC, or equally, the defining relation would begin with
I = ABD = ACE = BFC = ABCG. Since this is not a half-fraction design, we must add all “words”
than can be constructed from all multiplications of these four generators to complete the generating
relation. For example, ABD ´ ACE = A
2
BCDE = BCDE, since A
2
= 1. The complete generating
relation is thus:

I = ABD = ACE = BCF = ABCG = BCDE = ACDF = CDG = ABEF = BEG = AFG = DEF =
ADEG = BDFG = CEFG = ABCDEFG

The smallest “word” in the generating relation has three letters, thus it is a resolution III design.
Multiplying through the defining relation by A gives the alias structure of factor A:

A = BD = CE = ABCF = BCG = ABCDE = CDF = ACDG = BEF = ABEG = FG = ADEF = DEG
= ABDFG = ACEFG = BCDEFG

245
Assuming all ternary interactions or higher are insignificant, the abbreviated alias pattern
would be A = BD = CE = FG, meaning the effects of A are combined with the effects of the
indicated binary interactions. The rest of the alias patterns are found similarly, and the entire alias
structure can be summarized in table form:

Table A.10. Alias structure of a 2III
7-4
fractional factorial design.
Contrast Factor Effect of each factor
a A A + BD + CE + FG
b B B + AD + CF + EG
c C C + AE + BF + DG
ab D D + AB + EF + CG
ac E E + AC + DF + BG
bc F F + BC + DE + AG
abc G G + CD + BE + AF

Now, try practicing the above method to generate the alias structure and determine the
design resolution for this design in Table A.11 (SOLUTION = it is a resolution III).

Table A.11. Example 2III
5-2
fractional factorial design (¼ fraction of the full factorial in Table
A.5).
contrast a b c ab abc
Run Factor A Factor B Factor C Factor D Factor E
1 -1 -1 -1 +1 -1
2 +1 -1 -1 -1 +1
3 -1 +1 -1 -1 +1
4 +1 +1 -1 +1 -1
5 -1 -1 +1 +1 +1
6 +1 -1 +1 -1 -1
7 -1 +1 +1 -1 -1
8 +1 +1 +1 +1 +1

A.2.4. Design Augmentation
As previously mentioned, further runs or sequential assembly of designs may be needed
when fractional factorial designs yield ambiguities amongst the factors. Below are several options
to achieve this.
246

Single Column Foldover – A foldover is achieved by sign-switching. If a specific factor is thought
to interact with other factors but is aliased with other effects, the signs of only that factor’s column
can be switched and this second set of experiments can be run (i.e., the positive signs of that
column switched to negative and the negative signs switched to positive). This resolves all the
main effects and all binary interaction effects of that factor. This is sometimes referred to as
“clearing a factor”.

Multi-Column Foldover – If aliased interaction effects are suspected to be significant but the
specific factors attributing to the significance are unknown, a multi-column foldover can be
performed. This is achieved by switching the signs of every column, resulting in the mirror image
of the original fraction. Looking at the example of the 2III
7-4
design from Table A.9 in the previous
section, the generating relation would be identical for the foldover design except all odd-lettered
“words” would now have a minus sign. So, adding together the generating relation of the original
2III
7-4
design with the foldover would give I = ABCG = ACDF = ABEF = ADEG = BCDE = BDFG
= CEFG. The smallest “word” now has four letters, and thus the design went from a resolution III
to a resolution IV (2IV
7-3
fractional factorial). This type of augmentation is sometimes referred to
as “clearing the main effects” of a design.

Add a Fraction – Adding an additional fraction of the runs in the full factorial can be identical to
a multi-column foldover, but the exact operation depends on the current design. The resolution
must be re-calculated to ensure the additional fraction will increase the resolution of the design.
The new runs may or may not be placed in a new block, depending upon the nature of the design.
247

Collapse Design – This removes all reference to a selected factor from the design. This can be
used to simplify the analysis output when one or more factors appear to have no impact.

Add Star Points – Available for resolution V and higher screening designs, this adds additional
runs to the design to permit the estimation of quadratic effects. This is done by creating a central
composite design with two star points for each factor. The star points for a given factor are located
at a distance ±a from the center of the experimental region along the axis for that factor. The star
points are placed in a new block, and their distance from the center of the experimental region is
determined so that the design will be orthogonally blocked.

Replicate Design – Applicable for all designs, this option adds additional copies of the runs that
have already been performed. Replicating experiments is a way to obtain a pure estimate of the
experimental error. A dialog box is displayed in most software requesting the number of replicates
to be added. Each replicate of the current experiment is made into its own block. If blocking is not
desired, the block numbers can be ignored when the data is analyzed.

A.2.5. Mixture Designs
When the experimental factors to be studied are ingredients or components of a mixture,
the response function typically depends upon the relative proportions of each component, not the
absolute amount. Since the proportions must sum to a fixed amount, usually 100%, the factors
cannot be varied independently of one another. Consequently, the designs normally used for
248
screening and optimization cannot be applied directly, nor is the rectangular coordinate space the
most desirable representation of the experimental region.
For example, consider a blending problem in which q components may be mixed in
different proportions. Let X
j
= amount of component j in the mixture, usually represented as a
proportion or percentage, although the factors may be expressed in any convenient units. The sum
of the q components is constrained to equal a fixed value T (i.e., X
1
+ X
2
+ ... + X
q
= T), where T
often equals 100%. In addition, each component is subject to lower and upper bounds (eq. A3):
𝐿
𝑗

£ 𝑋
𝑗

£𝑈
𝑗
, (A.3)
which may be as simple as L
j
= 0% and U
j
= 100%, or more restrictive. The only legitimate
experimental runs are those that satisfy all the above constraints. Several different types of designs
intended to study the effect of up to 12 components on one or more responses exist. These include:
Simplex-Lattice Designs – These designs consist of a uniformly spaced set of points on a simplex.
As will be discussed more fully below, the simplex is the natural representation of the experimental
region for a mixture experiment. The simplex-lattice design uses m+1 equally spaced values
between 0 and 1, defined by eq. A.4:
𝑥
𝑗

= 0,
'
S
,
(
S
,…,1 (A.4)
where m is the order of the model to be fit. To fit a linear model, only pure, or primary blends are
used, such as

(x1=0, x2=1, x3=0)
(x
1
=0, x
2
=1, x
3
=0)
(x
1
=0, x
2
=0, x
3
=1)

249
To fit a quadratic model, binary blends such as

(x
1
=0.5, x
2
=0.5, x
3
=0)
(x
1
=0.5, x
2
=0, x
3
=0.5)
(x
1
=0, x
2
=0.5, x
3
=0.5)

are added. A cubic model adds the following tertiary blends instead of the binary blends:

(x
1
=.667, x
2
=.333, x
3
=0)
(x
1
=.667, x
2
=0, x
3
=.333)
(x
1
=0, x
2
=.667, x
3
=.333)
(x
1
=.333, x
2
=.667, x
3
=0)
(x
1
=.333, x
2
=0, x
3
=.667)
(x
1
=0, x
2
=.333, x
3
=.667)
(x
1
=.333, x
2
=.333, x
3
=.333)

See Figure A.2 for the design of a cubic or special cubic model.

250

Figure A.2. The design for a simplex lattice cubic or special cubic design, where P = primary
blends, T = tertiary blends, C = the centroid, and the triangle represents the simplex.

Simplex-Centroid Designs – These designs consist of 2
q
- 1 runs at all primary blends, binary
blends, tertiary blends, etc. up to the design centroid given by eq. A.5:
O𝑥
'
=
'
V
,𝑥
(
=
'
V
,…,𝑥
V
=
'
V
P (A.5)
This design tends to place more points at interior locations of the simplex, rather than on the design
boundaries as does the simplex-lattice design (Figure A.3).

Figure A.3. The design for a simplex-centroid where P = primary blends, B = binary blends, C =
the centroid, and the triangle represents the simplex.

251

Extreme Vertices Designs – If restrictive lower and upper bounds exist on the components, the
design space may be constrained such that it is not a regularly shaped simplex. In such cases, the
simplex-lattice and simplex-centroid designs are not available. The only design that will be
available in such cases consists of one that places runs at each vertex of the design region,
regardless of its shape. As discussed more fully in Myers and Montgomery (1995),
2
there may be
quite a few such vertices. Most software automatically finds all vertices when this type of design
is selected. The design can then be optimized if necessary to reduce the number of design points.

A.2.6. Multi-Factor Categorical Designs
Multi-factor categorical designs are experimental designs for situations where the primary
interest centers on comparing levels of two or more categorical factors. The procedure will create
a multilevel factorial design with runs at each combination of the levels of the factors. See Tables
A.12 and A.13 for an example of a 3-factor categorical design. These designs are analyzed via
multifactor ANOVA procedure. Graphical ANOVAs are a good way to visualize these results.

Table A.12. Example table of a multi-factor categorical experiment.
Category Name No. of levels Labels
Solvent used 3 Solvent 1, solvent 2, solvent 3
Air-free synthetic conditions 2 Yes, no
Precursor used 2 Precursor 1, precursor 2

Levels of 3´2´2 = 12 total combinations replicated once = 24 experiments.

252
Table A.13. Example table of the multi-factor categorical design matrix.
run Solvent Air-free Precursor response
1 1 Yes 1
2 1 Yes 1
3 2 Yes 1
4 2 Yes 1
5 3 Yes 1
6 3 Yes 1
7 1 No 1
8 1 No 1
9 2 No 1
10 2 No 1
11 3 No 1
12 3 No 1
13 1 Yes 2
14 1 Yes 2
15 2 Yes 2
16 2 Yes 2
17 3 Yes 2
18 3 Yes 2
19 1 No 2
20 1 No 2
21 2 No 2
22 2 No 2
23 3 No 2
24 3 No 2

A.2.7. Continuous and Categorical Factors
The construction and analysis of designs that include both quantitative and categorical
factors occur via:
1. Creating a multilevel factorial design involving all combinations of selected levels of each
factor.
2. Reducing the number of runs if desired using the D-efficiency criterion.
3. Analyzing the results using a general linear model.
As an example, consider the following 3-factor experiment in Table A.14:

253
Table A.14. Example study of mixed continuous and categorical factors.
Variable Low level High level Coded low level Coded high level Response Y: Yield (%)
Factor A: Temperature 170 ˚C 320 ˚C -1 +1
Factor B: Solvent
volume (concentration) 4 mL 12 mL -1 +1
Factor C: Precursor type A, B or C -1 0 +1

Had there been only two levels of factor C, that factor could have been handled as a
quantitative factor via a single indicator variable taking the value (-1) for one type of precursor
and (+1) for the other, making it a 2
3
factorial screening problem. However, the three levels of
precursor make it more appropriate to handle that factor as a true categorical variable. Two general
classes of designs are offered:

(1) Multilevel Factorial – Designs involving different numbers of levels for each experimental
factor. For this example, this would mean a 2
2
full factorial matrix repeated three times for each
category of precursor, or 12 runs plus three replicates for degrees of freedom (Table A.15).

Table A.15. Example 3-factor factorial design with a categorical factor.
run Precursor Temperature (˚C) Solvent Volume (mL) Yield (%)
1 A -1 -1
2 A -1 +1
3 A +1 -1
4 A +1 -1
5 B -1 -1
6 B -1 +1
7 B +1 -1
8 B +1 -1
9 C -1 -1
10 C -1 +1
11 C +1 -1
12 C +1 -1
13 A -1 -1
14 B -1 +1
15 C +1 -1

254
(2) Computer Generated – If this type of design is selected, the computer will select a set of runs
that are optimal for the model to be fit.
For the current experiment, a 3´3´3 factorial design with 27 runs has been selected by the
computer. This design leaves 15 degrees of freedom available for estimating the experimental
error. If each run was very expensive, a smaller design might be desired. Optimizing the design in
the following ways could achieve this:

Optimize – Different optimizing criteria can be used to select the experimental runs:
(1) D-efficiency measures the information generated by the experiment about the model
parameters. (2) A-efficiency measures the average variance of the estimates of the model
parameters. (3) G-efficiency measures the maximum variance of the predicted response at the
design points.

Model Coefficients – The number of coefficients in the model to be estimated. This is the minimum
number of runs that must be selected.

Number of Runs Desired – The number of experimental runs to be selected. It is usually a good
idea to select at least three more runs than there are coefficients in the selected model.

Select Runs using Forward Algorithm – Begins with the runs that have already been performed (if
any) and adds runs one at a time, adding at each step the run that adds the most to the efficiency
of the experiment. This algorithm is only available if the number of runs that has already been
performed is greater than or equal to the number of model coefficients.
255

Select Runs using Backward Algorithm – Begins with all the candidate runs and removes runs one
at a time, removing at each step the run that adds the least to the efficiency of the experiment.

Apply Exchange Algorithm at End – Once the desired number of runs has been selected, an
exchange algorithm can be performed. This algorithm tests all pairs of runs consisting of one that
has been selected and one that has not, making any exchanges that would increase the efficiency
of the experiment. Exchanges continue until no further improvements can be made by switching
one run that has been selected with one run that has not been selected. For the example, the program
was asked to find 17 runs that maximize the D-efficiency of the design using backward selection
with the exchange algorithm. When the algorithm is complete, the selected rows are shown in
Table A.16:

Table A.16. Optimized design matrix for a 3-factor study with categorical and continuous factors.
run Temperature (˚C) Solvent Volume (mL) Precursor Yield (%)
1 170 4 A
2 170 4 B
3 170 4 C
4 170 8 C
5 170 12 A
6 170 12 B
7 170 12 C
8 245 4 C
9 245 8 B
10 245 12 C
11 320 4 A
12 320 4 B
13 320 4 C
14 320 8 C
15 320 12 A
16 320 12 B

256
For each precursor, runs are performed at the four combinations of low and high
temperature and low and high concentration. A run is also performed with precursor B at a middle
level of the quantitative factors. For precursor C, four star points are added to estimate the quadratic
effects of temperature and concentration. In this type of design, the coefficients in the model output
by the regression equation are based on a standardized model in which the quantitative factors are
coded as -1and +1 (as usual) and the categorical factors at k levels, k – 1 indicator variables are
created according to:
X
1
= -1 for level 1, 1 for level 2, and 0 for all other levels
X
2
= -1 for level 1, 1 for level 3, and 0 for all other levels
...
X
k–1
= -1 for level 1, 1 for level k, and 0 for all other levels.

A.2.8. Analyzing a Screening Output
The following analysis is based on the output of a 2III
5-2
screening design using Statgraphics
Centurion software.
3
This data corresponds to the study in ref. 4, with the design based on Table
A.17. The response value Y corresponds to nanocrystal size in nm (Figure 1.5).

Table A.17. Real and coded values of the reaction parameters for the factors under investigation
in the screening and optimization designs. Reproduced from Chapter 2. Copyright 2021 American
Chemical Society.
Variable Low level High level Coded low value Coded high value
A = Co:Ni precursor ratio 1:1 1:3 -1 +1
B = Co:DDT ratio 1:2 1:16 -1 +1
C = Temperature of hot
injection
170 ˚C 190 ˚C -1 +1
D = Reaction time 1 h 5 h -1 +1
E = Volume of oleylamine 4 mL 10 mL -1 +1

257
Table A.18. Correlation matrix for a 2III
5-2
fractional factorial design.
Average A B C D E AB AD BC BD BE CD DE
Average 1.000 0.041 0.036 0.108 -0.120 0.041 -0.031 -0.031 -0.120 -0.036 0.113 0.036 0.113
A 0.041 1.000 -0.036 0.041 -0.031 0.108 0.031 -0.120 0.120 0.036 -0.113 0.113 0.036
B 0.036 -0.036 1.000 -0.113 -0.049 0.113 0.024 0.049 0.128 -0.119 0.054 -0.029 0.029
C 0.108 0.041 -0.113 1.000 0.031 0.041 0.120 0.120 0.031 -0.036 -0.036 -0.113 -0.036
D -0.120 -0.031 -0.049 0.031 1.000 0.120 0.041 0.067 -0.041 0.048 0.049 0.128 0.024
E 0.041 0.108 0.113 0.041 0.120 1.000 -0.120 0.031 -0.031 0.036 0.036 -0.036 -0.113
AB -0.031 0.031 0.024 0.120 0.0412 -0.120 1.000 -0.041 0.067 -0.048 0.128 0.049 -0.049
AD -0.031 -0.120 0.049 0.120 0.067 0.031 -0.041 1.000 0.041 -0.048 -0.049 0.024 0.128
BC -0.120 0.120 0.128 0.031 -0.041 -0.031 0.067 0.041 1.000 0.048 0.024 -0.049 0.049
BD -0.036 0.036 -0.119 -0.036 0.048 0.036 -0.048 -0.048 0.048 1.000 0.119 -0.119 0.119
BE 0.113 -0.113 0.054 -0.036 0.049 0.036 0.128 -0.049 0.024 0.119 1.000 0.029 -0.029
CD 0.036 0.113 -0.029 -0.113 0.128 -0.036 0.049 0.024 -0.049 -0.119 0.029 1.000 0.054
DE 0.113 0.036 0.029 -0.036 0.024 -0.113 -0.049 0.128 0.049 0.119 -0.029 0.054 1.000

As mentioned in the main text, the correlation matrix in Table A.18 shows the extent of
the confounding amongst the effects. A perfectly orthogonal design will show a diagonal matrix
with 1s on the diagonal and 0s on the off diagonal. Any non-zero terms off the diagonal imply that
the estimates of the effects corresponding to that row and column will be correlated. In this case,
there are 66 pairs of effects with non-zero correlations. However, since none are greater than or
equal to 0.5, the results are interpretable without much difficulty.

Table A.19. Estimated effects for a single response Y (nanocrystal size).
Effect Estimate Standard Error
a
V.I.F.
Average 6.57045 0.216492
A 3.06035 0.432984 1.10045
B -2.01159 0.430373 1.08722
C -0.270353 0.432984 1.08045
D 0.571929 0.427799 1.05472
E 0.239103 0.432984 1.10045
AB -1.86714 0.427799 1.07425
AD 0.324321 0.427799 1.07425
BC -0.407863 0.427799 1.05472
BD -0.867624 0.430747 1.08911
BE -0.323408 0.430373 1.06745
CD -0.0113841 0.430373 1.08722
DE -0.604866 0.430373 1.06745
a
Standard errors are based on total error with 8 d.f.

258
After the screening is performed, Table A.19 shows each of the estimated effects and
interactions based on the linear regression equation shown in eq. A.6:
𝑦 = 𝛽
1
∑ 𝛽
𝔦
𝓍
𝔦
+𝜀
4
𝔦0'
(A.6)
Also shown is the standard error of each of the effects, which measures their sampling error. Note
also that the largest variance inflation factor (V.I.F.) equals 1.10045. For a perfectly orthogonal
design, all the factors would equal 1. Factors of 10 or larger are usually interpreted as indicating
serious confounding amongst the effects. A Pareto chart as described in the main text plots these
estimates in decreasing order of importance.

Table A.20. Analysis of variance for a given response Y (nanocrystal size).
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A 44.581 1 44.581 49.96 0.0001
B 19.496 1 19.496 21.85 0.0016
C 0.348 1 0.3479 0.39 0.550
D 1.595 1 1.595 1.79 0.218
E 0.272 1 0.272 0.30 0.596
AB 16.999 1 16.999 19.05 0.0024
AD 0.513 1 0.513 0.57 0.470
BC 0.811 1 0.811 0.91 0.368
BD 3.620 1 3.620 4.06 0.079
BE 0.504 1 0.504 0.56 0.474
CD 0.000624 1 0.000624 0.00 0.980
DE 1.763 1 1.763 1.98 0.198
Total error 7.139 8 0.892
Total (corr.) 108.096 20
R-squared = 93.396%
R-squared (adjusted for d.f.) = 83.489%
Standard Error of Est. = 0.945
Mean absolute error = 0.455
Durbin-Watson statistic = 1.0428 (P = 0.0707)
Lag 1 residual autocorrelation = 0.462

The ANOVA table (Table A.20) is the standard output for most software. It partitions the
variability in nanocrystal size into separate pieces for each of the effects. It then tests the statistical
significance of each effect by comparing the mean square against an estimate of the experimental
error. In this case, three effects (A, B, and AB) have p-values less than 0.05, indicating that they
259
are significantly different from zero at the 95.0% confidence level. These variables will
consequentially move on to the optimization (along with factor C because of its significance for
another response in the study, not mentioned here). As mentioned in the main text, calculating the
t-statistic rather than the p-values sometimes makes more sense for 2-level factorial designs.
The R-squared statistic indicates that the linear regression model as fitted explains
93.3957% of the variability in nanocrystal size. The adjusted R-squared statistic, which is more
suitable for comparing models with different numbers of independent variables, is 83.4892%. The
standard error of the estimate shows the standard deviation of the residuals to be 0.944655. The
mean absolute error (MAE) of 0.455044 is the average value of the residuals. The Durbin-Watson
(DW) statistic tests the residuals to determine if there is any significant correlation based on the
order in which they occur in the data file. Since the p-value is greater than 5.0%, there is no
indication of serial autocorrelation in the residuals at the 5.0% significance level.

A.3. Optimization Designs
A.3.1. Example Optimization Designs
For the designs exemplified in Tables A.21-A.25:
Rotatable – Rotatability is achieved by setting the axial distance at
𝛼 = √𝐹
+
(A.7)
where F is the number of runs in the factorial portion of the design (not counting any center points).
Any number of center points may be added (3-5 center points are usually desirable), which may
be placed in a separate block if desired. For a 2
3
factorial, the star points are placed at a =
1.68179.

260
Orthogonal – Places the star points at an axial distance that ensures that all second order terms are
orthogonal to one another. The proper axial distance depends on the number of center points n
c
added to the design:

a =

(√2𝑘+𝑛
𝑐
+𝐹- √𝐹)
2
𝐹/4
4
(A.8)
For a 2
3
factorial with two center points in the axial block, the star points are placed at a =
1.28719.

Rotatable and Orthogonal – For a 2
3
factorial, the star points are placed at a = 1.68179 and nine
center points are added to the design.

Face Centered – Places the star points at the low and high levels of the factorial design (i.e., sets
a = 1.0). Although less desirable from a statistical perspective, such a design may be easier to run
since it involves only low, medium, and high levels of each factor. For such designs, 1-2 center
points are usually sufficient.

261
Table A.21. Central composite design for three factors.
Run Factor A Factor B Factor C
1 -1 -1 -1
2 +1 -1 -1
3 -1 +1 -1
4 +1 +1 -1
5 -1 -1 +1
6 +1 -1 +1
7 -1 +1 +1
8 +1 +1 +1
9 -a 0 0
10 a 0 0
11 0 -a 0
12 0 a 0
13 0 0 -a
14 0 0 a

Table A.22. Three-level factorial design for three factors.
run Factor A Factor B Factor C
1 -1 -1 -1
2 0 -1 -1
3 1 -1 -1
4 -1 0 -1
5 0 0 -1
6 1 0 -1
7 -1 1 -1
8 0 1 -1
9 1 1 0
10 -1 -1 0
11 0 -1 0
12 1 -1 0
13 -1 0 0
14 0 0 0
15 1 0 0
16 -1 1 0
17 0 1 0
18 1 1 0
19 -1 -1 1
20 0 -1 1
21 1 -1 1
22 -1 0 1
23 0 0 1
24 1 0 1
25 -1 1 1
26 0 1 1
27 1 1 1

262
Table A.23. Box-Behnken design for three factors.
run Factor A Factor B Factor C
1 -1 -1 0
2 1 -1 0
3 -1 1 0
4 1 1 0
5 -1 0 -1
6 1 0 -1
7 -1 0 1
8 1 0 1
9 0 -1 -1
10 0 1 -1
11 0 -1 1
12 0 1 1

Table A.24. Doehlert matrix for three factors.
run Factor A Factor B Factor C
1 0 0 0
2 1 0 0
3 0.5 0.866 0
4 0.5 0.289 0.817
5 -1 0 0
6 -0.5 -0.866 0
7 -0.5 -0.289 -0.0817
8 0.5 -0.866 0
9 0.5 -0.289 -0.817
10 -0.5 0.866 0
11 0 0.577 -0.817
12 -0.5 0.289 0.817
13 0 -0.577 0.817

Table A.25. Draper-Lin design for four factors.
run Factor A Factor B Factor C Factor D
1 1 1 1 -1
2 1 1 -1 -1
3 1 -1 1 1
4 -1 1 -1 1
5 1 -1 -1 1
6 -1 -1 1 -1
7 -1 1 1 1
8 -1 -1 -1 -1
9 -a 0 0 0
10 a 0 0 0
11 0 -a 0 0
12 0 a 0 0
13 0 0 -a 0
14 0 0 a 0
15 0 0 0 -a
16 0 0 0 a

263
The designs in Tables A.21-A.24 are described in chapter 1. The first eight runs in Table
A.25 (Draper-Lin design) comprise a half-fraction of a 2
4
factorial, which is resolution IV. The
second set of eight runs are the star points, where a is the axial distance as defined for central
composite designs. Several center points would also be added. The resulting design can estimate
the full second-order model, although some correlation will exist among the estimated model
coefficients. For cases in which the cost of each run is very large, however, the saving of over runs
over the full central composite design can be significant.

A.3.2. Analyzing an Optimization Output
The following example is based on a Doehlert optimization for three factors A, B, and C
on a response value, Y. The coded values for the experiments in this optimization are shown in
Table A.24 and correspond to the real-life values described in reference 4, where A, B, and C are
the same as the A, B, and C defined in Table A.17 and Y is nanocrystal size. The three factors
were chosen because they were the only three significant variables after assessing the output of the
screening design.

Table A.26. Correlation matrix for estimated effects.
Average A B C AA AB AC BB BC CC
Average 1.0000 0.0000 0.0000 0.0000 -0.5774 0.0000 0.0000 -0.5774 0.0003 -0.6295
A 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
B 0.0000 0.0000 1.0000 0.0721 0.0000 0.0000 0.0000 0.0000 -0.1017 0.0533
C 0.0000 0.0000 0.0721 1.0000 0.0000 0.0000 0.0000 0.0000 0.1476 -0.0773
AA -0.5774 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.1111 -0.2317 0.2424
AB 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 -0.3166 0.0000 0.0000 0.0000
AC 0.0000 0.0000 0.0000 0.0000 0.0000 -0.3166 1.0000 0.0000 0.0000 0.0000
BB -0.5774 0.0000 0.0000 0.0000 0.1111 0.0000 0.0000 1.0000 0.2310 0.2422
BC 0.0003 0.0000 -0.1017 0.1476 -0.2317 0.0000 0.0000 0.2310 1.0000 0.1075
CC -0.6295 0.0000 0.0533 -0.0773 0.2424 0.0000 0.0000 0.2422 0.1075 1.0000

As described previously, any non-zero terms off the diagonal in Table A.26 imply that the
estimates of the effects corresponding to that row and column will be correlated. In this case, there
264
are 12 pairs of effects with non-zero correlations. However, since none are greater than or equal
to 0.5, the results can be interpreted without much difficulty.

Table A.27. Estimated effects for a single response Y (nanocrystal size).
Effect Estimate Standard Error
a
V.I.F.
average 26.3 2.28314
A -3.725 4.56629 1.0
B 4.05 3.85922 1.02699
C 5.24844 3.54627 1.04662
AA -5.5 7.90904 1.16912
AB 12.5 9.13258 1.11137
AC -16.1715 9.62769 1.11137
BB -7.175 5.93178 1.12957
BC -11.5611 7.58765 1.20892
CC -2.76777 4.83584 1.15453
a
Standard errors are based on total error with 7 d.f.

Table A.27 shows each of the estimated effects and interactions calculated as described in
the main text. Also shown is the standard error of each of the effects, which measures their
sampling error. Note also that the largest variance inflation factor (V.I.F.) equals 1.21. For a
perfectly orthogonal design, all the factors would equal 1. Factors of 10 or larger are usually
interpreted as indicating serious confounding amongst the effects. The Pareto chart shown in
Figure 1.5 in chapter 1 plots the estimates in decreasing order of importance.

Table A.28. Analysis of variance for a single response, Y (nanocrystal size).
Source Sum of Squares D.f. Mean Square F-Ratio P-Value
A 13.8756 1 13.8756 0.67 0.4415
B 22.9635 1 22.9635 1.10 0.3289
C 45.6714 1 45.6714 2.19 0.1824
AA 10.0833 1 10.0833 0.48 0.5092
AB 39.0625 1 39.0625 1.87 0.2134
AC 58.8277 1 58.8277 2.82 0.1369
BB 30.507 1 30.507 1.46 0.2657
BC 48.4076 1 48.4076 2.32 0.1714
CC 6.83035 1 6.83035 0.33 0.5850
Total error 145.957 7 20.851
Total (corr.) 392.419 16
R-squared = 62.8058 percent
R-squared (adjusted for d.f.) = 14.9848 percent
Standard Error of Est. = 4.56629
Mean absolute error = 2.47059
Durbin-Watson statistic = 2.50456 (P=0.7486)
Lag 1 residual autocorrelation = -0.29373

265
The ANOVA table (Table A.28) partitions the variability in the response into separate
pieces for each of the effects. It then tests the statistical significance of each effect by comparing
the mean square against an estimate of the experimental error. In this case, zero effects have p-
values less than 0.05, indicating that they are significantly different from zero at the 95.0%
confidence level.
The R-squared statistic indicates that the model as fitted explains 62.8058% of the
variability in nanocrystal size. The adjusted R-squared statistic, which is more suitable for
comparing models with different numbers of independent variables, is 14.9848%. The standard
error of the estimate shows the standard deviation of the residuals to be 4.56629. The mean
absolute error (MAE) of 2.47059 is the average value of the residuals. The Durbin-Watson (DW)
statistic tests the residuals to determine if there is any significant correlation based on the order in
which they occur in the data file. Since the p-value is greater than 5.0%, there is no indication of
serial autocorrelation in the residuals at the 5.0% significance level.

The regression equation fit to model the data in this example corresponds to eq. A.9:

𝑅𝑒𝑠𝑝𝑜𝑛𝑠𝑒 𝑣𝑎𝑙𝑢𝑒 = 26.3 − 1.8625×𝐹𝑎𝑐𝑡𝑜𝑟 𝐴 + 2.33834×𝐹𝑎𝑐𝑡𝑜𝑟 𝐵+ 3.21202×𝐹𝑎𝑐𝑡𝑜𝑟 𝐶 −
2.75×𝐴
"
+ 7.21709×𝐴𝐵− 9.89687×𝐴𝐶 − 4.78361×𝐵
"
− 8.17017×𝐵𝐶− 2.07327×𝐶
"

(A.9)

266
Table A.29. Regression coefficients for nanocrystal size (Y).
Coefficient Estimate
constant 26.3
A -1.8625
B 2.33834
C 3.21202
AA -2.75
AB 7.21709
AC -9.89687
BB -4.78361
BC -8.17017
CC -2.07327

The regression coefficients for each effect, creating the model in eq 8, are shown in Table
A.29.

Table A.30. Estimation results for nanocrystal size (Y).
Observed Fitted Lower 95.0% CL Upper 95.0% CL
Row Value Value for Mean for Mean
1 27.4 26.3 20.9012 31.6988
2 19.5 21.6875 12.3365 31.0385
3 31.4 26.2437 16.9486 35.5389
4 18.3 21.2688 11.9736 30.5639
5 27.6 25.4125 16.0615 34.7635
6 18.9 24.0562 14.7611 33.3514
7 19.5 16.5313 7.2361 25.8264
8 16.0 15.9438 6.6486 25.2389
9 22.8 20.6688 11.3736 29.9639
10 21.8 21.8562 12.5611 31.1514
11 20.8 25.9 16.7744 35.0256
12 27.0 29.1313 19.8361 38.4264
13 30.8 28.45 21.3813 35.5187
14 31.2 28.45 21.3813 35.5187
15 28.6 26.3 20.9012 31.6988
16 26.2 26.3 20.9012 31.6988
17 23.0 26.3 20.9012 31.6988

Table A.30 contains information about values of the nanocrystal size (response value, Y)
generated using the fitted model in eq 8. The table includes:
(1) the observed value of the response value Y (if any)
(2) the predicted value of the response value Y using the fitted model
(3) 95.0% confidence limits for the mean response.
267
Each item corresponds to the values of the experimental factors in a specific row of your
data file (i.e., an experimental run). To generate forecasts for additional combinations of the
factors, add additional runs to the bottom of your data file. In each new row, enter values for the
experimental factors but leave the cell for the response empty. When you return to this pane,
forecasts will be added to the table for the new rows, but the model will be unaffected.
Goal: maximize nanocrystal size (Y)

Optimum value = 41.1084

Table A.31. Predicted optimum of nanocrystal size (response value Y).
Factor Low High Optimum
A -1.0 1.0 -1.0
B -0.866 0.866 -0.866
C -0.817 0.817 0.81413

Table A.31 shows the combination of factor levels which maximizes the nanocrystal size
(response value Y) over the indicated region set by the optimization design. These can be turned
back into their real values to validate the prediction. The optimum value of Y is also predicted by
the model, and in this case is 41.1084 (a.u.).

A.3.3. Multi-Response Optimization
The following example of a multi-response optimization is based on the effects of three
different factors A, B, and C on three different responses X, Y, and Z, as given in Table A.32. The
real values correspond to Table A.17 and ref. 4.

Table A.32. Multi-response optimization design.
Observed Observed Specified
Response Minimum Maximum Goal Impact Sensitivity Low
Response X 5.04 13.89 Minimize 3.0 Medium 5.04
Response Y 16.0 31.4 Minimize 3.0 Medium 13.4507
Response Z 42.0 100.0 Maximize 3.0 Medium 0.0
268

Table A.33. ANOVA of the multi-response model equation.
Source D.F.
Model 9
Total Error 7
Lack-of-fit 3
Pure error 4
Total (corr.) 16

Table A.33 shows the degrees of freedom that will be available for estimating experimental
error. Two estimates are commonly used – total error, which includes degrees of freedom that
could have been used to estimate effects that are not in the current model, and pure error which
comes only from replicated runs. In this case, the total error has seven degrees of freedom, while
there are four degrees of freedom for pure error. In general, at least three or four error degrees of
freedom need to be available when testing the statistical significance of estimated effects.
Otherwise, the statistical tests will have very little power.

Table A.34. Model coefficients.
Power at Power at Power at
Coefficient Standard Error VIF Ri-Squared SN = 0.5 SN = 1.0 SN = 2.0
constant 0.5 7.20% 13.99% 40.80%
A 0.5 1.0 0.0 7.20% 13.99% 40.80%
B 0.422577 1.02699 0.026282 8.09% 17.68% 53.15%
C 0.38831 1.04662 0.0445451 8.67% 20.06% 60.10%
AA 0.866025 1.16912 0.144654 5.73% 7.94% 17.06%
AB 1.0 1.11137 0.100208 5.54% 7.20% 13.99%
AC 1.05421 1.11137 0.100208 5.49% 6.97% 13.07%
BB 0.649519 1.12957 0.114706 6.30% 10.27% 26.59%
BC 0.830833 1.20892 0.172813 5.79% 8.20% 18.13%
CC 0.529516 1.15453 0.133849 6.96% 13.00% 37.16%
a
a = 5.0%, sigma estimated from total error with 7 d.f.

Table A.34 shows the standard errors of the coefficients in the model to be fit, as a multiple
of the experimental error s. The smaller the standard error, the more precise the estimates of the
269
coefficients will be. Also included are variance inflation factors (VIF), Ri-squared, and the power
of the design for each effect as described in the main text.

Table A.35. Alias matrix.
Effect AAA AAB AAC ABB ABC ACC BBB BBC BCC CCC
constant
A 0.6250 0.2778 0.0834 0.2500
B 0.2024 0.0714 0.8094 -0.0950 0.2857
C 0.0635 0.1547 -0.0424 0.2381 -0.0473 1.0000
AA
AB
AC
BB
BC -0.0238 0.0357 -0.0953 -0.0475 0.1429
CC 0.0079 -0.0119 0.0318 0.0159 -0.0477

The alias matrix in Table A.35 indicates the extent to which the effects to be estimated are
confounded with effects that are not in the current model. Each row represents an effect to be
estimated. A non-zero value in any cell indicates that the effect in the corresponding column,
multiplied by the value in the cell, is added to the estimated effect when the model is fit.

Table A.36. Leverage of each run in the multi-response optimization design.
Run Leverage Location
1 0.25 Center
2 0.75 Face-center
3 0.741071 Other
4 0.741071 Other
5 0.75 Face-center
6 0.741071 Other
7 0.741071 Other
8 0.741071 Other
9 0.741071 Other
10 0.741071 Other
11 0.714286 Other
12 0.741071 Other
13 0.428571 Other
14 0.428571 Other
15 0.25 Center
16 0.25 Center
17 0.25 Center

270
Table A.36 displays the leverage of each run in the designed experiment. Leverage is a
statistic which measures how influential each run is in determining the coefficients of the estimated
model. In this case, an average run would have a leverage value equal to 0.588235. Runs with high
leverage will have an unusually large impact on the fitted model compared to other runs. There
are no runs with more than three times the average leverage.

Table A.37. The observed and predicted response values of all three responses based on the
quadratic models from their individual optimizations.
Observed Predicted Observed Predicted Observed Predicted
Run Response X Response X Response Y Response Y Response Z Response Z
1 6.44 6.31 27.4 23.8886 75.0 72.5
2 5.86 6.01375 19.5 22.0261 66.0 68.25
3 7.94 8.01437 31.4 28.1254 100.0 94.6964
4 5.95 5.72188 18.3 21.5029 79.0 82.0536
5 9.9 9.74625 27.6 25.7511 76.0 73.75
6 5.96 5.88562 18.9 25.9018 78.0 83.3036
7 7.11 7.33813 19.5 16.8015 58.0 54.9464
8 5.04 6.53437 16.0 17.7893 78.0 67.5536
9 7.36 5.71188 22.8 20.939 42.0 50.1964
10 13.89 12.3956 21.8 23.7379 74.0 84.4464
11 8.41 9.83 20.8 26.1165 80.0 74.8571
12 6.18 7.82813 27.0 29.3654 91.0 82.8036
13 6.93 6.085 30.8 28.5944 100.0 90.5714
14 6.66 6.085 31.2 28.5944 76.0 90.5714
15 6.18 6.31 28.6 23.8886 85.0 72.5
16 6.47 6.31 26.2 23.8886 70.0 72.5
17 6.15 6.31 23.0 23.8886 60.0 72.5

271
Table A.38. The observed desirability and the corresponding predicted desirability from the
desirability model for the multi-response optimization.
run Observed Predicted
1 Desirability Desirability
2 0.725535 0.763054
3 0.795563 0.774701
4 0.70359 0.737838
5 0.855259 0.836272
6 0.647118 0.665634
7 0.845178 0.783042
8 0.735561 0.741172
9 0.895709 0.803657
10 0.626967 0.715544
11 0.566962 0.63516
12 0.772579 0.662688
13 0.785056 0.692841
14 0.735693 0.763051
15 0.670609 0.763051
16 0.745047 0.763054
17 0.72348 0.763054
18 0.729162 0.763054

Table A.38 displays the calculated desirability of the responses at each run in the
experiment. Based on the observed responses, the most desirable results were obtained for run
nine. Based on the predicted responses from the fitted model, the most desirable results correspond
to run five. One way to visualize this is via combinations of contour plots, exemplified in Figure
1.6 of Chapter 1.

A.4. References
(1) Box, G. E. P.; Hunter, J. S.; Hunter, W. G. Statistics for Experimenters. Design, Innovation,
and Discovery, 2nd ed.; Wiley-Interscience, 2005.

(2) Myers, R. H.; Montgomery, D. C.; Anderson-Cook, C. Response Surface Methodology:
Process and Product Optimization Using Designed Experiments, Fourth edition.; Wiley
series in probability and statistics; Wiley: Hoboken, New Jersey, 2016.

(3) Statgraphics Centurion, version 19; Statgraphics Technologies, Inc.: The Plains, VA, 2020.

272
(4) Williamson, E. M.; Tappan, B. A.; Mora-Tamez, L.; Barim, G.; Brutchey, R. L. Statistical
Multiobjective Optimization of Thiospinel CoNi2S4 Nanocrystal Synthesis via Design of
Experiments. ACS Nano 2021, 15, 9422–9433.

Abstract (if available)

Abstract The search for new materials is driven by the need to improve existing technologies in addition to the discovery of novel functionality to realize next-generation technologies. Engineered colloidal nanocrystals are of interest because of their unique size- and shape- dependent chemical and physical properties, which offer routes to the development of champion materials for applications including catalysis, plasmonics, photovoltaics, optoelectronics, and thermoelectrics. Control over colloidal nanocrystal syntheses is essential for materials discovery and the optimization of desired properties, and therefore plays a key role in the applications of these materials. Despite this, a significant bottleneck exists in the Edisonian nature of materials synthesis. Traditional one-variable-at-a-time methods for synthetic optimizations are inefficient, providing one-dimensional insight into complex, multidimensional experimental domains. This wastes precious resources in the process. Moreover, applying these lessons learned from one system to another is exceptionally challenging because of the disparate nature of material structures and compositions. This dissertation will address the inefficiencies in colloidal nanocrystal synthesis and development by utilizing data-driven learning. These multivariate techniques provide improved and robust predictive frameworks that can map reaction coordinates from precursors to the final crystalline solid in a minimal number of experiments. By constructing a fuller picture of nanocrystal syntheses, precise control over process-structure-property relationships can be obtained, better facilitating material discovery and optimization.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Expanding the library of surface ligands for semiconductor nanocrystal synthesis and photovoltaic applications

PDF

The chemistry of polymorphism in semiconductor nanocrystals

PDF

Synthesis and characterization of metal chalcogenide semiconductor nanocrystals using dialkyl dichalcogenide precursors

PDF

Solution phase synthesis routes to functional nanomaterials for energy storage

PDF

Nanocrystal surface engineering as a route towards improved photovoltaics

PDF

Semiconductor inks for solution processed electronic thin‐films

PDF

Synthesis and surface modification of perovskite-based nanocrystals for use in high energy density nanocomposites

PDF

Applications of continuous flow reactors towards screening catalytically active nanoparticle syntheses

PDF

Expanding sputtering synthesis domains for the discovery of novel film microstructures and morphologies

PDF

Artificial Decision Intelligence: integrating deep learning and combinatorial optimization

PDF

Data-driven image analysis, modeling, synthesis and anomaly localization techniques

PDF

Revisiting the azide-alkyne chemistry: scouting uncharted reactivity for synthesis of new molecular frameworks

PDF

Low temperature synthesis of functional metal oxide nanocrystals using a vapor diffusion sol-gel method

PDF

Model-based approaches to objective inference during steady-state and adaptive locomotor control

PDF

Integer optimization for analytics in high stakes domain

PDF

Quantifying the surface chemistry of semiconductor nanocrystals spanning covalent to ionic materials

PDF

Synthesis of stereoregular non-conjugated pendant electroactive polymers

PDF

Sustainable methods for manufacturing catalytically active nanoparticles

PDF

Towards trustworthy and data-driven social interventions

PDF

Development of organic-inorganic optical microcavities for studying polymer thin films

Asset Metadata

Creator Williamson, Emily Mae (author)

Core Title The implementation of data-driven techniques for the synthesis and optimization of colloidal inorganic nanocrystals

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Chemistry

Degree Conferral Date 2023-08

Publication Date 07/17/2023

Defense Date 05/22/2023

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag catalysis,colloidal nanocrystals,computer vision,data analytics,data science,data-driven learning,deep learning,design of experiments,fluid dynamics,machine learning,material discovery,materials chemistry,Materials Engineering,materials optimization,materials synthesis,mixed variables,multivariate analysis,nano materials,nano science,nanocrystal synthesis,nanocrystals,OAI-PMH Harvest,optimization,precursors,process-structure-property relationships,response surface methodology,statistical analysis,synthetic optimization

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Brutchey, Richard (committee chair), Feakins, Sarah (committee member), Pratt, Matthew (committee member)

Creator Email emwilliamson2214@aol.com,will330@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113281409

Unique identifier UC113281409

Identifier etd-Williamson-12099.pdf (filename)

Legacy Identifier etd-Williamson-12099

Document Type Dissertation

Format theses (aat)

Rights Williamson, Emily Mae

Internet Media Type application/pdf

Type texts

Source 20230719-usctheses-batch-1069 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu