University of Southern California Dissertations and Theses

Stochastic oilfield optimization under uncertainty in future development plans

(USC Thesis Other)

Stochastic oilfield optimization under uncertainty in future development plans

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Stochastic Oilfield Optimization under Uncertainty in
Future Development Plans

By

Atefeh Jahandideh

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
VITERBI SCHOOL OF ENGINEERING
In Partial Fulfillment of the
Requirements for the Degree
Doctor of Philosophy (PETROLEUM ENGINEERING)
UNIVERSITY OF SOUTHERN CALIFORNIA

May 2019

ii

@Copyright 2019 Atefeh Jahandideh

iii

Abstract
Reservoir simulation is a valuable tool for model-based field development and production
performance optimization. In recent years, significant progress has been made in developing
automated workflows for production optimization and field development by combining reservoir
simulation forecasts with numerical optimization schemes. While field development optimization
under geologic uncertainty has received considerable attention, to-date future developments and
their associated uncertainties have not been considered explicitly in field development
optimization. In practice, reservoirs undergo extensive development activities throughout their
life-cycle. Disregarding the possibility of future developments can lead to field performance
predictions and optimization results that may be far from optimal. A complexity in accounting for
future developments is related to the uncertainty in development plans.
In this work, we develop a stochastic field development optimization formulation to
account for the uncertainty in future infill drilling scenarios. The proposed approach optimizes the
decision variables for current stage of planning (e.g. well locations and operational settings) while
accounting for future development uncertainties, where the uncertainty is represented through
drilling scenario trees and probabilistic description of future drilling events/parameters. In the
developed method, for sequential field development under uncertainty in future decisions, multi-
stage stochastic programming is implemented, in which the decision-maker selects an optimal
strategy (e.g. well locations and operational controls) for current stage while accounting for the
residual uncertainty in future development activities. Using a multi-stage stochastic optimization
workflow this process is repeated after each decision stage. Several numerical experiments are
presented to discuss various aspects of the proposed stochastic optimization formulation and to
compare the solutions from different methods adopted for treatment of future development plans.

iv

The results indicate that stochastic treatment of future development events (1) can hedge against
uncertain future development activities by obtaining optimization solutions that are robust against
changes in future decisions, and (2) considerably reduces the performance losses that can result
from field development when uncertainty is disregarded.
The proposed approach offers a fresh perspective on formulating and solving production
optimization problems to explicitly incorporate the uncertainty in future development activities.

v

Acknowledgments
First, and most of all, I would like to extend my sincere appreciation and gratitude to my
Ph.D. adviser, Professor Jafarpour for his patience, guidance, kindness, wisdom and continuous
encouragement throughout my PhD studies. He always gave me constant encouragement and
advice and without his coherent and illuminating instructions, this dissertation would not have
reached its present form. I feel very privileged to have been one of his students. I am indebted for
his tremendous support and opportunities he provided throughout my doctoral studies at USC. My
thanks also go to the members of my committee Professor Ershaghi and Professor Ghanem for
their valuable insight and comments.
I would like to express my heartfelt gratitude to Professor Ershaghi for his constant
inspiration, encouragement and guidance. His office door was always open whenever I needed his
advice on the research or other problems and I always left his office with motivation. He is a
mentor and like a second father to me and mere words do no justice to the level of gratitude I feel
towards him.
I also would like to thank my dear friends and fellow lab mates in SEES research group. A
big thanks goes to my dear friend Arlen Navasartian for his incredible support and helping me to
overcome the many challenges of living far from family.
Last, but not least, I would like to thank my parents, Hassan Jahandideh and Seyedeh
Khadijeh Jalali and my beloved brother Iman for their love, dedication and many years of support.
Without them, none of this would indeed be possible.

vi

Contents
Abstract ............................................................................................................................................... iii
Acknowledgments ................................................................................................................................ v
List of Tables .................................................................................................................................... viii
List of Figures...................................................................................................................................... ix
Chapter 1 - Introduction ....................................................................................................................... 1
1.1 Optimization in Field Development Planning ............................................................. 1
1.2 Governing Flow Equations: Two-Phase Fluid Flow in Porous Media....................... 2
1.3 Motivation ...................................................................................................................... 5
1.4 Scope of Work and Dissertation Outline ...................................................................... 7
Chapter 2 - Stochastic Oilfield Optimization under Uncertain Future Development Plans .......... 10
2.1 Introduction .................................................................................................................. 10
2.2 Motivating Example .................................................................................................... 16
2.2.1 Case I: Disregarding Future Drilling ................................................................... 18
2.2.2 Case II: Incorporating Future Infill Drilling ....................................................... 20
2.3 Problem Statement ....................................................................................................... 23
2.4 Optimization Problem Formulation ............................................................................ 24
2.4.1 Deterministic Field Development Optimization................................................. 24
2.4.2 Stochastic Field Development Optimization ...................................................... 26
2.5 Solution Approach ....................................................................................................... 35
2.5.1 Deterministic Field Development Optimization................................................. 35
2.5.2 Stochastic (Here-and-now) Field Development Optimization .......................... 37
2.6 Numerical Experiments ............................................................................................... 39
2.6.1 Case Study 1: Two-Stage Optimization .............................................................. 40
2.6.2 Case Study 2: Multi-Stage Optimization ............................................................ 50
2.7 Conclusion .................................................................................................................... 65
Chapter 3 - Optimization under Uncertainty in Future Operations................................................. 69
3.1 Introduction .................................................................................................................. 69
3.2 Problem Formulation ................................................................................................... 69
3.2.1 Modeling the Uncertainty in Future Operations ................................................. 70
3.3 Optimization Solution Approach ................................................................................ 73
3.4 Numerical Experiments ............................................................................................... 75
3.4.1 Case Study I: Uncertainty in future infill well location and control settings ... 75
3.4.2 Case Study II: PUNQ-S3 Model ......................................................................... 83
3.5 Conclusion .................................................................................................................... 96

vii

Chapter 4 - Closed-Loop Stochastic Oilfield Optimization under Uncertainty in Geologic
Description and Future Development Plans...................................................................................... 99
4.1 Introduction .................................................................................................................. 99
4.2 Mathematical Formulation ........................................................................................ 105
4.2.1 Closed-loop Optimization Workflow ................................................................ 106
4.2.2 Data Assimilation with EnKF ............................................................................ 108
4.2.3 Handling Uncertainty in future Well Locations and Controls ......................... 109
4.3 Numerical Experiments ............................................................................................. 111
4.3.1 Case Study 1: Two-stage Closed-loop Optimization under uncertainty in future
number of wells............................................................................................................ 112
4.3.2 Case study 2: Two-stage Closed-loop Optimization under uncertainty in future
number of wells, locations and controls ..................................................................... 120
4.3.3 Case study 3: Multi-stage Closed-loop Optimization under uncertainty in future
number of wells, locations and controls ..................................................................... 128
4.4 Conclusion .................................................................................................................. 138
Chapter 5 - Summary, Conclusions and Future Work ................................................................... 139
5.1 Summary .................................................................................................................... 139
5.2 Conclusions ................................................................................................................ 140
5.3 Discussion .................................................................................................................. 141
5.4 Future Research Directions ....................................................................................... 143
Nomenclature ................................................................................................................................... 145
Bibliography ..................................................................................................................................... 147
Appendix A: SPSA Algorithm ........................................................................................................ 152
Appendix B: Multi-Stage Stochastic Optimization........................................................................ 155

viii

List of Tables
Table 2.1: Reservoir model parameters for PUNQ-S3 model ........................................................ 17
Table 2.2: NPV, water injection, oil production and water production results of the case where the
knowledge of future drilling is not incorporated in the optimization scheme from the beginning
and Perfect Information case (knowledge of future drilling is accounted for from the start of the
project) for one infill well .................................................................................................................. 22
Table 2.3: Decision stages and the order of observations and decisions in multi-stage stochastic
optimization with recourse ................................................................................................................. 31
Table 2.4: Pseudo-code for integer SPSA ........................................................................................ 36
Table 2.5: Parameter used for calculating the objective function (-NPV) for the first and second
case studies .......................................................................................................................................... 40
Table 2.6: Results of two-stage field development and control optimization (Case study 1) ...... 43
Table 2.7: Optimal drilling schdule for deterministic field development optimization ................ 51
Table 2.8: NPV, water injection, oil production and water production results of deterministic
(look-ahead) field development optimization ................................................................................... 51
Table 2.9 NPV, water injection, oil production and water production results for the case where the
uncertainty in possible future drilling is disregarded ...................................................................... 54
Table 2.10: NPV, water injection, oil production and water production results for the Expected
Value Problem with 𝜔 =2 ................................................................................................................ 55
Table 2.11: NPV, water injection, oil production and water production results of stochastic (here-
and-now) field development optimization ........................................................................................ 61
Table 3.1 : Reservoir model paramters for SPE10 model ............................................................... 76
Table 3.2: Reservoir model parameters ............................................................................................ 84
Table 3.3: Results of two-stage field development and control optimization .............................. 86
Table 4.1: Closed-loop field development optimization pseudo-code. ........................................ 106
Table 4.2: Reservoir model parameters for PUNQ-S3 model ...................................................... 112
Table 4.3: Results of two-stage closed-loop field development optimization ............................. 117
Table 4.4: Reservoir model parameters for second case study ..................................................... 121
Table 4.5: Reservoir parameters for the Norne model .................................................................. 129

ix

List of Figures
Figure 2.1: Horizontal permeability map (millidarcy) for 5 layers of PUNQ-S3 model .............. 17
Figure 2.2: a) top: Optimal configuration for 3 producers and 3 injectors without accounting for
the future drilling opportunity; bottom: Oil saturation map after 5 years of production
corresponding to optimal configuration of the top figure b) top: Optimal location for the infill well
(PROD4) obtained by re-implementing the optimization algorithm; bottom: Oil saturation map
after a total of 10 years of production. .............................................................................................. 19
Figure 2.3: Optimal well control trajectories for the case where the knowledge of future infill
drilling is disregarded; solid lines are the control trajectories for the 3 producers and 3 injectors
obtained at the beginning of the project without accounting for the future infill well, dashed lines
are the optimal control trajectories for the second 5 years of the project when it is decided to drill
an infill well (PROD4). ...................................................................................................................... 20
Figure 2.4: a) top: Optimal configuration for 4 producers and 3 injectors in Perfect Information
problem; bottom: Oil saturation map after 5 years of production corresponding to optimal
configuration of the top figure b) top: Optimal configuration for 4 producers and 3 injectors in
Perfect Information problem; bottom: Oil saturation map after a total of 10 years of production.
.............................................................................................................................................................. 21
Figure 2.5: Optimal control trajectory of 4 producers and 3 injectors for Perfect Information case
where the knowledge of future infill drilling (PROD4 after 5 years) is incorporated in the
optimization scheme upfront.............................................................................................................. 21
Figure 2.6: NPV comparison of the optimization approach where the knowledge of future infill
drillings is disregarded and Perfect Information optimization (knowledge of future drilling is
accounted upfront) in field development optimization scheme for different years of drilling one
single infill well. ................................................................................................................................. 22
Figure 2.7: Results for case 1 field development (well location and operational control)
optimization problem where the uncertainty in future infill drillings is disregarded; top: optimal
well configuration, bottom: saturation distribution after 10 years of production for (a) No infill
well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ................................. 41
Figure 2.8: Optimal control trajectories of the wells for case 1 where the possibility of future infill
drillings is neglected; solid lines are the control trajectories for the 3 producers and 3 injectors
obtained at the beginning of the project without accounting for the future infill well; dashed lines
are the optimal control trajectories for the second 5 years of the project when it is decided to drill
one infill well; dotted lines are the optimal control trajectories for the second 5 years of the project
when it is decided to drill two infill wells. ........................................................................................ 42

x

Figure 2.9: Results for perfect information optimization (case II); top: optimal well configuration,
bottom: saturation distribution after 10 years of production for (a) No infill well (𝜔 1=0), (b) one
infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ......................................................................... 44
Figure 2.10: Optimal control trajectories of Perfect Information optimization (case II) for (a) No
infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ........................ 44
Figure 2.11: Results for Expected Value Problem (EVP) optimization (case III); top: optimal well
configuration, bottom: saturation distribution for each individual layer after 10 years of production
for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1) and (c) two infill wells (𝜔 3=2). 46
Figure 2.12: Optimal control trajectories of the wells for Expected Value Problem (EVP)
optimization (case III): solid lines are the control trajectories for the 4 producers (PROD4 is open
to flow after 5 years) and 3 injectors obtained at the beginning; dashed lines are the optimal control
trajectories for the second 5 years of the project when it is decided to drill no infill well; dotted
lines are the optimal control trajectories for the second 5 years of the project when it is decided to
drill two infill wells............................................................................................................................. 46
Figure 2.13: First stage solution of stochastic (here-and-now) problem; top: optimal configuration
for 3 producers and 3 injectors; bottom: saturation distribution after 5 years of production. ....... 47
Figure 2.14: Evolution of the objective function for first stage of stochastic (here-and-now)
optimization (sequential implementation)......................................................................................... 47
Figure 2.15: Results for stochastic (here-and-now) optimization (case IV); top: optimal well
configuration, bottom: saturation distribution for each individual layer after 10 years of production
for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ...... 48
Figure 2.16: Optimal control trajectories of stochastic optimization (case IV) for (a) No infill well
(𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ......................................... 48
Figure 2.17: Percetage-wise change in final values of NPV for No Drilling (Disregarfing the
uncertainty), EVP (Expected value problem) and Perfect Information optimization approaches
compared with stochastic (here-and-now) solution.......................................................................... 50
Figure 2.18: a) top: Initialization for deterministic field development optimization (injectors and
producers are shown with red crosses and black circles, respectively) ; bottom: Oil saturation map
after 10 years of production corresponding to initial configuration b) top: Optimal configuration
obtaind by implementing the optimization algorithm for well placement and well scheduling;
bottom: Oil saturation map after a total of 10 years of production for optimal design. ................. 51
Figure 2.19: Evolution of the objective function for deterministic (look-ahead) well placement
and well scheduling optimization (sequential implementation). ..................................................... 51
Figure 2.20: sequential well placement without accounting for the future drilling plans for the
scheduling 4-1-1-2 (4 wells (2 injectors+ 2 producers) at beginning, 1 well after two years, 1 well
after four years and 2 wells after 6 years) (a) left: optimal location for producers P1 & P2 and
injectors I1 & I2 scheduled to be drilled at the beginning of project; right: Saturation map after a

xi

total of 2 years of production (b) left: optimal location for P3 scheduled to be drilled at the
beginning of the second stage; right: Saturation map after a total of 4 years of production (c) left:
optimal location for P4 scheduled to be drilled at the beginning of the third stage; right: Saturation
map after a total of 6 years of production (d) left: optimal locations for P5 and P6 scheduled to be
drilled at the beginning of the fourth stage; right: Saturation map after a total of 10 years of
production. .......................................................................................................................................... 53
Figure 2.21: sequential well placement using expected value approach 4-1-1-2 (4 wells (2
injectors+ 2 producers) at beginning, 1 well after two years, 1 well after four years and 2 wells
after 6 years) (a) left: optimal location for producers P1 & P2 and injectors I1 & I2 scheduled to
be drilled at the beginning of project; right: Saturation map after a total of 2 years of production
(b) left: optimal location for P3 scheduled to be drilled at the beginning of the second stage; right:
Saturation map after a total of 4 years of production (c) left: optimal location for P4 scheduled to
be drilled at the beginning of the third stage; right: Saturation map after a total of 6 years of
production (d) left: optimal locations for P5 and P6 scheduled to be drilled at the beginning of the
fourth stage; right: Saturation map after a total of 10 years of production. .................................... 55
Figure 2.22: Scenario tree for sequential well placement considering future development
possibilities with 4 stages ................................................................................................................... 56
Figure 2.23: left: Optimal configuration obtaind by implementing the optimization algorithm for
well placement while accounting for future drilling possilibities (injectors and producers are
shown with red crosses and black circles, respectively); right: Oil saturation map after a 2 years of
production according to optimal design. ........................................................................................... 58
Figure 2.24: Scenario tree for the second stage well placement (highlighted in red) assuming it is
decided to drill one infill well in the second stage ........................................................................... 59
Figure 2.25: NPV of sequential four-stage well placement for all 64 drilling scenarios vs. their
corresponding probabilities (with incorporating the uncertainy in future drilling possibilties) .... 60
Figure 2.26: 4-stage well placement for the scheduling 4-1-1-2 (4 wells (2 injectors+ 2 producers)
at beginning, 1 well after two years, 1 well after four years and 2 wells after 6 year) (a) left: optimal
location for P3 scheduled to be drilled at the beginning of the second stage; right: Saturation map
after a total of 4 years of production (b) left: optimal location for P4 scheduled to be drilled at the
beginning of the third stage; right: Saturation map after a total of 6 years of production (c) right:
optimal locations for P5 and P6 scheduled to be drilled at the beginning of the fourth stage; left:
Saturation map after a total of 10 years of production. .................................................................... 60
Figure 2.27: Comparison of (a) normalized NPV, (b) normalized water production, (c) normalized
oil production and (d) recovery for sequential well placement of for the case with no incorporation
of uncertainty in future drilling in optimization scheme (No Drilling-white bar), Expected value
approach (EVP-light gray bar), stochastic approach where the uncertainty in possible future
drillings is accounted for (dark gray bar) and perfect information approach (black bar) .............. 62
Figure 2.28: Comparison of NPV differences (in percentage) between stochastic approach and the
case where uncertainty in future drilling is disregarded (No Drilling) for all 64 possible drilling
scenarios .............................................................................................................................................. 63

xii

Figure 2.29: Comparison of normalized NPV vs. the occurrence probability of 64 drilling
scenarios between stochastic (here-and-now) approach and the approach in which the uncertainty
in future drillings is disregarded (No Drilling). ................................................................................ 64
Figure 2.30: Comparison of normalized NPV vs. the occurrence probability of 64 drilling
scenarios between Perfect Information (wait-and-see) and stochastic (here-and-now) approaches
.............................................................................................................................................................. 65
Figure 3.1: (a) Initial saturation map; (b) Normalized permeability map; (c) Normalized porosity
map; (d) Corresponding quality map of the reservoir. ..................................................................... 71
Figure 3.2: (a) Quality map after drilling two injectors (I1 & I2) and two producers (P1 & P2); (b)
Smoothed-out quality map; (c) Clustered quality map to identify promising drilling locations; (d)
Labelled potential drilling regions based on quality map. ............................................................... 72
Figure 3.3: Evolution of objective function for the base case by implementing sequential well
placement and well control optimization .......................................................................................... 76
Figure 3.4: (left) Optimized well configuration for 10 years of production; the injectors and
producers are marked with (×) and (•), respectively (P5 is an infill well scheduled to be drilled
after 6 years); (right) saturation map after 10 years ......................................................................... 77
Figure 3.5: Optimized 5-step control trajectories of the producers and injectors in base case ..... 77
Figure 3.6: Evolution of objective function for location uncertainty case by implementing
sequential well placement and well control optimization ................................................................ 78
Figure 3.7: (left) Optimized well configuration for 10 years of production; the injectors and
producers are marked with (×) and (•), respectively, the realizations for possible P5 locations are
marked with (o) (P5 is an infill well scheduled to be drilled after 6 years); (right) saturation map
after 10 years for one the realizations................................................................................................ 78
Figure 3.8: Optimized 5-step control trajectories of the producers and injectors when the location
of the infill well (P5) is an uncertain variable................................................................................... 78
Figure 3.9: 50 random realizations generated for the location of P5 ............................................. 79
Figure 3.10: (left) NPV values for each 50 realizations of Figure 19 calculated based on base case
(nominal values) and uncertain location example (stochastic values); (right) empirical cumulative
density function for NPV values of nominal and stochastic cases. ................................................. 79
Figure 3.11: Evolution of objective function for control uncertainty case by implementing
sequential well placement and well control optimization. ............................................................... 80
Figure 3.12: (left) Optimized well configuration for 10 years of production (P5 is an infill well
scheduled to be drilled after 6 years) when the second stage control trajectories are uncertain;
(right) saturation map after 10 years for one of the control realizations. ........................................ 80

xiii

Figure 3.13: Optimized first stage (T0:T6) and uncertain second stage (T6:T10) control
trajectories when the future control trajectories are posed as random variables. ........................... 81
Figure 3.14: New set of random realizations generated for second stage’s control trajectories for
year 6-10 for all 7 wells (5 producers and 2 injectors). .................................................................... 81
Figure 3.15: (left) total NPV values for each 50 realizations of Figure 24 calculated for base case
(nominal values) and uncertain location example (stochastic values); (right) empirical cumulative
density function for NPV values of nominal and stochastic cases. ................................................. 81
Figure 3.16: Evolution of objective function for location & control uncertainty case by
implementing sequential well placement and well control optimization. ....................................... 82
Figure 3.17: (left) Optimized well configuration; the initial injectors and producers are marked
with (×) and (•), respectively, the 20 realizations for possible P5 locations are marked with (o) (P5
is an infill well scheduled to be drilled after 6 years); (right) saturation map after 10 years for one
the realizations. ................................................................................................................................... 82
Figure 3.18: Optimized first stage (T0:T6) and uncertain second stage (T6:T10) control
trajectories when the future control trajectories and well locations are posed as random variables.
.............................................................................................................................................................. 83
Figure 3.19: empirical cumulative density function for NPV values of nominal and stochastic
cases. .................................................................................................................................................... 83
Figure 3.20: Results for case 1 field development (well location and operational control)
optimization problem where the uncertainty in future infill drillings is disregarded; top: optimal
well configuration, bottom: saturation distribution for each individual layer after 10 years of
production for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=
2).......................................................................................................................................................... 85
Figure 3.21: Optimal control trajectories of the wells for case 1 where the possibility of future
infill drillings is neglected; solid lines are the control trajectories for the 3 producers and 3 injectors
obtained at the beginning of the project without accounting for the future infill well; dashed lines
are the optimal control trajectories for the second 5 years of the project when it is decided to drill
one infill well; dotted lines are the optimal control trajectories for the second 5 years of the project
when it is decided to drill two infill wells......................................................................................... 86
Figure 3.22: Results for perfect information optimization (case II); top: optimal well
configuration, bottom: saturation distribution for each individual layer after 10 years of production
for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ...... 87
Figure 3.23: Optimal control trajectories of Perfect Information optimization (case II) for (a) No
infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). ........................ 88
Figure 3.24: Results for stochastic (here-and-now) optimization (case 3- variant I); top: optimal
well configuration, bottom: saturation distribution for each individual layer after 10 years of

xiv

production for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=
2).......................................................................................................................................................... 89
Figure 3.25: Optimal control trajectories of stochastic optimizations (case 3 & case 4) for (a) No
infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2).[solid lines for
variant I (case 3) and dashed lines for variant II (case 4) of the stochastic programming]. .......... 90
Figure 3.26: Final location realizations for stochastic (here-and-now) optimization (case 4- variant
II) for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2). the
injectors and producers are marked with (×) and (•), respectively, the realizations for possible
future well locations are marked with (o). ........................................................................................ 91
Figure 3.27: Realizations of control trajectories for second stage (year 6-10) of case 4 for (a) no
infill well (𝜔 1=0) (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=2) for one of the
location realizations. ........................................................................................................................... 91
Figure 3.28: Results for stochastic (here-and-now) optimization (case 4- variant II); top: optimal
well configuration, bottom: saturation distribution for each individual layer after 10 years of
production for (a) No infill well (𝜔 1=0), (b) one infill well (𝜔 2=1), (c) two infill wells (𝜔 3=
2).......................................................................................................................................................... 92
Figure 3.29: Comparing the final normalized NPV values No Drilling (Disregarding the
uncertainty), stochastic (here-and-now) solution I and II and Perfect Information optimization. 93
Figure 3.30: 25 random realizations generated for the location of (a) one infill well (𝜔 2=1), (b)
two infill wells (𝜔 3=2). .................................................................................................................. 94
Figure 3.31: Empirical cumulative density function for NPV values of perfect information and
stochastic solution I&II for location realizations of Figure 3.30 (a) one infill well (ω2=1), (b)
two infill wells (ω3=2). .................................................................................................................. 95
Figure 3.32: Random samples for control trajectories for the second stage (year 6-10) for (a) one
infill well (ω2=1), (b) two infill wells (ω3=2). ........................................................................ 96
Figure 3.33: Empirical cumulative density function for NPV values of perfect information and
stochastic solution I&II for control realizations of Figure 3.32 (a) one infill well (ω2=1), (b)
two infill wells (ω3=2). .................................................................................................................. 96
Figure 4.1: Closed-loop oilfield optimization under geologic and future development
uncertainties. ..................................................................................................................................... 108
Figure 4.2: (a) Log-horizontal permeability map (millidarcy), and (b) porosity map for the
reference PUNQ-S3 model. ............................................................................................................. 114
Figure 4.3: (a) Initial ensemble mean, and (b) & (c) two realizations from the initial ensemble for
𝐿𝑜𝑔 (𝑘 ) (first row) and 𝜑 (second row). .......................................................................................... 114

xv

Figure 4.4: Optimal well configurations for the initial wells (Prod1, Prod2, Inj1 and Inj2) under
No Drilling assumption shown on reference (a) permeability (b) porosity map. ......................... 115
Figure 4.5: (top) Optimal control trajectories for the initial wells (Prod1, Prod2, Inj1 and Inj2) in
the No Drilling case; (bottom) predicted reservoir performance of the optimal solution over the
initial ensemble of 100 geologic realizations.................................................................................. 115
Figure 4.6: Optimal control trajectory for the No Drilling case after 5 years with 6 steps of model
calibration (solid line) and initial optimal control trajectory (dashed line). ................................. 115
Figure 4.7: (top) final configuration and the mean of final updated (after 10 years) ensemble of
geologic realizations representing logarithm of permeability; (bottom) the mean of final updated
(after 10 years) ensemble of geologic realizations representing porosity for (a) No infill drilling,
(b) one infill well, (c) two infill wells scenarios. ............................................................................ 116
Figure 4.8: Reservoir response (BHP and WOR) of the wells for each final geologic realization
(cyan) and the observations from the reference model (red) for (a) no infill drilling, (b) one infill
well, (c) two infill well scenarios. ................................................................................................... 116
Figure 4.9: Optimal well configurations for the initial wells (Prod1, Prod2, Inj1 and Inj2) and for
the Stochastic Solution approach shown on the reference (a) permeability (b) porosity maps. .. 117
Figure 4.10: (top) Optimal control trajectories of the initial wells (Prod1, Prod2, Inj1 and Inj2) for
stochastic solution; (bottom) predicted reservoir performance of the optimal solution over the
initial ensemble of 100 geologic realizations.................................................................................. 118
Figure 4.11: (top) Final well configurations and the updated ensemble mean (after 10 years) of
model realizations representing log-permeability; (bottom) mean of final (after 10 years) ensemble
of geologic realizations representing porosity for (a) no infill drilling, (b) one infill well, (c) two
infill well scenarios for Stochastic Solution. .................................................................................. 118
Figure 4.12: Reservoir response (BHP and WOR) of the wells for each final geologic realization
(cyan) and the observations from the reference model (red) for (a) no infill drilling, (b) one infill
well, (c) two infill well scenarios. ................................................................................................... 119
Figure 4.13: NPV performance for No Drilling and Stochastic Solution under each drilling
scenario. ............................................................................................................................................. 119
Figure 4.14: (a) Reference Log-horizontal permeability map (millidarcy); (b) mean of prior
ensemble for Log-horizontal permeability maps; (c) sample #1 of prior and (d) sample #2 of prior.
............................................................................................................................................................ 121
Figure 4.15: decision tree for a two-stage field development optimization considering 14 future
development possibilities. ................................................................................................................ 121
Figure 4.16: Optimal well configurations for the initial wells (P1, P2, P3, I1 and I2) under No
Drilling assumption shown on mean of prior ensemble. ................................................................ 123

xvi

Figure 4.17: Optimal control trajectories for the initial wells (P1, P2, P3, I1 and I2) at the end of
year 5 after 6 steps of model updating in the No Drilling case. .................................................... 123
Figure 4.18: (top) final optimal configuration and the mean of final updated (after 10 years)
ensemble of geologic realizations representing logarithm of permeability and their corresponding
saturation map (at year 10) for four drilling scenarios (a) No Infill well, (b) 1 𝑝𝑟𝑜𝑑 +1 𝑖𝑛𝑗 , (c)
2 𝑝𝑟𝑜𝑑 +2 𝑖𝑛𝑗 and (d) 4 𝑝𝑟𝑜𝑑 +1 𝑖𝑛𝑗 and their corresponding saturation map (at year 10) for
the case where the possibility of future drilling opportunities are unaccounted for (No Drilling);
(bottom) final optimal configuration and the mean of final updated (after 10 years) ensemble of
geologic realizations representing logarithm of permeability for the same four drilling scenarios
and their corresponding saturation map (at year 10) for the case where future drilling possibilities
are modelled as uncertain variable (stochastic approach). ............................................................. 124
Figure 4.19: (a) Evolution of Objective function with different number of samples for geologic
realizations, future drilling scenarios, future well locations and future control trajectories (b) Mean
of coefficient of variations over all iterations for different number of samples. .......................... 126
Figure 4.20: (a) Initial configuration for the stochastic case; Optimal configuration for the
stochastic case with (b) 300 samples, (c) 500 samples and (d) 2000 samples for geologic
realizations, future drilling scenarios, future well locations and future control trajectories ........ 126
Figure 4.21: Optimal control trajectories of the initial wells (P1, P2, P3, I1 and I2) of the first five
years for stochastic solution for 300 (dashed lines), 500 (dotted lines) and 2000 (solid lines)
samples for geologic realizations, future drilling scenarios, future well locations and future control
trajectories. ........................................................................................................................................ 126
Figure 4.22: NPV performance for No Drilling and Stochastic Solution under each drilling
scenario. ............................................................................................................................................. 128
Figure 4.23: (a) Reference Log-horizontal permeability map (millidarcy); (b) initial saturation
map; (c) mean of initial ensemble for Log-horizontal permeability maps.................................... 129
Figure 4.24: decision tree for a multi-stage field development optimization considering 75 future
development possibilities ................................................................................................................. 130
Figure 4.25: (a) Optimal well configurations for the initial wells (Prod1, Prod2, Prod3, Inj1 and
Inj2) under No Drilling assumption shown on initial mean of ensemble; (b) Optimal configuration
for the stochastic case with 500 samples for geologic realizations, future drilling scenarios, future
well locations and future control trajectories. ................................................................................. 131
Figure 4.26: (top) final optimal configuration and the mean of final updated (after 8 years)
ensemble of geologic realizations representing logarithm of permeability for three drilling
scenarios (a) No Infill well 00,00; (b) 00,+10 (1 producing infill well in third stage) and (c) +1−
1,+10 (one producing and one injecting infill wells in second stage and an additional producer in
third stage) and their corresponding saturation map (at year 8) for the case where the possibility of
future drilling opportunities are unaccounted for (No Drilling); (bottom) final optimal
configuration and the mean of final updated (after 8 years) ensemble of geologic realizations
representing logarithm of permeability for the same three drilling scenarios and their

xvii

corresponding saturation map (at year 8) for the case where future drilling possibilities are
modelled as uncertain variable (stochastic approach) .................................................................... 132
Figure 4.27: (top) final optimal configuration and the mean of final updated (after 8 years)
ensemble of geologic realizations representing logarithm of permeability for three drilling
scenarios (a) 0−2,+2−1 (2 injectors in second stage and 2 producers and 1 injector in third
stage); (b) +20,+1−1, (2 producers in second stage and 1 producer and 1 injector in third stage)
and (c) +20,+2−1 (2 producers in second stage and 2 producers and 1 injector in third stage)
and their corresponding saturation map (at year 8) for the case where the possibility of future
drilling opportunities are unaccounted for (No Drilling); (bottom) final optimal configuration and
the mean of final updated (after 8 years) ensemble of geologic realizations representing logarithm
of permeability for the same three drilling scenarios and their corresponding saturation map (at
year 8) for the case where future drilling possibilities are modelled as uncertain variable (stochastic
approach) ........................................................................................................................................... 133
Figure 4.28: Evolution of the objective function in multi-stage optimization case where the
possibility of future drillings is accounted for upfront. .................................................................. 135
Figure 4.29: NPV performance for No Drilling and Stochastic Solution under each drilling
scenario in multi-stage optimization. .............................................................................................. 136
Figure 4.30: (a) Empirical CDF for NPV values of No Drilling and stochastic based on 50 well
location realizations (b) Empirical CDF for NPV values of No Drilling and stochastic based on 50
control realizations............................................................................................................................ 137

Chapter 1: Introduction

1

Chapter 1
Introduction

1.1 Optimization in Field Development Planning
Optimization problems frequently arise in Reservoir Engineering applications where the
goal is to find a set of optimal design and control variables that optimize some objective function
that is a measure of reservoir performance. Typically, in addition to operating constraints, there is
a significant level of uncertainty associated with reservoir parameters (permeability, porosity,
initial oil saturation, etc.) as well as future drilling decisions (e.g. number of infill wells) that needs
to be taken into account while solving for the field development optimization problems.
Because decision variables in field development optimization consists of both discrete and
continuous variables and typically the objective function is nonlinear, the optimization problem is
classified as mixed-integer nonlinear programing (MINLP).
As mentioned earlier, the objective function can be any measure of reservoir performance
such as Net Present Value (NPV) or cumulative oil production or recovery factor during project’s
lifespan. For conventional reservoirs the decision variables can include number of the wells,
location of the wells, type of the wells (injector/producer and vertical/horizontal), drilling schedule
and well controls.
In addition to different decision variables, there are different types of constraints such as
bound and linear constraints on decision variables and nonlinear constraints on simulator output
in order to ensure physical feasibility of optimal solution.
Chapter 1: Introduction

2

A challenge associated with field development MINLP problems are the uncertainty due
to scarcity of the data needed to describe the reservoir properties. Uncertainty in reservoir
properties are usually characterized by multiple equi-probable realizations, where all of them are
considered in the optimization. Another source of uncertainty in field development optimization
is the uncertainty in the future development plans which brings in another classification of
optimization called stochastic programming in which the present decision variables are optimized
based on possible future events. A common tool in considering future uncertainty is to work with
scenarios, which are particular representations of how the future might unfold. Some kind of
probabilistic model or simulation is used to generate a batch of such scenarios.
In addition to all these complexities, the field development optimization is very
computationally demanding, every objective function evaluation requires performing one forward
run using the simulator which can be expensive for large models. In general, all of these issues
must be considered in order to obtain truly optimal field development plans.
1.2 Governing Flow Equations: Two-Phase Fluid Flow in Porous
Media
Reservoir simulation is used to predict multiphase fluid flow of hydrocarbons and water
within porous rocks. Reservoir heterogeneity often leads to non-uniform displacement of fluids,
resulting in complex spatiotemporal evolution of the oil and water saturation fronts. Throughout
the dissertation, we consider immiscible two-phase fluid flow systems, consisting of oil and water,
in heterogeneous geologic formations. Assuming oil as the non-wetting phase fluid with subscript
o and water as the wetting phase with subscript w, the mass conservation equations are combined
with the Darcy’s law to arrive at the following system of partial differential equations that describe
simultaneous flow of oil and water in a porous formation (Peaceman, 1977):

Chapter 1: Introduction

3

∇.[
𝛼 𝜌 𝑜 𝐾 𝑘 𝑟𝑜
𝜇 𝑜 (∇𝑃 𝑜 −𝜌 𝑜 𝑔 ∇𝐷 )]+𝛼 𝜌 𝑜 𝑞 𝑜 =𝛼 𝜕 (𝜑 𝜌 𝑜 𝑆 𝑜 )
𝜕𝑡

(1.1)

∇.[
𝛼 𝜌 𝑤 𝐾 𝑘 𝑟𝑤
𝜇 𝑤 (∇𝑃 𝑤 −𝜌 𝑤 𝑔 ∇𝐷 )]+𝛼 𝜌 𝑤 𝑞 𝑤 =𝛼 𝜕 (𝜑 𝜌 𝑤 𝑆 𝑤 )
𝜕𝑡

where the notations used are as follows, 𝐾 is absolute permeability, 𝜌 is fluid density, 𝑔 is
gravitational acceleration, 𝜑 is rock porosity, ∇. is the divergence operator (i.e., ∇.𝑢⃗ =
𝜕 𝑢 𝑥 𝜕𝑥
+
𝜕 𝑢 𝑦 𝜕𝑦
+
𝜕 𝑢 𝑧 𝜕𝑧
), 𝛼 is a geometric factor (1D: 𝛼 = 𝐴 (𝑥 ) cross section area, 2D: 𝛼 = ℎ(𝑥 ,𝑦 ) reservoir
thickness, 3D: 𝛼 =1), 𝑞 is the flow rate that the fluid is injected/produced per unit volume, 𝜇 is
fluid viscosity, 𝑝 is fluid pressure, 𝐷 is depth coordinate, 𝑆 is fluid saturation with the physical
constraint 𝑆 𝑜 +𝑆 𝑤 =1, and 𝑘 𝑟 is fluid relative permeability (function of saturation). For a two-
phase, 2D, immiscible, incompressible displacement with no capillary pressure, in a gravity-free
environment and ignoring the variation of porosity with pressure, the governing equations for
phase 𝑛 (𝑛 = oil, water) reduce to:

−
ℎ
𝜇 𝑛 [
𝜕 𝜕𝑥
(𝑘 𝑘 𝑟𝑛
𝜕𝑃
𝜕𝑥
)+
𝜕 𝜕𝑦
(𝑘 𝑘 𝑟𝑛
𝜕𝑃
𝜕𝑦
)]+ℎ[𝜑 𝑆 𝑛 (𝑐 𝑛 +𝑐 𝑟 )
𝜕𝑃
𝜕𝑡
+𝜑 𝜕 𝑆 𝑛 𝜕𝑡
]−ℎ𝑞 𝑛 =0
(1.2)

Where 𝑐 𝑛 and 𝑐 𝑟 define the compressibility of the fluid and rock, respectively. Note that
the subscript 𝑛 has been dropped for the pressures because the absence of capillary pressure
implies that 𝑃 𝑜 =𝑃 𝑤 . The first and second terms in Equation (1.2) are flux and accumulation
terms, respectively, while the third term accounts for the source/sink terms. All boundaries of the
domain are considered as no-flow and only well locations are used to represent boundary
conditions. Fluid injection or production takes place at wells that are represented by point
sources/sinks with a negative sign for production and a positive sign for injection. By including
Chapter 1: Introduction

4

the saturation constraint and capillary pressure functions (assumed zero in our example), the PDE
system is closed and can be solved numerically after discretization in time and space.
The finite difference discretized version of Equation (1.2) by applying block-centered
central-difference scheme with uniform grid to approximate the spatial differentials can be written
as (Jansen, 2013):

𝑉 [𝜑 𝑆 𝑛 (𝑐 𝑛 +𝑐 𝑟 )
𝜕𝑃
𝜕𝑡
+𝜑 𝜕 𝑆 𝑛 𝜕𝑡
]
𝑖 ,𝑗 −(𝑇 𝑛 )
𝑖 −
1
2
,𝑗 𝑃 𝑖 −1,𝑗 −(𝑇 𝑛 )
𝑖,𝑗 −
1
2
𝑃 𝑖 ,𝑗 −1
+[(𝑇 𝑛 )
𝑖 −
1
2
,𝑗 +(𝑇 𝑛 )
𝑖 ,𝑗 −
1
2
+(𝑇 𝑛 )
𝑖 +
1
2
,𝑗 +(𝑇 𝑛 )
𝑖 ,𝑗 +
1
2
]𝑃 𝑖 ,𝑗 −(𝑇 𝑛 )
𝑖 ,𝑗 +
1
2
𝑃 𝑖 ,𝑗 +1
−(𝑇 𝑛 )
𝑖 +
1
2
,𝑗 𝑃 𝑖 +1,𝑗 =𝑉 (𝑞 𝑛 )
𝑖 ,𝑗

(1.3)

Where the discretized transmissibilities are defined as:

(𝑇 𝑛 )
𝑖,𝑗 ≜
∆𝑦 ∆𝑥 ℎ
𝜇 𝑛 (𝑘 𝑘 𝑟𝑛
)
𝑖 ,𝑗
(1.4)

And unit volume 𝑉 is defined as:

𝑉 =ℎ∆𝑥 ∆𝑦 (1.5)

The matrix form of the resulting discretized equations (for water and oil, 𝑛 = 𝑤 ,𝑜 ) can
be expressed as:

[
𝑉 𝑤𝑝
𝑉 𝑤𝑠
𝑉 𝑜𝑝
𝑉 𝑜𝑠
][
𝑃 ̇ 𝑆 ̇ ]
⏟
Acumulation term
+[
𝑇 𝑤 0
𝑇 𝑜 0
][
𝑃 𝑆 ]
⏟
flux term
= [
𝑞 𝑤 𝑞 𝑜 ]
⏟
source term

(1.6)

In Equation (1.6), the saturation and pressure values in all cells are denoted by the vectors
𝑃 and 𝑆 . Vectors 𝑞 𝑤 and 𝑞 𝑜 in the source/sink term denote the flow rates of water and oil phases
in all the cells with their entries arranged in the following format:
Chapter 1: Introduction

5

𝑞 𝑤 𝑇 ≜[⋯(𝑞 𝑤 )
𝑖,𝑗 ⋯]
(1.7)

𝑞 𝑜 𝑇 ≜[⋯(𝑞 𝑜 )
𝑖 ,𝑗 ⋯]

For any given time, the entries of the source/sink vector are zero for cells that do not contain
a well. Therefore, the source/sink vectors in Equation (1.6) contain the information about well
locations, types, and control settings and hence, these vectors constitute the decision variables of
the generalized field development optimization problem. In a typical oilfield, only a small number
of grid blocks are intersected by wells and, thus, the vast majority of entries in the source/sink
vector (𝑞 𝑇 =[
𝑞 𝑤 𝑇 𝑞 𝑜 𝑇 ]) are zeros.
1.3 Motivation
The development of the computational optimization procedures for oil field operations has
been an area of active research in recent years. Optimization techniques have been developed for
several types of field developments and operational decisions both for conventional and
unconventional reservoirs.
An important issue in field development optimization is the significant uncertainty in the
available information, especially at the early stages of the project. In general, three main sources
of uncertainty can be defined in field development planning: geologic uncertainty, economic
uncertainty, and operation/development uncertainty. Geological uncertainty is related to
incomplete information about reservoir description, including rock property distributions such as
permeability and porosity. Geologic uncertainty has been considered by several investigators in
the literature, where the uncertainty is treated using a probabilistic distribution for the underlying
rock properties. Monte-Carlo simulation is often used to propagate this uncertainty through the
forecast models to approximate the resulting uncertainty in the adopted objective function (e.g.,
Chapter 1: Introduction

6

estimated NPV). Economic uncertainty is also a significant contributor to field development and
project management. It often involves future oil prices as one of the main drivers. In addition to
regular changes in oil prices, for example due to supply and demand variations, unpredictable
factors such as geopolitical effects can also result in significant economic uncertainty. The latter
uncertainty is difficult to incorporate in field development studies as they are not predictable. A
typical approach to deal with (predictable) economic uncertainty is by using a probabilistic
economic model that incorporates a range of plausible scenarios for changes in oil prices. Such a
model can be used to describe the uncertainty in the NPV or monetary performance metrics (Siraj
et al., 2015). One source of uncertainty that has not been studied in production optimization and
reservoir development is the uncertainty in future development activities (e.g., infill drilling). In
real-life field development projects due to high cost of drilling as well as limited information at
the early stages, a few number of wells are drilled and operated for some time to acquire more
information about the reservoir and based on the collected information, decisions regarding drilling
new wells (infill wells) are made. Therefore, optimization techniques should always consider the
possibility of changes in field configuration, mainly due to the introduction of new wells. Two
general approaches can be used to include future decision (events) in optimization problems. The
first approach is to include future events as decision variables in the optimization problem in which
all the decision regarding future drillings as well as current drillings are optimized based on the
current information about the reservoir. The main drawback of this approach is that all the
decisions regarding the planning and operations of the entire reservoir life-cycle is optimized based
on current information. However, the knowledge about the reservoir is constantly evolving as
information is collected and the development plan is consistently updated with the new information
making initial development decisions suboptimal and suboptimal decisions made early in the field
Chapter 1: Introduction

7

life may constrain field operations for years. In practice, it is a complex task to make decisions
about future events in conditions of uncertainty. Hence, a more conservative and robust approach
is to model future drilling events as random variables (as opposed to decision variables), in this
approach multiple plausible drilling scenarios are generated to represent the uncertainty in future
drilling decisions. at each decision-making stage, the decision variables (e.g., location, control,
etc.) for the current-time wells are optimized while accounting for the residual uncertainty in future
drilling events. This has motivated us to incorporate the uncertainty in future development plans
while making the decision for optimal designs at each stage of operations by solving multi-stage
stochastic mixed integer programs. The basic idea is to make some decisions now and to take
some corrective action (recourse) in the future, after revelation of the uncertainty. In multiple
stages, and the decision-maker can take corrective action over a sequence of stages while cost of
the decisions and the expected cost of the recourse actions are optimized (Gupta and Grossmann,
2011).
1.4 Scope of Work and Dissertation Outline
The objective of this research is to perform field development optimization while
accounting for future infill drilling possibilities. With a motivating example in chapter 2 the
significance of including future possible drillings in the well placement optimization is discussed
and the problem is posed in stochastic programming framework in which the uncertainty in future
development plans is incorporated as random variables and the decision making process becomes
dynamic in the sense that the decisions are allowed to depend on observed data as they unfold.
Numerical examples for two-stage and multi-stage stochastic field development optimization will
be presented to show the improvement in reservoir performance by considering the future drilling
events upfront.
Chapter 1: Introduction

8

In chapter 3, we extend random vector to account for uncertainty in future well locations
and control settings in addition to number of infill wells and will show that the obtained solutions
are robust to changes in well location and control trajectories as well as drilling scenario.
In chapter 4, we extend the method to include the uncertainty in the reservoir model
(geologic uncertainty) and the uncertainty in number of future infill wells and their respective
location and control settings in a closed-loop implementation with dynamic model updating
involving repeating the optimization procedure after each model updating step. In this closed-loop
implementation, updatable decision variables (those that are not irreversible) can be adjusted after
each model updating stage. The goal is (1) to solve the optimization problem for well location and
control settings of the current wells while accounting for geologic uncertainty and the uncertainty
in future development scenarios (infill drillings) and (2) incorporate hard data from newly drilled
wells and dynamic response (production) data from all the wells to continuously update reservoir
description and decision variables. The loop of optimization and data integration is repeated at
each step that data is collected. The optimization is performed over multiple geologic realizations
and future development scenarios to account for geologic and development uncertainty,
respectively. Multiple numerical examples are presented to demonstrate the improvement in
reservoir performance in closed-loop stochastic field development optimization where the last
example resembles a realistic reservoir with multiple stages of drilling.
Chapter 5 includes a summary, conclusions, and recommendations for future research
directions.
We have presented our work in several conferences, workshops and meetings including
“Society of Petroleum Engineering, West regional Meeting (2014 & 2018)”. “ECMOR XVI-16th
European Conference on the Mathematics of Oil Recovery (2018)”, and “Society of Petroleum
Chapter 1: Introduction

9

Engineering, Annual Technical Conference and Exhibition (2018)” and scheduled for
presentation in “Society of Petroleum Engineering, Reservoir Simulation Conference (2019) ”. The
results of the first chapter of the dissertation is accepted for publication in SPE Journal, results of
Chapter 3 and 4 have been submitted to Journal of Petroleum Science and Engineering and
Computational Geosciences, respectively.

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

10

Chapter 2
Stochastic Oilfield Optimization
under Uncertain Future Development
Plans
2.1 Introduction
The advent of smart well completion and instrumented oilfields has motivated the
development of numerical optimization procedures to automate oilfield operations. Particularly in
reservoir engineering applications, several decision variables need to be considered in optimizing
the performance of both field development and production operations. The optimization problems
often involve an economic or production efficiency measure as objective function. The main
distinction between field development and field operation optimization is that in the latter the
configuration of the wells (including the location, number, and type of wells) is fixed and only the
operational settings (pressures/flowrates) are considered as decision variables. In that case, the
objective function does not involve any capital costs. In general, optimization of well operational
settings is performed over a long-period (e.g., reservoir life-cycle) to ensure that the control
decisions account for the long-term performance of the reservoir. However, in practice, it is highly
unlikely for a reservoir to have a fixed well configuration throughout its life-cycle. Therefore, the
solutions obtained from life-cycle optimization approaches that assume fixed well configurations
may be far from optimal. To ensure a realistic problem formulation, it is imperative to consider
the possibility of future developments when optimizing the production performance.
One approach to include future developments is to formulate the problem as a field
development optimization. In this case, the decision variables can include well locations, well
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

11

controls, number and type of wells (injector/producer, vertical/horizontal/multilateral) and drilling
schedule. In a recent article, Navabi et al. (2016) demonstrate how optimization of various decision
variables can be formulated in a unified framework with a single decision vector consisting of
source/sink terms in reservoir simulation. An alternative approach to include the future
developments in production optimization is to treat such developments as uncertain events (rather
than decision variables) and formulate the problem in a stochastic programming framework. The
underlying motivation for treating future events as uncertain (variables) is that even when future
development plans are optimized, for various (often unanticipated) reasons, they may not be
implemented perfectly as planned. Therefore, a more robust approach to include possible future
developments in oilfield optimization is to model them as uncertain events. While geologic
uncertainty has been considered in formulating oilfield development and operations optimization
problems (e.g. Van Essen et al., 2009; Siraj et al., 2016), to our knowledge, the uncertainty in
future development plans has not yet been investigated in the literature.
Several optimization methods have been studied for well control and field development
optimization problems. For well control optimization, adjoint-based gradient methods are known
to be computationally efficient. While these methods can get trapped in local solutions, studies
have shown that the objective function in these problems tend to have too many local solutions
with similar values (Siraj et al., 2017). Brouwer and Jansen (2002) used the optimal control theory
to optimize inflow control valve (ICV) settings in smart wells during waterflooding of
heterogeneous reservoirs. They implemented a closed-loop control in which the ensemble Kalman
filter (EnKF) is used to assimilate production data and update reservoir models. The updated
models were then used to predict and maximize the project net present value. Sarma et al. (2006)
used gradient-based methods for optimization of ICV settings and deterministic model updating.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

12

Wang et al. (2009) solved an optimal control problem to determine the operating conditions that
maximize the project NPV using both adjoint and stochastic gradient methods. They also applied
the EnKF for production data integration and model updating. Chen et al. (2009) proposed an
ensemble control approach for production optimization, where they used ensemble-based
approximate gradient information as search direction. Other applications of gradient-based
methods in production optimization can be found in (e.g., Emerick et al, 2009; Beckner and Song,
1995; Centilmen et al, 1999; Montes et al, 2001; Humphries et al, 2013; Humphries and Haynes,
2014).
For well placement optimization in heterogeneous reservoirs, due to discrete nature of well
locations in numerical reservoir simulation models, derivative-free stochastic algorithms have
been popular. Among derivative-free methods, genetic algorithm (GA) has been a popular
evolutionary method for well placement (Guyaguler and Horne, 2001; Badru and Kabir, 2003).
The integer form of the simultaneous perturbation stochastic approximation (SPSA) algorithm has
also been applied to the well placement problem. The integer SPSA algorithm uses one random
direction at each iteration and computes the objective function by placing the well on either sides
of a current location along a selected direction. The solution is then moved to the point that
improves the objective function. Bangerth et al. (2006) compared the efficiency of the SPSA, finite
difference gradient and very fast simulated annealing (VFSA) algorithms for well placement and
showed that the SPSA and VFSA are very efficient in finding nearly optimal solutions with a high
probability. The SPSA algorithm has also been applied to unconventional field development
optimization (Jahandideh and Jafarpour, 2016) where it is used to optimize the location of
hydraulic fracture stages and their corresponding half-length in a shale gas reservoir.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

13

Well locations and control settings are important decision variables that affect the recovery
efficiency during reservoir life-cycle. These decision variables are not independent of each other,
in the sense that optimal well locations depend on the specified control settings of the wells. Li
and Jafarpour (2012) showed that a combined well placement and well control optimization
approach can significantly improve the optimization performance metric (i.e., NPV). They
implemented a decoupled approach where sequential solutions of the well placement and control
sub-problems were used until no significant improvement in the objective function was observed.
Li et al. (2013) implemented a coupled SPSA algorithm to simultaneously optimize well locations
and their control settings and could show similar outcomes. Other authors have also investigated
the joint (generalized) field development optimization problems (Forouzanfar and Reynolds, 2014;
Humphries et al, 2013; Isebor et al, 2014). In (Isebor et al., 2014), the authors use a hybrid approach
using particle swarm optimization (PSO) to solve the joint problem of well placement and control
optimization problem. Humphries et al. (2013) also use the PSO algorithm in combination with
the generalized pattern search technique to optimize the locations and control settings of the wells
simultaneously. They reported improved solutions when the joint problem is decoupled into
consecutive stages of well placement and control optimization and solved sequentially. The
authors hypothesized that this behavior could be due to judicious selection of well controls. While
theoretically the solution of the decoupled problem cannot be better than that of the joint
optimization (Shu et al., 2006), convergence to a local solution can explain the observed behavior
in (Shu et al., 2006). Forouzanfar and Reynolds (2014) present a formulation for simultaneously
optimizing the number, location and rate allocations of the wells, whereby they initially assign a
well to each cell in the reservoir domain and use a gradient-based algorithm to iteratively eliminate
non-profitable wells to increase the NPV value. In a recent study, Navabi et al. (2016) show that
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

14

all the decision variables in a generalized field development problem are collectively captured in
the source/sink terms of the related discretized governing flow equations. In their formulation, the
decision variables consisted of the source/sink terms (as continuous variables) in all grid cells. The
capital cost associated with each well was used as a penalty term to minimize the number of wells
while maximizing the overall project NPV.
Another important optimization problem is related to drilling schedule. In general,
optimization of well scheduling is performed to find the optimum sequence of drilling. Given the
considerable cost of drilling, it is important to find an optimal drilling sequence to maximize the
production performance while minimizing cost. Well scheduling optimization has not been
investigated in literature as widely as well placement and well control optimization. Beckner and
Song (1995) use simulated annealing for optimizing the net present value of a full field
development by varying the placement and sequence of production wells for Clearview field for
12 horizontal wells. Cullick et al. (2004) present an approach to optimize reservoir planning and
decision making under uncertainty. Isebor et.al (2014) present procedures to simultaneously
determine the optimal number and type of wells, as well as their drilling sequence and their
corresponding locations and controls using Particle Swarm Optimization (PSO), Mesh Adaptive
Direct Search (MADS), and PSO-MADS hybrid algorithm. In their unified formulation for
generalized field development, Navabi et al. (2016) use a discount factor associated with the capital
cost to encourage later drilling and obtain optimal drilling schedule.
An important issue in field development optimization is the significant uncertainty in the
available information, especially at the early stages of the project. In general, three main sources
of uncertainty can be defined in field development planning: geologic uncertainty, economic
uncertainty, and operation/development uncertainty. Geological uncertainty is related to
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

15

incomplete information about reservoir description, including rock property distributions such as
permeability and porosity. Geologic uncertainty has been considered in optimization studies by
several investigators, where the uncertainty is treated using probability distributions to describe
the underlying rock properties. Monte-Carlo simulation is often used to propagate this uncertainty
through the forecast models to approximate the resulting uncertainty in the adopted objective
function (e.g., estimated NPV). Economic uncertainty is also a significant contributor to field
development and project management. It often involves future oil/gas prices as one of the main
drivers. In addition to regular changes in oil prices, for example due to supply and demand
variations, unpredictable factors such as geopolitical effects can also result in significant economic
uncertainty. The latter uncertainty is difficult to predict and systematically incorporate into field
development studies. A typical approach to deal with (predictable) economic uncertainty is by
using a probabilistic economic model that incorporates a range of plausible scenarios for changes
in oil prices. Such a model can be used to describe the uncertainty in the NPV or monetary
performance metrics (Siraj et al., 2015). One source of uncertainty that has not been studied in
production optimization and reservoir development is the uncertainty in future development
activities (e.g., infill drilling).
In general, the high cost of drilling, coupled with the limited information at the early stages
of the project, results in a strategy to delay drilling decisions until more information about the
reservoir is acquired. Therefore, optimization techniques should always consider the possibility of
changes in field configuration, mainly due to the introduction of new wells. Two general
approaches can be used to include future decision (events) in optimization problems. The first
approach is to include future events as decision variables in the optimization problem. This
approach leads to a deterministic optimization, in which it is assumed that optimization results can
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

16

be implemented perfectly. However, there is always a good chance that future developments may
differ from optimized strategies. Hence, a more conservative and robust approach is to consider
the future development plans as uncertain events. In the latter case, the problem can be formulated
in a stochastic optimization framework. This approach is used to hedge against future development
plans and to avoid potential losses from disregarding such scenarios. In this work, we present the
two approaches and primarily focus on decision rules in stochastic optimization formulations
where future events are treated as random variables.

2.2 Motivating Example
In this section, a combined well placement and control optimization problem is presented
to demonstrate the significance of incorporating the possibility of future infill drilling in
optimization. In the first example, deterministic optimization is used without accounting for future
events. In this case, only the decision variables corresponding to the current time-step (today) are
optimized without accounting for the future infill drilling opportunities. This approach has also
been referred to as myopic optimization (Scott, 1971) and static optimization (Jaikumar and Bohn,
1992) where the future information is not included in the current-time decision making process. In
the second example, we consider the case of perfect information (about the drilling scenario) and
optimize current and future decision variables simultaneously at the beginning of the project. This
approach has also been called as lookahead optimization (Bellman, 1961) where the knowledge of
future information and actions are included in the optimization. The results show a remarkable
improvement in reservoir performance when the knowledge of future drilling is incorporated in
the optimization, thereby motivating the need to consider future development plans.
Our motivating examples is based on the PUNQ-S3 reservoir model, which is a 3D
synthetic reservoir model derived from real field data. The model contains 19×28×5 grid blocks,
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

17

of which 1761 blocks are active. The geological model is composed of five independent layers.
Figure 2.1 shows the horizontal permeability distribution for the five layers of the PUNQ-S3
model. Layers 1, 3 and 5 have relatively high permeability and porosity distributions whereas
Layers 2 and 4 have lower values for these properties and act as semi-permeable barriers. For this
example, the simulation is performed for 10 years and 3 injectors and 3 producers are planned to
be drilled at the first stage (first five years) while one infill producer is scheduled to be drilled after
the first five years. The reservoir model description is provided in Table 2.1.
Layer 1

Layer 4

Layer 2

Layer 5

Layer 3

Figure 2.1: Horizontal permeability map (millidarcy) for 5 layers of PUNQ-S3 model

Table 2.1: Reservoir model parameters for PUNQ-S3 model
Number of the grid cells 19×28×5
Number of active cells 1,761
Grid cell dimensions 40 ft× 40 ft× 40 ft
Initial water saturation 0.05
Oil Price $50/𝑏𝑏𝑙
Water injection and production cost $10/𝑏𝑏𝑙
Annual Discount rate 0
Fluid Phases Oil-Water
Simulated reservoir life cycle 10 years
Injectors control mode Total water injection rate
Producers control mode Total fluid production rate

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

18

2.2.1 Case I: Disregarding Future Drilling
For this case, only current stage decision variables are optimized. Therefore, at the
beginning of the project the controls and locations of the 6 wells (3 producers and 3 injectors) are
optimized without accounting for the additional infill well that is scheduled to be drilled after 5
years. The optimization is implemented over a 10-year life cycle of reservoir. The total number of
decision variables in this problem is equal to 72, of which 12 correspond to coordinates of 6 wells
and 60 represent control variables (assuming 10 control time-steps for the 10-year life cycle of the
project). After 5 years, when it is decided to drill the infill well, the optimization needs to be re-
implemented to find the optimal controls for current and new wells as wells as the location of the
new infill well. The top row of Figure 2.2(a) shows the optimal configuration of 3 producers
(PROD1, PROD2 and PROD3) and 3 injectors (INJ1, INJ2, INJ3) at the beginning of the project
without accounting for the impact of the infill production well PROD4 that is scheduled to be
drilled at year 5. In this case, for the combined well placement and well control optimization, a
sequential optimization procedure is implemented by alternating between well placement
optimization solution (SPSA) and well control optimization solution (Quasi-Newton) until
marginal improvement in the NPV is achieved (Li and Jafarpour, 2012). The simulator used in this
work is the Matlab Reservoir Simulation Toolbox (MRST), which is developed by the
Computational Geosciences group in the Department of Applied Mathematics at SINTEF ICT
(MRST, 2016). The second row in Figure 2.2(a) show the oil saturation map after 5 years of
production with the current 3 producers and 3 injectors. As can be seen in this figure, the region
on the right side of the reservoir has not been swept entirely. After 5 years of production, an infill
well must be drilled. However, because this plan was not known at the beginning, the new well
was not included in the optimization. At year 5, a new optimization procedure is implemented to
find the optimal location of the infill well as well as the control settings for all the wells (4
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

19

producers and 3 injectors) for the remaining five years of the project. In this case, the number of
decision variables is 37 where 2 of them is for the location of the infill well (𝑥 and 𝑦 ) and 35 of
them are control variables for the 7 wells (3 initial producers, 3 initial injectors, and the additional
infill production well) for the remaining 5 control timesteps. As shown in the top row of Figure
2.2(b), the new production well PROD4 is placed in the right corner of the unswept region of the
reservoir away from the injectors. The figure on the bottom of Figure 2.2(b) show the final oil
saturation map after 10 years of production (in this case, the knowledge of future drilling was not
included in the optimization scheme). Figure 2.3 displays the control trajectory of the wells for
the entire project life. The solid lines show the initial optimal trajectories while the dashed lines
show the optimal control trajectories for the second 5 years of project, after the infill well is
introduced. Once the new infill well (PROD4) is placed in the right corner of the reservoir, the
injection rate of INJ2, which supports the new production well, is decreased to reduce the water
production in new infill well PROD4. This reduction is accompanied by increases in injectors INJ1
and INJ3.
Perm. map

Sat. map

(a) (b)

Figure 2.2: a) top: Optimal configuration for 3 producers and 3 injectors without accounting for the future drilling
opportunity; bottom: Oil saturation map after 5 years of production corresponding to optimal configuration of the top
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

20

figure b) top: Optimal location for the infill well (PROD4) obtained by re-implementing the optimization algorithm;
bottom: Oil saturation map after a total of 10 years of production.

Figure 2.3: Optimal well control trajectories for the case where the knowledge of future infill drilling is disregarded;
solid lines are the control trajectories for the 3 producers and 3 injectors obtained at the beginning of the project
without accounting for the future infill well, dashed lines are the optimal control trajectories for the second 5 years of
the project when it is decided to drill an infill well (PROD4).

2.2.2 Case II: Incorporating Future Infill Drilling
We now consider a different approach in which both today’s and future decision variables
are optimized simultaneously. In this case, it is assumed that the exact number of infill wells are
known a-priori (the Perfect Information case) and the optimization scheme is only implemented
once at the beginning of the project with the knowledge that an infill well will be drilled after 5
years. Although the Perfect Information case may not be realistic, this case is considered for
comparison purposes and later in the work we will relax this assumption by representing the future
drilling plans as uncertain parameters. The top row of Figure 2.4 shows the optimal configuration
achieved in this case. Note that the final configuration of the two approaches (Figure 2.2(b) &
Figure 2.4), are different. Second row in Figure 2.4(a) and Figure 2.4(b) depict the saturation
maps after 5 and 10 years, respectively. Figure 2.5 demonstrates the optimal control trajectories
for all the wells. In this case, the wells are placed and operated to allow for optimizing the location
and control trajectory of PROD4 in the last 5 years of the project (compared to Figure 2.3). As
seen from the saturation plots in Figure 2.4(a), the optimization solution anticipates the infill well
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

21

for the second stage of the simulation and creates a drilling potential zone on the top part of the
reservoir, where PROD4 is drilled. Table 2.2 compares the NPV of the two cases after 10 years.
By including the knowledge of future infill drilling, a 14% increase in NPV is achieved. This
increase is achieved because of 8% increase in oil production, 8% decrease in water production
and 3% decrease in water injection. Because for both cases the number of drilled wells is the same,
the NPV is only affected by the operating costs.
Permeability Map

Sat.

(a) (b)

Figure 2.4: a) top: Optimal configuration for 4 producers and 3 injectors in Perfect Information problem; bottom: Oil
saturation map after 5 years of production corresponding to optimal configuration of the top figure b) top: Optimal
configuration for 4 producers and 3 injectors in Perfect Information problem; bottom: Oil saturation map after a total
of 10 years of production.

Figure 2.5: Optimal control trajectory of 4 producers and 3 injectors for Perfect Information case where the
knowledge of future infill drilling (PROD4 after 5 years) is incorporated in the optimization scheme upfront.

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

22

Table 2.2: NPV, water injection, oil production and water production results of the case where the knowledge of
future drilling is not incorporated in the optimization scheme from the beginning and Perfect Information case
(knowledge of future drilling is accounted for from the start of the project) for one infill well

NPV
($ MM)
Water Injected
(MMSTB)
Oil Produced
(MMSTB)
Water Produced (MMSTB)
Deterministic: First 5 years 60.01 1.76 1.59 0.18
Deterministic: Second 5 years 13.1 2.02 0.89 1.12
Deterministic: total 10 years 73.2 3.78 2.48 1.30
Perfect Information: total 10 years 83.11 3.88 2.68 1.20

In the above example, the infill well was drilled after 5 years. Next, we investigate the
impact of drilling time on the performance of the two cases. Figure 2.6 shows the NPV comparison
between the two methods when the infill well is drilled at different times (Year 1 to Year 9). The
results show that the sooner the infill well is drilled, the higher will be the NPV for the case where
future drilling knowledge is neglected. This is intuitive as when the optimization scheme is not
aware of possible future drilling, earlier drilling times tend to decrease the loss due to the initial
suboptimal decisions. In the Perfect Information case, the NPV is also decreased with time,
suggesting that the earlier the well is drilled, the more beneficial its impact will be.

Figure 2.6: NPV comparison of the optimization approach where the knowledge of future infill drillings is
disregarded and Perfect Information optimization (knowledge of future drilling is accounted upfront) in field
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

23

development optimization scheme for different years of drilling one single infill well.

2.3 Problem Statement
In the previous section, exactly one infill well with a known drilling time was considered
to highlight the importance of including future drilling in the optimization problem. However, in
real applications the number of infill wells and their drilling times are typically unknown or
uncertain. The infill drilling can generally be modelled in two ways: by treating them as decision
variables, in which case the number of infill wells and their drilling schedule must be optimized,
or by modeling them as uncertain events that can be described using random variables, which leads
to stochastic optimization. The former approach assumes that the optimized strategies will be
implemented exactly. In the latter case, however, at each development stage, the decision variables
(e.g., location, control, etc.) are optimized while accounting for the residual uncertainty in future
development plans. The perfect knowledge implementation assumption is not realistic as
unforeseen future constraints (e.g., economic, physical, and regulatory) can change the initial
decisions. Furthermore, reservoir models are routinely updated with incoming dynamic data,
which can lead to changes in future predictions and development strategies. The second approach
assumes that not all the decisions regarding infill drilling can be made from the beginning. Some
decisions will be made at later stages when more information becomes available to resolve the
existing uncertainties. In this work, we assume that the future development events are complex
and depend on several factors, not all of which can be modelled (either as a function of reservoir
states/parameters or external factors such as oil price or availability of drilling rigs). We also note
that while geologic uncertainty almost always results in future changes to the model, our focus in
this chapter is related to the uncertainty in future decisions, without including the complexity
associated with geologic uncertainty.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

24

In the next section, we formulate the problem for both deterministic (where future decisions
are modelled as decision variables) and stochastic (where future decisions are modelled as random
variables) approaches, followed by numerical examples for each approach.
2.4 Optimization Problem Formulation
2.4.1 Deterministic Field Development Optimization
In this approach, the infill drilling opportunities are modeled as decision variables and their
optimal locations, control, and drilling times are determined from the beginning. If we denote the
decision vector by 𝑥 , the optimization problem can be formulated as follows:
𝑥̂=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 ∈𝒳 𝑓 (𝑥 )
(2.1)
𝑠 .𝑡 . 𝑔 (𝑥 ,𝑚 )=0 ; ℎ(𝑥 ,𝜔 )≤0

where the decision vector 𝑥 is defined over a set of feasible well locations, operating
controls, and drilling sequence; that is, 𝑥 =[𝑞 ,𝑢 ,𝑡 ]
𝑇 and 𝑞 (𝑢 ,𝑡 )∈Θ
𝑞 , 𝑢 ∈Θ
𝑢 ,𝑡 ∈Θ
𝑡 , where
Θ
𝑞 ,Θ
𝑢 ,Θ
𝑡 are the feasible sets for well operating controls, locations, and drilling times,
respectively. The nonlinear constraints 𝑔 (𝑥 ,𝑚 ) and ℎ(𝑥 ,𝜔 ) represent the conservation equations
and inequality constraints, respectively. Because the conservation constraint is automatically
satisfied by solving the reservoir simulation equations, for compactness, we drop this constraint
throughout our formulation. The notation 𝑚 is used to represent input reservoir parameters, which
will be treated as known inputs in this chapter. In this study, we adopt the objective function 𝑓 (∙)
to be the negative of the reservoir NPV:
𝑓 (𝑥 )=−𝑁𝑃𝑉
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

25

=−∑[∑
(𝑄 𝑜 ,𝑗 𝑘 𝑟 𝑜 −𝑄 𝑤 ,𝑗 𝑘 𝑐 𝑤𝑝
).∆𝑡 𝑘 (1+𝑏 )
τ
𝑘 365
𝑁 𝑃 𝑗 =1
−∑
𝑄 𝑤 ,𝑖 𝑘 𝑐 𝑤𝑖
∆𝑡 𝑘 (1+𝑏 )
τ
𝑘 365
𝑁 𝐼 𝑖 =1
]
𝐾 𝑘 =1
+∑
𝐶 𝑗 (1+𝑏 )
τ
𝑗 365
+
𝑁 𝑃 𝑗 =1
∑
𝐶 𝑖 (1+𝑏 )
τ
𝑖 365
𝑁 𝐼 𝑖 =1

(2.2)
where, 𝑄 is the fluid flow rate, 𝑐 denotes the cost associated with water injection and water
production (depending on the subscript), 𝑟 𝑜 is the price of oil per unit volume, 𝐶 represents the
capital cost associated with drilling of injectors and producers, 𝑏 is the annual interest rate, ∆𝑡 𝑘 is
the 𝑘 𝑡 ℎ
(time) step-size, 𝜏 𝑘 is total time (in days) passed since the start of project till time-step 𝑘 .
𝜏 𝑗 and 𝜏 𝑖 are the times (in days) that producer 𝑗 and injector 𝑖 are drilled. 𝑁 𝑝 and 𝑁 𝐼 denote the
number of producers and injectors, respectively.
In practice, however, the exact distribution of reservoir properties (such as permeability
and porosity) can be highly uncertain. A practical approach to account for geologic uncertainty in
predicting the reservoir flow behavior and response is to use several plausible realizations of
uncertain model parameters. Under ensemble-based reservoir modeling and production
forecasting, one can define the corresponding optimization objective function in terms of the point
statistics of the NPV function. A simple choice is to use the expected value of the NPV over the
existing models, that is:
𝑓 𝑅𝑜𝑏𝑢𝑠𝑡 (𝑥 )=𝐸 𝑚 [𝑓 (𝑥 )]=
1
𝑁 𝑒𝑛𝑠 ∑𝑓 𝑖 (𝑥 )
𝑁 𝑒𝑛𝑠 𝑖 =1

(2.3)
where 𝐸 𝑚 [·] denotes the statistical expectation operator over the model parameters and
𝑁 𝑒𝑛𝑠 denotes the number of model realizations (ensemble size) used to calculate the objective
function.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

26

We note that in stage-wise optimization problems that are discussed in subsequent section
the function 𝑓 (∙) consists of NPV contributions from different stages and is referred to as 𝑓 (∙)=
𝑓 1
(∙)+𝑓 2
(∙)+⋯. In the deterministic approach, the decision variables for all the wells (including
the infill wells) are optimized at the beginning of the project, with the hope that the information
regarding the reservoir model is perfect and the operator will not have any constraint in carrying
out the exact proposed optimized development plan. Therefore, the infill wells, their corresponding
locations, control setting, and time of drilling are also optimized upfront.

2.4.2 Stochastic Field Development Optimization
In reservoir development where time and uncertainty play an important role, the decision
model can be designed to allow the decision maker to adopt a decision that can respond to
observations as they unfold. Therefore, the decision maker does not have to make all the decision
at once. Instead, decisions can be delayed until the existing uncertainties are resolved by additional
data and information. However, in anticipation of changes in reservoir configurations due to
development plans, the decision maker must consider possible future events with their associated
uncertainties. In this chapter, we merely focus on uncertainty in future drilling events (time-
dependent uncertainty) where a decision has to be made at the present time under uncertainty in
future decisions, while the uncertain parameter(s) gradually resolve over time as more information
is collected and the decision maker is allowed to take corrective or recourse action. For
optimization under uncertainty different approaches can be taken. One approach is to eliminate the
uncertainty and solve the problem deterministically by disregarding the uncertainty in future
drilling events. In this approach, one may argue that field development optimization problems are
difficult enough to solve without getting entangled in modeling the uncertain parameters. For this
case, the problem can be formulated as following:
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

27

𝑥̂
1
=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 1
∈𝒳 1
𝑓 (𝑥 1
)

(2.4) (5)

𝑠 .𝑡 . ℎ(𝑥 1
)≤0

where 𝑥 1
denotes the decision variables in the present time. Disregarding the uncertainty
in future decisions significantly reduces the computational complexity of the problem. In the
motivating example we observed that disregarding the possibility of drilling a single infill well
could lead to a potentially large decrease in the final profit (compared to the case where a-priori
knowledge about the exact number of infill wells is available).
In another approach, which is known as Expected Value Problem (EVP), the uncertain
parameter (𝜔 ) is replaced with its expected value (𝜔̅) to obtain a deterministic problem. Therefore,
the problem is formulated as:

𝑥̂(𝜔̅)=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 ∈𝒳 𝑓 (𝑥 ,𝜔̅)

(2.5) (6)

𝑠 .𝑡 . 𝑔 (𝑥 (𝜔̅))=0 ; ℎ(𝑥 (𝜔̅))≤0

where 𝑥 is the decision variable and 𝜔̅ is expected value of the random variable over all
the scenarios and 𝑓 (∘) is the objective function which is a function of decision vector 𝑥 and
expected value of random variable 𝜔 . This approach assumes the knowledge of future event 𝜔̅.
However, there is no guarantee that 𝑥̂(𝜔̅) is an optimal solution in the face of uncertain future
events. Another way to keep the problem deterministic is to take a Wait-and-See approach. In this
approach, which is also known as Perfect Information problem, we do not take any action unless
the uncertainty is resolved (realization prior to action). However, in field development
optimization the perfect information may not be available at any price and a decision (𝑥 ) must be
made before the realization of uncertain parameter (𝜔 ).
The next approach for incorporating the uncertainty in optimization is called the stochastic
(here-and-now) approach where the objective function and constraints can be expressed in terms
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

28

of some probabilistic representation (e.g. expected value, variance and fractiles). For example, in
chance constrained programming, the objective function is expressed in terms of expected value,
while the constraints are expressed in terms of fractiles (probability of constraint violation)
(Diwekar, 2008). In chance-constrained stochastic programming the constraints are interpreted
probabilistically, and inequality constraints can be violated with a pre-defined probability (𝛼 ). In
the “here-and-now” (stochastic) problem where the decisions are denoted as 𝑥 and uncertain
parameters as 𝜔 , the problem can be written as:

𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝐽 =𝑃 1
(𝑓 (𝑥 ,𝜔 ))
𝑠 .𝑡 𝑃 2
(ℎ(𝑥 ,𝜔 )≤0)≥1−𝛼
(2.6) (5)

where 𝑃 represents the cumulative distribution functional such as the expected value,
mode, variance or fractiles, ℎ(𝑥 ,𝑚 ,𝜔 )≤0 refers to inequality constraints such as maximum water
production or minimum inter-well distances 𝛼 represents the probability of constraint violation.
Unlike the previous deterministic approaches, in stochastic optimization one considers
probabilistic functional of the objective function and constraints. The goal in stochastic
programming is to find some policy that is feasible for all possible future outcomes by optimizing
a probabilistic measure of the objective function. A simple measure is to choose the expected value
of the random function as optimization objective function (for brevity, the constraints are
dropped):

𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝐽 ≔𝐸 𝜔
{𝑓 (𝑥 ,𝜔 )}
(2.7) (6)
The most widely applied stochastic programming models are two-stage programs where
the decision-maker needs to make some decisions under the uncertainty of future decisions at the
first stage; after the revelation of the uncertain parameter in the second stage, a corrective or
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

29

recourse action is taken to compensate for the consequences of the first stage decisions. The two-
stage recourse problem is formulated as:
𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 1
,𝑥 2
𝑓 1
(𝑥 1
)+𝐸 𝜔
{𝑓 2
(𝑥 2
,𝜔 ) }
𝑠 .𝑡 . 𝑥 1
∈𝒳 1
, 𝑥 2
∈𝒳 2
(𝑥 1
,𝜔 )
(2.8)
where 𝑓 1
(∘) is the objective function of the first stage that depends on first stage decision
variable 𝑥 1
and 𝑓 2
(∘) is the objective function of the second stage that depends on second stage
decision variable 𝑥 2
and the parameter 𝜔 .The variable 𝑥 1
(first stage decisions) needs to be
determined prior to the realization of the uncertain parameter 𝜔 , whearas variable 𝑥 2
(second stage
decisions (recourse decisions)) can be made after the disclosure of the uncertainty. 𝒳 1
is the
feasible set for first stage decision variables and 𝒳 2
(𝑥 1
,𝜔 ) is the feasible set for the decision
variables at stage 2 (which depends on stage 1 decisions (𝑥 1
) and the random outcome at second
stage (𝜔 ).
For a discrete uncertain parameter 𝜔 with a finite support Ω={𝜔 1
,𝜔 2
,…,𝜔 𝑆 }, the
problem in (2.8) can be restated as:

𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 1
,𝑥 2
𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑓 2
(𝑥 2
,𝜔 𝑠 )
𝑆 𝑠 =1

𝑠 .𝑡 . 𝑥 1
∈𝒳 1
, 𝑥 2
∈𝒳 2
(𝑥 1
,𝜔 )
(2.9)
where each realization (scenario) 𝑠 =1,…,𝑆 has an associated probability 𝑝 𝑠 . However,
in real field development problems decisions should be made sequentially at certain periods of
time (𝑥 1
,𝑥 2
,..,𝑥 𝑇 ) based on information available at each period. Therefore, the uncertainty in
development plans is resolved at different stages (𝜔 2
,𝜔 3
,…,𝜔 𝑇 ) and the problem can be posed in
a multi-stage stochastic programming framework. Multi-stage stochastic programming is applied
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

30

to address multi-period optimization problems with dynamic stochastic data during the project life.
In its generic form, a T-stage stochastic problem can be written as (Shapiro, 2011):

𝑚𝑖𝑛𝑖𝑚𝑖 𝑧 𝑒 𝑥 1
,𝑥 2
,…,𝑥 𝑇 𝐸 𝜔 [2:𝑇 ]
[ 𝑓 1
(𝑥 1
)+𝑓 2
(𝑥 2
(𝜔 2
),𝜔 2
)+⋯
+𝑓 𝑇 (𝑥 𝑇 (𝜔 [2:𝑇 ]
),𝜔 𝑇 )]

(2.10)

𝑠 .𝑡 . 𝑥 1
∈𝒳 1
, 𝑥 𝑡 ∈𝒳 𝑡 (𝑥 [1:𝑡 −1]
,𝜔 [2:𝑇 ]
), 𝑡 =2,…,𝑇
where 𝜔 2
,𝜔 3
,…,𝜔 𝑇 are the random events, 𝑥 𝑡 ∈ℝ
𝑛 𝑡 ,𝑡 =1,…,𝑇 are the decision
variables and 𝑓 𝑡 (𝑥 𝑡 (𝜔 [2:𝑡 ]
),𝜔 𝑡 ) is the cost function for stage 𝑡 , which depends on the past
decisions and the random variables assigned to the current stage. 𝒳 𝑡 (𝑥 [1:𝑡 −1]
,𝜔 [2:𝑡 ]
) is the feasible
set for the decision variables at stage 𝑡 (which depends on all decisions and random outcomes
between stages 1 to 𝑡 ). Note the first stage function 𝑓 1
:ℝ
𝑛 1
→ℝ is deterministic. Also, the notation
𝜔 [2:𝑇 ]
refers to the joint distribution of the uncertain events throughout the [2:𝑇 ] time intervals.
Equation (2.10) can be reformulated using a nested expectation (see Appendix B):
𝑚𝑖𝑛 𝑥 1
𝜖 𝜒 1
𝑓 1
(𝑥 1
)+𝐸 𝜔 2
[ 𝑚𝑖𝑛 𝑥 2
𝜖 𝒳 2
(𝑥 1
,𝜔 2
)
𝑓 2
(𝑥 2
,𝜔 2
)
+𝐸 𝜔 3
|𝜔 2
[…+𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[ 𝑚𝑖𝑛 𝑥 𝑇 𝜖 𝒳 𝑇 (𝑥 [1:𝑇 −1]
,𝜔 [2:𝑇 ]
)
𝑓 𝑇 (𝑥 𝑇 ,𝜔 𝑇 )]]]

(2.11) (
8
)
Where𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 ) is the cost function for stage 𝑡 (which depends on the decision variable
and random outcome for that stage).
The presence of a nested collection of conditional expectations poses serious computational
challenges for multi-stage optimization. However, if the stochastic process 𝜔 1
,𝜔 2
,…,𝜔 𝑇 is stage-
wise independent (meaning that the random variable 𝜔 𝑡 +1
is independent of 𝜔 [1:𝑡 ]
=(𝜔 1
,…,𝜔 𝑡 )),
the joint probability of the stochastic process can be specified as the multiplication of the marginal
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

31

distributions and the conditional expectations are simplified to unconditional expectations of the
form:
𝑚𝑖𝑛 𝑥 1
𝜖 𝜒 1
𝑓 1
(𝑥 1
)+𝐸 𝜔 2
[ 𝑚𝑖𝑛 𝑥 2
𝜖 𝒳 2
(𝑥 1
,𝜔 2
)
𝑓 2
(𝑥 2
,𝜔 2
)]+…+𝐸 𝜔 𝑇 [ 𝑚𝑖𝑛 𝑥 𝑇 𝜖 𝒳 𝑇 (𝑥 [1:𝑇 −1]
,𝜔 [2:𝑇 ]
)
𝑓 𝑇 (𝑥 𝑇 ,𝜔 𝑇 )]

(2.12)

which can be compactly expressed as:
𝑚𝑖𝑛 𝒙 {𝑓 1
(𝑥 1
)+∑𝐸 𝜔 𝑡 [𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 )]
𝑇 𝑡 =2
}

(2.13)

We note that the number of infill wells at each stage may depend on the number of infill
wells in other stages. In that case, the decision tree will show dependence between the number of
wells at different stages of each scenario. However, the exact form of the dependence among the
drilling decision will be problem-specific.
The solution of the stochastic programming problem with recourse involves a process in
which the decisions alternate with observations (Table 2.3). The initial stage of the process is
focused on the choice of 𝑥 1
. This is followed by recourse stages, each consisting of an observation
(of the realization of the random variable) and a decision that needs to be made in response to that
observation. That is, if 𝜔 𝑡 is observed before taking a decision 𝑥 𝑡 , we can adapt 𝑥 𝑡 to the realization
of 𝜔 𝑡 .
Table 2.3: Decision stages and the order of observations and decisions in multi-stage stochastic optimization with
recourse
Horizon Available information for taking decisions Decision
Prior decisions Observed outcome Residual uncertainty
1 none none 𝜔 2
,…,𝜔 𝑇 𝑥 1

2 𝑥 1
𝜔 2
𝜔 3
,…,𝜔 𝑇 𝑥 2

3 𝑥 1
,𝑥 2
𝜔 2
,𝜔 3
𝜔 4
,...,𝜔 𝑇 𝑥 3

⋮
T 𝑥 1
,…,𝑥 𝑇 −1
𝜔 1
,𝜔 2
,…,𝜔 𝑇 none 𝑥 𝑇

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

32

For a discrete random process (𝜔 ), with finite number of scenarios Ω={𝜔 1
,𝜔 2
,…,𝜔 𝑆 },
the problem in (2.13) can be restated as:
𝑚𝑖𝑛 𝒙 {𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 𝑠 )
𝑇 𝑡 =2
}

(2.14)
To summarize, in the “stochastic” (here-and-now) approach, the uncertainty in future
drilling events is modeled with random variables. As the uncertain variable, i.e., number of wells
to drill, is realized (based on new information from the field and other external factors), future
decisions can be adapted according to the realized uncertain parameters and the information
available about the current state of the field. This formulation leads to a sequential decision-making
problem. In field development optimization, at early stages both the number of infill wells (𝜔 )
and their respective locations (𝑢 ) are typically uncertain. We denote the vector of uncertain
variables as 𝜃 , where 𝜃 =[𝜔 ,𝑢 ]
𝑇 and write the multi-stage stochastic programming as:
min
𝑥 1
𝜖 𝜒 1
𝑓 1
(𝑥 1
)+𝐸 𝜃 2
[ min
𝑥 2
𝜖 𝒳 2
(𝑥 1
,𝜃 2
)
𝑓 2
(𝑥 2
,𝜃 2
)
+𝐸 𝜃 3
|𝜃 2
[…+𝐸 𝜃 𝑇 |𝜃 2
,…,𝜃 𝑇 −1
[ min
𝑥 𝑇 𝜖 𝒳 𝑇 (𝑥 [1:𝑇 −1]
,𝜃 [2:𝑇 ]
)
𝑓 𝑇 (𝑥 𝑇 ,𝜃 𝑇 )]]]

(2.15)
Assuming stage-wise independent drilling decisions, the uncertainty in the number of
future infill wells (𝜔 [𝑡 +1:𝑇 ]
) can be formulated as a sequential decision-making problem with
unconditional expectations in multi-stage formulation. On the other hand, characterizing the
uncertainty in future well locations is more complex because the number of possible scenarios can
be extremely large, and the well locations are stage-wise dependent (the location of the wells at
stage 𝑡 depends on the well locations decided at previous stages). Therefore, incorporating the
uncertainty in future infill well locations (𝑢 [𝑡 +1:𝑇 ]
) is more involved.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

33

Potential well locations are assigned to future infill wells using a similar approach that has
been used in supply network design problems, where the goal is to select a subset of potential
facilities and assign each customer to a selected facility, such that the total cost of serving all
customers is minimum (Shmoys et al., 1997; Vyjen, 2005; Snyder, 2006). For allocating potential
locations to future infill wells, a subset of possible well site combinations is generated and then
those with the highest value of the desired productivity metric (e.g., NPV) are selected. The subset
of possible well site combinations can be generated based on reservoir quality maps and a set of
constraints such as minimum interwell spacing or other well configuration constraints. However,
unlike supply network design problems where the potential locations are fixed, the potential
locations for future wells must be updated at each iteration as for each iterate (proposed current
well locations) the plausible/promising future well locations are different (and are needed to
evaluate the corresponding realizable NPV values). In optimization with recourse, the updated
future locations are not fixed and can be corrected at each stage using new information and past
decisions. Accounting for the uncertainty in well locations can lead to extremely large number of
scenarios, in which the advantage of stochastic programming can be greatly compromised. To keep
the number of scenarios at an acceptable level, at the end of each development stage, based on a
quality index (
∑ 𝐾 𝑙 ℎ
𝑙 𝜑 𝑙 𝑆 𝑜𝑙
𝑛 𝑙 =1
𝑛 ), where 𝐾 𝑙 ,ℎ
𝑙 ,𝜙 𝑙 and 𝑆 𝑜𝑙
are the permeability, thickness, porosity and
oil saturation of layer 𝑙 , respectively and 𝑛 is total number of reservoir layers in vertical direction, a
subset of potential well locations can be selected. While porosity and permeability are static
reservoir properties, saturation is a dynamic quantity and undergoes drastic changes during the
project life, making the quality map time-variant. For multi-stage field development, once a well
is placed in a gridblock with relatively high-quality index value, the entire quality map for the
subsequent stages will change. Therefore, at each iteration, when the locations of the wells at the
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

34

current stage are updated, a procedure is used to automatedly generate multiple plausible future
well location samples for the remaining stages of the development.
Based on the above discussion and depending on the type of uncertainty and decision
variables in field development optimization, two variants of Equation (2.15) can be formulated. In
Variant I, the uncertain variables include only the number of infill drilling (𝜔 [𝑡 +1:𝑇 ]
) and the
uncertainty in their locations (𝑢 [𝑡 +1:𝑇 ]
) is removed by optimizing future infill wells locations. In
Variant II, both the number and locations of infill drilling for the remaining horizons
(𝜔 [𝑡 +1:𝑇 ]
,𝑢 [𝑡 +1:𝑇 ]
) remain uncertain. Equation (2.12) and its simplified form (Equation (2.13)) are
the formulations for variant I while Equation (2.15) represents the formulation for Variant II. In
this chapter, we focus on the first variant of the multi-stage well placement problem, in which
potential future well locations are optimized for each iteration of the current well locations.
Treatment of Variant II is currently under investigation by the authors. Because the uncertain
parameter (number of infill wells) have discrete and finite distribution and the planning horizon
consists of a fixed number of time periods that correspond to decision points, the stochastic process
can be represented with a finite scenario tree. Equation (2.14) shows the stochastic formulation for
a discrete uncertain parameter (𝜔 ), where the expectation form of Equation (2.13) is written as
probability-weighted average of all possible scenarios.
We can further augment the decision vector by adding the number of wells for each stage.
The resulting objective function takes the form:
𝑚𝑖𝑛 𝑥 𝑡 𝜖 𝒳 𝑡 ,𝑥 𝑤 ∈{0,1}
{𝑓 𝑡 𝑂𝑃𝐸𝑋 (𝑥 𝑡 ,𝜔 𝑡 )+∑𝑝 𝑠 𝑆 𝑠 =1
∑ 𝑓 𝑘 𝑂𝑃𝐸𝑋 (𝑥 𝑘 ,𝜔 𝑘 𝑠 )+∑𝐶 𝑤 𝑡 𝑥 𝑤 𝑡 𝑊 𝑤 =1
𝑇 𝑘 =𝑡 +1
}

(2.16)
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

35

where W is the maximum number of infill wells that can be drilled at each stage, 𝐶 𝑤 𝑡 is the
discounted cost (capital cost) of drilling well 𝑤 in horizon 𝑡 and 𝑓 𝑡 𝑂𝑃𝐸𝑋 is the operational cost at
stage t. The binary decision variable 𝑥 𝑤 𝑡 can be defined as:

𝑥 𝑤 𝑡 ={
1 if well 𝑤 is drilled in horizon t
0 otherwise

(2.17)
Generalization of the formulation to consider geologic uncertainty in a closed-loop format
with dynamic model updating is straight forward. In that case, and the objective function is
expressed as:
𝑚𝑖𝑛 𝒙 𝐸 𝑚 𝑡 {𝑓 𝑡 (𝑥 𝑡 )+ ∑ 𝐸 𝜔 𝑘 [𝑓 𝑘 (𝑥 𝑘 ,𝜔 𝑘 )]
𝑇 𝑘 =𝑡 +1
} (2.18)
where 𝑚 represents the model input parameters (e.g. petrophysical properties). A common
approach to account for geologic uncertainty in predicting the reservoir fluid flow
behavior/response is to use an ensemble of probable model realizations and optimize an objective
function consisting of some point statistics (e.g., expected value) of the desired performance metric
(e.g., NPV). Here, 𝑚 𝑡 denotes the latest updated model at stage 𝑡 of reservoir development which
is achieved because of model updating based on hard data (if any) and dynamic production data
available till stage 𝑡 . The optimization procedure is repeated after each model updating step. To
account for oil price uncertainty, a similar approach can be taken where the objective function is
optimized over multiple plausible scenarios of the economic model.
2.5 Solution Approach
2.5.1 Deterministic Field Development Optimization
In deterministic field development approach, the number and schedule of the infill wells
are considered as decision variables. In this work, we use a sequential approach to solve for the
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

36

complete deterministic field development planning, where the number, locations, control
trajectories and drilling schedule (𝑥 =[𝑞 ,𝑢 ,𝑡 ]
𝑇 ) of the wells are all optimized at once. We adopt
a sequential well location and scheduling optimization to solve the problem. The framework is
initialized with well placement optimization using fixed control setting and schedule. Once
converged, the well locations are used to start the well scheduling optimization solution. The cycle
of well placement and drilling time optimizations is repeated until no improvement in the objective
function is observed.
The well placement optimization part of the general problem in Equation (2.1) is obtained
by fixing well controls and drilling schedules:

𝑢̂=𝑎𝑟𝑔𝑚𝑖 𝑛 𝑢 ∈𝑍 𝑓 (𝑞 0
,𝑢 ,𝑡 0
)
𝑠 .𝑡 . 𝑢 ∈𝛩 𝑢
(2.19)

In this work, we implement the discrete version of the SPSA algorithm (see Appendix A)
for well placement optimization (Bangerth et al., 2006, Li and Jafarpour 2012). Table 2.4 shows
the pseudo-code used to implement the integer SPSA in this work. The well scheduling
optimization is solved by fixing well controls and locations:
𝑡 ̂
=𝑎𝑟𝑔𝑚𝑖 𝑛 𝑡 ∈𝑍 𝑓 (𝑞 0
, 𝑢 0
,𝑡 )
𝑠 .𝑡 . 𝑡 ∈𝛩 𝑡
(2.20)
Table 2.4: Pseudo-code for integer SPSA
𝐼𝑛𝑖𝑡𝑖𝑎𝑙𝑖𝑧𝑎𝑡𝑖𝑜𝑛 𝑓𝑜𝑟 𝑖 =1→ 𝑢 1
,𝛾 =0.101,𝛼 =0.602
𝑤 ℎ𝑖𝑙𝑒 𝑖 ≤𝑖 𝑚𝑎𝑥 𝑜𝑟 𝑡 ℎ𝑒 𝑐𝑜𝑛𝑣𝑒𝑟𝑔𝑒𝑛𝑐𝑒 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑎 𝑛𝑜𝑡 𝑠𝑎𝑡𝑖𝑠𝑓𝑖𝑒𝑑 𝑑𝑜
𝐶 ℎ𝑜𝑜𝑠𝑒 𝑎 𝑟𝑎𝑛𝑑𝑜𝑚 𝑠𝑒𝑎𝑟𝑐 ℎ 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛 ∆
𝑖 ∈{−1,+1}
𝐶𝑎𝑙𝑐𝑢𝑙𝑎𝑡𝑒 𝑐 𝑖 =⌈
𝑐 𝑖 𝛾 ⌉,𝑎 𝑖 =
𝑎 𝑖 𝛼
𝐸𝑣𝑎𝑙𝑢𝑎𝑡𝑒 𝑓 +
=𝑓 (∏(𝑢 𝑖 +𝑐 𝑖 ∆
𝑖 ))𝑎𝑛𝑑 𝑓 −
=𝑓 (∏(𝑢 𝑡 𝑖 −𝑐 𝑖 ∆
𝑖 ))
𝐶𝑜𝑚𝑝𝑢𝑡𝑒 𝑠𝑡𝑜𝑐 ℎ𝑎𝑠𝑡𝑖𝑐 𝑔𝑟𝑎𝑑𝑖𝑒𝑛 𝑡 𝑔 𝑖 =
𝑓 +
−𝑓 −
|∏(𝑢 𝑖 +𝑐 𝑖 ∆
𝑖 )−∏(𝑢 𝑖 −𝑐 𝑖 ∆
𝑖 )|

𝑢 𝑖 +1
=∏(𝑢 𝑖 −⌈𝑎 𝑖 𝑔 𝑖 ⌉∆
𝑖 )
𝑖 ←𝑖 +1
𝑒𝑛𝑑 (𝑤 ℎ𝑖𝑙𝑒 )

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

37

The well schedule optimization is implemented over the same K intervals used as control
timesteps, implying that the feasible set 𝛩 𝑡 is finite and limited to the number of time intervals. In
this work, we also implement the SPSA algorithm for well schedule optimization.
The well control optimization part of the general problem in Equation (2.1) is obtained by
fixing well locations and drilling schedules:

𝑞̂=𝑎𝑟𝑔𝑚𝑖 𝑛 𝑞 ∈𝑅 𝑓 (𝑞 ,𝑢 0
,𝑡 0
)
𝑠 .𝑡 . 𝑞 ∈𝛩 𝑞
(2.21)
A gradient-based method of the form
𝑞 𝑘 +1
=𝑞 𝑘 −𝛼 𝑘 𝐵 𝑘 −1
𝛻 𝑞 𝑓 (𝑞 𝑘 ,𝑢 0
,𝑡 0
)
(2.22)
can be used to minimize the objective function in Eq. (2.21). In this work, we adopt the
BFGS quasi-Newton method and compute the gradients using an efficient adjoint model. In our
examples, we have used Matlab optimization toolbox to solve for the nonlinear optimization
problem, where the default built-in functions are used to adaptively select step length (𝛼 𝑘 ) for
each iteration while satisfying the Wolfe conditions. The notation 𝑩 𝒌 −𝟏 refers to the approximate
inverse Hessian calculated as:

𝐵 𝑘 +1
=𝐵 𝑘 +
𝑦 𝑘 𝑦 𝑘 𝑇 𝑦 𝑘 𝑇 𝑠 𝑘 −
𝐵 𝑘 𝑠 𝑘 𝑠 𝑘 𝑇 𝐵 𝑘 𝑇 𝑠 𝑘 𝑇 𝐵 𝑘 𝑠 𝑘

(2.23)
where
𝑠 𝑘 =𝑞 𝑘 +1
−𝑞 𝑘 ,𝑦 𝑘 =∇𝑓 (𝑞 𝑘 +1
)−∇𝑓 (𝑞 𝑘 )
2.5.2 Stochastic (Here-and-now) Field Development Optimization
In stochastic (here-and-now) field development, for the current optimization stage, future
infill drillings are modeled as random variables. The decision maker can wait for more information
to become available before deciding at later stages. In this case, the reservoir life is divided into a
finite number of horizons (stages). At the beginning of each horizon, a decision regarding the
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

38

number of infill wells at that stage is made and the optimization scheme is implemented to find
optimal locations and operational control settings while accounting for the uncertainty in drilling
activities for the remaining horizons. Because we implement optimization with recourse, at each
stage once the realization of the random variables (i.e., number of wells) is known, a recourse
decision (well locations and controls for that stage) will be made based on: (i) the outcome of past
decisions (which led to current reservoir states), and (ii) the realization of the random variables
(i.e., number of wells at the current stage). For stochastic optimization, we use a sequential
approach where the location and controls of the wells are optimized while the number of future
wells is represented as a random data process. The framework is initialized with well placement
optimization using fixed control settings. Once converged, the new well locations are used to start
the well control optimization solution. The cycle of well placement and control optimizations is
repeated until no further improvement in the objective function is observed. For a fixed control
setting, the well location optimization in Eq. (2.14) can be written as:
𝑚𝑖𝑛 𝑢 1
,𝑢 2
,…,𝑢 𝑇 {𝑓 1
(𝑢 1
,𝑞 1
0
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑𝑓 𝑡 (𝑢 𝑡 ,𝑞 𝑡 0
,𝜔 𝑡 𝑠 )
𝑇 𝑡 =2
}

(2.24)

where 𝑓 1
(∘) is the objective function of the first stage that depends on first stage decision
variables 𝑢 1
and 𝑓 𝑡 (∘) is the objective function of the stage 𝑡 that depends on that stage’s decision
variable (𝑢 𝑡 ) and the parameter 𝜔 𝑡 . Because the number of infill wells at each development stage
is discrete, the random variable 𝜔 𝑡 can be represented by finite number of development scenario
(S). 𝑢 𝑡 ∈ℝ
𝑛 𝑡 ,𝑡 =1,…,𝑇 are the decision variables representing the well locations for stages 1 to
T. In this work, SPSA algorithm (see Appendix A) is implemented to minimize the objective
function in Eq. (2.24). Once, the well placement optimization is converged the resulting well
locations (𝒖 =[𝑢̂
1
,𝑢̂
2
,…,𝑢̂
𝑇 ]) are used to define the following well control optimization problem
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

39

𝑚𝑖𝑛 𝑞 1
,𝑞 2
,…,𝑞 𝑇 𝐽 ={𝑓 1
(𝑢̂
1
,𝑞 1
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑𝑓 𝑡 (𝑢̂
𝑡 ,𝑞 𝑡 ,𝜔 𝑡 𝑠 )
𝑇 𝑡 =2
}

(2.25)

At each iteration, the decision variable 𝒒 =[𝑞 1
,𝑞 2
,…,𝑞 𝑇 ] is updated as follows:
𝒒 𝑘 +1
=𝒒 𝑘 −𝛼 𝑘 𝐵 𝑘 −1
𝛻 𝒒 𝐽 𝑘
𝛻 𝒒 𝐽 =𝛻 𝒒 {𝑓 1
(𝑢̂
1
,𝑞 1
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑𝑓 𝑡 (𝑢̂
𝑡 ,𝜔 𝑡 𝑠 ,𝑞 𝑡 )
𝑇 𝑡 =2
}
=𝛻 𝑞 1
𝑓 1
(𝑢̂
1
,𝑞 1
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑∇
𝑞 𝑡 𝑓 𝑡 (𝑢̂
𝑡 ,𝜔 𝑡 𝑠 ,𝑞 𝑡 )
𝑇 𝑡 =2

(2.26)
We note that in all of our optimization problems the decision variables pertain to the wells
at the currents stage (i.e., their locations and life-cycle control trajectories), As such, the cost of
drilling new and future wells is not a function of the optimization decision variables; hence, they
have zero gradients with respect to optimization decisions. The objective function includes a
weighted linear combination of the NPV values from different scenarios. Hence, a simple chain-
rule can be used to derive the gradient of Eq. (2.26) from the individual gradients calculated for
each scenario. In our implementation, we use a sequential well location and control optimization
where for the former we apply the SPSA (stochastic gradient approximation) while for the latter
we use an adjoint model to derive the gradients. For computational efficiency, the simulations and
gradient calculations are performed in parallel using a high-performance computing architecture.
2.6 Numerical Experiments
In this section, two sets of numerical examples are presented to compare the performance
of different approaches for field development optimization that are discussed in this chapter. The
first example is based on PUNQ-S3 reservoir model with 5 layers. A two-stage field development
optimization is implemented for this example in which well placement and well control
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

40

optimization are applied sequentially under different assumptions in future infill drilling. The
second example resembles the top layer of SPE10 benchmark model that is discretized into a
uniform 60×220×1 grid. A multi-stage field development optimization is performed for the second
example, in which well locations and controls are optimized at each stage. A two-phase
incompressible waterflooding experiment is used in both cases. The parameters used for
computing the NPV objective function are provided in Table 2.5.
Table 2.5: Parameter used for calculating the objective function (-NPV) for the first and second case studies
Oil price 50 $/𝑏𝑏𝑙
Water injection cost 10 $/𝑏𝑏𝑙
Water disposal/recycling cost 10 $/𝑏𝑏𝑙
Discount rate 0 (Case study 1) & 0.1 (Case study 2)
Drilling cost $ 2×10
6

2.6.1 Case Study 1: Two-Stage Optimization
In this section, a two-stage well placement and control optimization problem is presented
to demonstrate the significance of incorporating the uncertainty of future infill drillings in
formulating the optimization problem. This example is based on the PUNQ-S3 reservoir model
which was described in Section 2.2. We will compare the results of Perfect Information approach,
Expected Value Problem, Stochastic programming (here-and-now) and the deterministic problem
by disregarding the future infill well(s) uncertainty (No Drilling) to assess the performance of the
four approaches in field development optimization and their robustness to uncertainty in future
drilling events. We will follow the same scheduling as the motivating example where it is assumed
3 injectors and 3 producers are planned to be drilled at the beginning. The number of future infill
drilling is assumed uncertain with three different possibilities: 𝜔 1
=0,𝜔 2
=1, 𝜔 3
=2 (0, 1 or 2
infill wells after 5 years), each with equal probability of occurrence 𝑝 𝑠 =
1
3
; 𝑠 =1,2,3.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

41

2.6.1.1 Case I: Disregarding Future Drilling
As was addressed in Section 2.2, in this approach only the decision variables corresponding
to the current time-step are optimized without accounting for possible future infill drilling
opportunities over the entire life cycle. Therefore, in the first stage, Eq, (2.4) is solved to find 𝑥 1
.
In this problem 𝑥 1
represent the decision variables (location and control) of the first-stage wells (3
producers and 3 injectors). This formulation does not account for the possibility of future drilling
events. In the second stage after the revelation of the number of infill wells, the optimization
problem needs to be solved again to find optimal decision variables (𝑥 2
: location and control) for
the remainder of the project. Figure 2.7 shows the optimal configuration and the corresponding
final oil saturation plot, after 10 years of production for different number of future infill wells (a:
𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2). In this case, knowledge of future drilling is not included in the
optimization scheme from the beginning.

Perm. map

Sat. map

(a) (b) (c)
Figure 2.7: Results for case 1 field development (well location and operational control) optimization problem where
the uncertainty in future infill drillings is disregarded; top: optimal well configuration, bottom: saturation distribution
after 10 years of production for (a) No infill well (𝜔 1
=0), (b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

42

Figure 2.8 shows the optimal control trajectories for each drilling scenario (a: 𝜔 1
=0, b:
𝜔 2
=1, c: 𝜔 3
=2) where the solid lines represent the initial optimal trajectory in which the
knowledge of future infill drilling was not incorporated. The dashed and dotted lines represent the
optimal trajectory for the second 5 years of the project when one and two infill wells are
introduced, respectively. Table 2.6 summarizes the final solutions for a two-stage field
development based on the approach where the uncertainty in future drillings is neglected in
optimization formulation (case I).

Figure 2.8: Optimal control trajectories of the wells for case 1 where the possibility of future infill drillings is
neglected; solid lines are the control trajectories for the 3 producers and 3 injectors obtained at the beginning of the
project without accounting for the future infill well; dashed lines are the optimal control trajectories for the second 5
years of the project when it is decided to drill one infill well; dotted lines are the optimal control trajectories for the
second 5 years of the project when it is decided to drill two infill wells.

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

43

Table 2.6: Results of two-stage field development and control optimization (Case study 1)

NPV
($ MM)
Water Injected
(MMSTB)
Oil Produced
(MMSTB)
Water Produced
(MMSTB)
Case I: Disregarding Uncertainty
Scenario 1: 𝜔 1
=0 72.90 3.80 2.48 1.32
Scenario 2: 𝜔 2
=1 73.2 3.78 2.48 1.30
Scenario 3: 𝜔 3
=2 74.1 3.78 2.49 1.28
Case II: Perfect Information
Scenario 1: 𝜔 1
=0 72.90 3.80 2.48 1.32
Scenario 2: 𝜔 2
=1 83.11 3.88 2.68 1.20
Scenario 3: 𝜔 3
=2 86.58 3.35 2.56 0.79
Case III: Expected Value Problem (EVP)
Scenario 1: 𝜔 1
=0 68.65 3.91 2.45 1.46
Scenario 2: 𝜔 2
=1 83.11 3.88 2.68 1.20
Scenario 3: 𝜔 3
=2 84.1 3.15 2.45 0.70
Case IV: Stochastic Solution
Scenario 1: 𝜔 1
=0 70.13 2.29 1.93 0.36
Scenario 2: 𝜔 2
=1 82.5 2.52 2.22 0.30
Scenario 3: 𝜔 3
=2 84.38 2.68 2.30 0.38

2.6.1.2 Case II: Perfect Information
This problem has also been considered in Section 2.2. In this approach the decision maker
does not take any action unless the uncertainty is resolved, which may be not be realistic as a
decision (𝑥 ) has to be made before the realization of uncertain parameter (𝜔 ). We consider and
solve for this approach only for comparison purposes. This approach offers the best solution under
perfect information assumption. In the absence of uncertainty in future events, we solve Eq. (2.1)
for the first stage and second stage decision variables simultaneously. Figure 2.9 shows the
optimal configuration and the final oil saturation plot after 10 years of production, for different
number of future infill wells (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2) in Perfect Information
optimization. Figure 2.10 shows the optimal control trajectories for each drilling scenario (a: 𝜔 1
=
0, b: 𝜔 2
=1, c: 𝜔 3
=2) and Table 2.6 summarizes the final solutions for a two-stage field
development for Perfect Information approach (case II). Note that in this case, for each drilling
scenario, the optimal well location and control trajectories are optimized independently, and the
solutions are different.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

44

Perm. map

Sat. map

(a) (b) (c)
Figure 2.9: Results for perfect information optimization (case II); top: optimal well configuration, bottom: saturation
distribution after 10 years of production for (a) No infill well (𝜔 1
=0), (b) one infill well (𝜔 2
=1), (c) two infill
wells (𝜔 3
=2).

(a)
(b)
(c)
Figure 2.10: Optimal control trajectories of Perfect Information optimization (case II) for (a) No infill well (𝜔 1
=
0), (b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

2.6.1.3 Case III: Expected Value Problem (EVP)
In the expected value problem, the uncertain parameter 𝜔 is replaced with its mean value
𝜔̅. For this example, the mean value for the number of infill wells is 1 (𝜔̅=1). Therefore, in the
first stage, the problem is solved assuming one infill well will be drilled in the second stage (Eq.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

45

2.5). In the second stage when the number of infill wells is revealed, if it is decided to drill one
infill well, it means solving for 𝜔̅ was a lucky guess, otherwise we need to re-solve for the second
stage decision variable 𝑥 2
(if the number of infill wells are 0 or 2). Figure 2.11 shows the optimal
well configuration and the final oil saturation map after 10 years of production for different number
of future infill wells (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2) for EVP optimization. Note that the
solution for 𝜔 2
=1 (Figure 2.11(b)) is identical to solution of the same scenario for the Perfect
Information case Figure 2.9(b) (lucky guess) because the assumption behind both formulations is
to have one infill well for the second stage. Figure 2.12 shows the optimal control trajectories for
each drilling scenario (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2), where the solid lines are the original
optimal trajectories obtained at the beginning by assuming that one infill well will be drilled; the
dashed and dotted lines correspond to drilling none and two wells, respectively, in the second
stage. For instance, when it is decided to not drill any infill well, INJ3 (close to the infill well
PROD4) injects larger amounts of water to sweep the oil towards PROD2 and PROD3 (Figure
2.11(a) and Figure 2.12). On the other hand, if two infill wells are drilled, the second infill well
(PROD5) is placed at right corner of the reservoir to produce residual oil trapped in that region
and accordingly INJ1 and INJ2 reduce their water injection to decrease the water production in the
infill well PROD5 (Figure 2.11(c) and Figure 2.12). Table 2.6 summarizes the final numeric
values for a two-stage field development optimization using expected value approach (case III).

Perm map

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

46

Sat. map

(a) (b) (c)
Figure 2.11: Results for Expected Value Problem (EVP) optimization (case III); top: optimal well configuration,
bottom: saturation distribution for each individual layer after 10 years of production for (a) No infill well (𝜔 1
=0),
(b) one infill well (𝜔 2
=1) and (c) two infill wells (𝜔 3
=2).

Figure 2.12: Optimal control trajectories of the wells for Expected Value Problem (EVP) optimization (case III):
solid lines are the control trajectories for the 4 producers (PROD4 is open to flow after 5 years) and 3 injectors obtained
at the beginning; dashed lines are the optimal control trajectories for the second 5 years of the project when it is
decided to drill no infill well; dotted lines are the optimal control trajectories for the second 5 years of the project
when it is decided to drill two infill wells.

2.6.1.4 Case IV: Stochastic (Here-and-now) Solution
As was discussed in Section 2.4, in stochastic (here-and-now) problem a probabilistic
functional of the objective function (usually the expected value) is optimized over all possible
outcomes of the random process (𝜔 ). Therefore, for the first stage the two-stage stochastic program
can be written as:

𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 1
,𝑥 2
𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑓 2
(𝑥 2
,𝜔 𝑠 )
3
𝑠 =1

(2.27)

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

47

Figure 2.13 shows the first stage solution of the problem in (2.27) while accounting for
the uncertainty in future infill drillings. Because for this example we are assuming equi-probable
drilling scenarios, the initial 3 producers are placed and operated in a way to maximize the revenue
regardless of which drilling scenario would occur after 5 years. Figure 2.14 illustrates the
evolution of the objective function (𝐽 =−𝐸 𝜔 [𝑁𝑃𝑉 ]) at each iteration.

Perm. map

Sat. map

Figure 2.13: First stage solution of stochastic (here-and-now) problem; top: optimal configuration for 3 producers
and 3 injectors; bottom: saturation distribution after 5 years of production.

Figure 2.14: Evolution of the objective function for first stage of stochastic (here-and-now) optimization (sequential
implementation).

Figure 2.15 shows the second stage of the stochastic solution for all 3 scenarios (a: 𝜔 1
=0, b:
𝜔 2
=1, c: 𝜔 3
=2) and Figure 2.16 depicts the optimal control trajectories for each of them. The
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

48

numerical values for the performance of the two-stage field development obtained through the
stochastic formulation are summarized in Table 2.6 (case IV).

Perm. map

Sat. map

(a) (b) (c)
Figure 2.15: Results for stochastic (here-and-now) optimization (case IV); top: optimal well configuration, bottom:
saturation distribution for each individual layer after 10 years of production for (a) No infill well (𝜔 1
=0), (b) one
infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

(a)
(b)
(c)

Figure 2.16: Optimal control trajectories of stochastic optimization (case IV) for (a) No infill well (𝜔 1
=0), (b)
one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

Figure 2.17 compares the NPV values of the four formulations for field development
optimization in Case Study 1 relative to the stochastic solution (being 1 or 100%). For all 3
scenarios (𝜔 ={0,1,2} ) the perfect information approach provides the best NPV because this
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

49

approach is based on perfect knowledge of the scenarios (which is ideal but not realistic). For the
No Drilling case (where the uncertainty in future drilling is disregarded and only the current time
decision variables are optimized as if no infill well would be drilled in the future) we observe the
lowest NPV values compared to all other three approaches except for the first scenario where no
infill well is drilled (i.e., the underlying assumption of the scenario). However, in real field
applications the probability of occurrence of the first scenario where no infill well is drilled is
smaller compared to other scenarios, although we assigned the same probability to all scenarios in
this case study. The Expected Value Problem (EVP) can obtain high NPV values for 𝜔 2
=1,
because in this formulation the mean value of the uncertain parameter (𝜔̅=1) is assumed for the
number of future infill wells. However, the stochastic (here-and-now) formulation has a
reasonably high NPV values for all 3 scenarios meaning that stochastic formulation is robust to
the uncertainty in number of infill wells. For the first drilling scenario (𝜔 1
=0), the stochastic
solution achieved 2% increase in NPV compared to the expected value formulation. For the second
drilling scenario (𝜔 2
=1), the stochastic solution results in 12% increase in NPV compared to No
Drilling formulation. This difference increases to 14% for the last drilling scenario (𝜔 3
=2). We
can conclude that in the No Drilling formulation as the number of infill wells increases, the NPV
value increasingly deviates from stochastic solution. It is important to point out that the results
from the Expected Value Problem are close to the stochastic solution mainly because the number
of drilling scenario assumed in this case study are limited to three scenarios. In the next case study,
we show that as the number of drilling scenarios and stages increases, the Expected Value Problem
solution will deviate significantly form the Stochastic Solution.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

50

Figure 2.17: Percetage-wise change in final values of NPV for No Drilling (Disregarfing the uncertainty), EVP
(Expected value problem) and Perfect Information optimization approaches compared with stochastic (here-and-
now) solution.

2.6.2 Case Study 2: Multi-Stage Optimization
2.6.2.1 Deterministic (Look-Ahead) Approach
In this approach, the infill drilling opportunities are modeled as decision variables and their
optimal locations and corresponding drilling times are optimized. This approach is implemented
on the top layer of the SPE10 benchmark model. The problem is initialized with 12 producers and
3 injectors, which are uniformly distributed throughout the reservoir (Figure 2.18(a)). Figure
2.18(b) shows the well configuration after implementing a deterministic optimization where future
drilling events are considered as decision variables. The cycle of well placement and drilling
schedule optimizations was repeated until marginal improvement in NPV was observed. As can
be seen in the optimal configuration of Figure 2.18(b), out of the initial twelve producers, only 6
of them are kept and out of the three initial injectors, one of them is removed. Figure 2.19 shows
the evolution of objective function (-NPV) at each iteration, the cycle of well placement and well
optimization was repeated 12 times, until no improvement in the objective function was achieved.
Table 2.7 provides the optimal drilling schedule obtained for the six remaining producers. Wells
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

51

P5 and P10 are drilled from the start of the project, while Wells P8 and P4 come online after 2 and
4 years, respectively. Wells P7 and P11 are planned to be drilled after 6 years of production. Table
2.8 provides the NPV, water production, oil production and water injection values for this example.

(a) (b)
Figure 2.18: a) top: Initialization for deterministic field development optimization (injectors and producers are shown
with red crosses and black circles, respectively) ; bottom: Oil saturation map after 10 years of production
corresponding to initial configuration b) top: Optimal configuration obtaind by implementing the optimization
algorithm for well placement and well scheduling; bottom: Oil saturation map after a total of 10 years of production
for optimal design.

Figure 2.19: Evolution of the objective function for deterministic (look-ahead) well placement and well scheduling
optimization (sequential implementation).

Table 2.7: Optimal drilling schdule for deterministic field development optimization
Year 0 P5 & P10
Year 2 P8
Year 4 P4
Year 6 P7 & P11

Table 2.8: NPV, water injection, oil production and water production results of deterministic (look-ahead) field
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

52

development optimization
NPV ($) Water Injected (MMSTB) Oil Produced (MMSTB) Water Produced (MMSTB)
47.91MM 2.2864 1.8318 0.4546

2.6.2.2 Disregarding Future Drillings
In this approach, the focus is merely on today’s decision variable without considering the
possible future actions. Unlike stochastic optimization, in which the current decision variables
(well locations) are optimized while accounting for the uncertainty in future drilling activities, in
this approach no future uncertainty is accounted for. In other words, at any horizon when
optimizing the location of the wells, the optimization algorithm will not have any knowledge about
possible future development activities. This approach is typically adopted in the traditional well
control and well placement optimization methods in the literature and is expected to provide only
suboptimal solution (when new wells are drilled).
In this section, we borrow the optimal drilling sequence that was achieved in the previous
section (deterministic field development); however, we do not include the uncertainty in future
development activities to quantify the loss resulted from ignoring future drilling uncertainty in the
same example. Figures 2.20(a)-(d) show the sequential well placement problem without
incorporating the uncertainty in future drilling, for each stage, by minimizing the following
deterministic problem for the entire reservoir’s life cycle:
𝑚𝑖𝑛 𝑢 𝑡 𝜖 𝒰 𝑡 (𝑢 [1:𝑡 −1]
)
𝑓 (𝑢 𝑡 ) 𝑓𝑜𝑟 𝑡 =1,…,𝑇
(2.28)
where 𝑢 𝑡 ∈ℝ
𝑛 𝑡 ,𝑡 =1,…,𝑇 are the decision variables representing well locations at stage
t, 𝑓 (𝑢 𝑡 ) is the cost function for the remaining life of project, and 𝒰 𝑡 (𝑢 [1:𝑡 −1]
) is the feasible set
for well locations at stage 𝑡 , which depends on all the past decisions up to stage 𝑡 . In other words,
at each stage, the well locations are optimized assuming that they are the only wells that will be
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

53

operating throughout the life-cycle of the reservoir. We minimized the above function using the
discrete version of the SPSA algorithm (Bangerth et al., 2006, Li and Jafarpour 2012).
(a) First stage well placement

(b) Second stage well placement

(c) Third stage well placement

(d) Fourth stage well placement

Figure 2.20: sequential well placement without accounting for the future drilling plans for the scheduling 4-1-1-2 (4
wells (2 injectors+ 2 producers) at beginning, 1 well after two years, 1 well after four years and 2 wells after 6 years)
(a) left: optimal location for producers P1 & P2 and injectors I1 & I2 scheduled to be drilled at the beginning of
project; right: Saturation map after a total of 2 years of production (b) left: optimal location for P3 scheduled to be
drilled at the beginning of the second stage; right: Saturation map after a total of 4 years of production (c) left: optimal
location for P4 scheduled to be drilled at the beginning of the third stage; right: Saturation map after a total of 6 years
of production (d) left: optimal locations for P5 and P6 scheduled to be drilled at the beginning of the fourth stage;
right: Saturation map after a total of 10 years of production.

Figure 2.20(a) shows the optimal locations for P1, P2, I1 and I2 (all of them drilled at the
beginning, based on the solution in the previous section). Figure 2.20(b-d) shows the optimal
configuration for Stage 2 (one infill well (P3)), Stage 3 (one infill well (P4)) and Stage 4 (two infill
wells (P5 & P6)). Table 2.9 summarizes the NPV values, water injection, water production and
oil production of each horizon for this optimization approach. Comparing the results with those in
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

54

Table 2.8 (deterministic method), we can see a 12% decrease in the NPV, relative to the look-
ahead optimized solution, when the uncertainty in future drilling is unaccounted for.
Table 2.9 NPV, water injection, oil production and water production results for the case where the uncertainty in
possible future drilling is disregarded
Horizon NPV ($) Water Injected
(MMSTB)
Oil Produced (MMSTB) Water Produced (MMSTB)
1 8.95 MM 0.4573 0.4403 0.0170
2 14.55 MM 0.4573 0.4329 0.0244
3 10.76 MM 0.4573 0.3665 0.0908
4 8.53 MM 0.9146 0.4034 0.5112
Sum 42.79 MM 2.2865 1.6431 0.6434

3.6.2.3 Expected Value Problem
To implement the expected value approach the uncertain parameter (𝜔 ) is replaced with its
expected value (𝜔̅) while keeping the problem deterministic. Because the number of wells has to
be a discrete value, we assume 𝜔̅=2, for each stage to be the mean value of the number of wells.
Therefore, at each stage the following problem is solved:
𝑚𝑖𝑛 𝒖 𝑓 𝑡 (𝑢 𝑡 )+ ∑ 𝑓 𝑘 (𝑢 𝑘 ,𝜔̅
𝑘 )
𝑇 𝑘 =𝑡 +1
𝑓𝑜𝑟 𝑡 =1,…,𝑇
(2.29)
Where 𝑢 𝑡 ∈ℝ
𝑛 𝑡 ,𝑡 =1,…,𝑇 are the decision variables representing well locations at stage
t, 𝑓 𝑡 (𝑢 𝑡 ) is the cost function for stage 𝑡 which is a function of the decision variable for that stage
(𝑢 𝑡 ). 𝑓 𝑘 (𝑢 𝑘 ,𝜔̅
𝑘 ) is the cost function for the remaining life of project ([𝑡 +1:𝑇 ]) which is a
function of the decision variable for each stage (𝑢 𝑘 ) and number of infill well for that development
stage (here 𝜔̅
𝜏 =2). Table 2.10 and Figure 2.21(a-d) show the NPV values, water injection, water
production and oil production for each horizon and the stage-wise optimal configuration for this
optimization approach, respectively. Comparing the results with those in Table 2.8 (deterministic
method), we can see a decrease in NPV (relative to the Look-Ahead Approach) when expected
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

55

value of the number of wells is assumed for each stage. However, compared to Table 2.9 (where
the possibility of future drilling is neglected), a 3.5% increase in NPV is achieved.
(a) First stage well placement

(b) Second stage well placement

(c) Third stage well placement

(d) Fourth stage well placement

Figure 2.21: sequential well placement using expected value approach 4-1-1-2 (4 wells (2 injectors+ 2 producers) at
beginning, 1 well after two years, 1 well after four years and 2 wells after 6 years) (a) left: optimal location for
producers P1 & P2 and injectors I1 & I2 scheduled to be drilled at the beginning of project; right: Saturation map
after a total of 2 years of production (b) left: optimal location for P3 scheduled to be drilled at the beginning of the
second stage; right: Saturation map after a total of 4 years of production (c) left: optimal location for P4 scheduled to
be drilled at the beginning of the third stage; right: Saturation map after a total of 6 years of production (d) left: optimal
locations for P5 and P6 scheduled to be drilled at the beginning of the fourth stage; right: Saturation map after a total
of 10 years of production.

Table 2.10: NPV, water injection, oil production and water production results for the Expected Value Problem with
𝜔̅=2
Horizon NPV ($) Water Injected
(MMSTB)
Oil Produced
(MMSTB)
Water Produced
(MMSTB)
1 8.76 MM 0.4573 0.4373 0.0200
2 14.61 MM 0.4573 0.4343 0.0229
3 10.95 MM 0.4573 0.3700 0.0873
4 9.95 MM 0.9146 0.5246 0.3900
Sum 44.27 MM 2.2865 1.7662 0.5202
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

56

2.6.2.4 Stochastic (Here-and-Now) Optimization
In this example, multi-stage well placement is implemented in which the location of infill
wells at each stage is optimized while accounting for the residual uncertainty associated with the
(possible) remaining future drillings. The project life of ten years is divided into 4 horizons. The
wells are planned to be drilled at the beginning of the project, and after 2, 4 and 6 years of
production. It is assumed that a maximum of 3 wells can be drilled at each horizon, resulting in 4
possible drilling outcomes at each horizon (0, 1, 2 and 3 wells being drilled). Over the four stages
in this problem, 64 possible drilling outcomes can occur. Figure 2.22 shows the decision tree for
this example. As mentioned earlier, we assign initial potential locations for future infill well
locations of each scenario and update those locations when the current well locations are updated.
To do this for each scenario, we generate a set of possible configurations according to a calculated
quality map and well distance criterion, and then select a single configuration with the highest
NPV value. Quality maps for drilling vertical wells is generated based on the perceived quality in
each grid-block as 𝐾 ℎ𝜑 𝑆 𝑜 , where 𝐾 , ℎ and 𝜑 are static properties representing permeability,
thickness and porosity of each grid block, respectively, and 𝑆 𝑜 represents oil saturation which is a
dynamic property.

Figure 2.22: Scenario tree for sequential well placement considering future development possibilities with 4 stages

To find the potential locations for infill wells using the quality map, future wells are placed
in grid blocks with high values of quality index and once a grid-block is selected for an infill well,
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

57

the updated quality map must be recalculated to account for changes in oil saturation due to new
wells. In addition to selecting potential future infill well locations for each scenario, prior estimates
of NPV values can be obtained and the resulting normalized NPV values can be used to assign
probabilities to each scenario. As expected, all the scenarios do not have the same probability of
occurrence. Here, scenarios with higher potential NPV values are assigned higher probabilities
(linearly). The basis for adopting this approach is that in the future the decision-maker is more
likely to adopt scenarios with higher NPV outcomes. However, other choices of these weights may
be assigned by the user, e.g., depending on the level of confidence in adopting each scenario. It is
important to note that while in this work we have specified a range for the number of wells to
define different drilling scenarios, in real applications, based on initial uncertainty in reservoir
geology, reservoir engineers may have a set of plausible development strategies that can serve as
possible future development scenarios. Furthermore, we have assumed that the time of drilling for
each stage is known. This assumption can also be relaxed to define additional scenarios based on
different drilling time. The focus of the current work is to demonstrate the importance of
considering future development scenarios as uncertain parameters. However, the scenarios defined
for each field should be decided based on the specific circumstances of the project at hand.
As can be observed, the first scenario in which no well is drilled for year 2, 4 and 6 years
has the lowest probability (1.24%), whereas the optimal configuration from the previous section
(with one well at year 2, one well at year 4, and two wells at year 6) is assigned the highest
probability. Note that the first stage decisions are scenario independent. Based on the solution of
the previous section, 2 producers and 2 injectors are planned for the first stage of the project in all
scenarios. Starting from 𝑡 =0, we solve Equation (2.24) by implementing the SPSA algorithm to
find optimal locations for the 2 injectors and 2 producers while accounting for the possibility of
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

58

future drilling activities for the remaining 3 horizons. Figure 2.23 shows the optimal well
configuration and the oil saturation map after 2 years of production (before starting the second
stage). As can be seen the configuration is very different from the optimal configuration of Figure
2.20(a) where the uncertainty in future drillings is not accounted for.

Figure 2.23: left: Optimal configuration obtaind by implementing the optimization algorithm for well placement while
accounting for future drilling possilibities (injectors and producers are shown with red crosses and black circles,
respectively); right: Oil saturation map after a 2 years of production according to optimal design.

In sequential decision making, new information can be used to update the model (reservoir
geology) and make decisions for later stages. However, in this chapter, because our aim is to
present methods for field development optimization under uncertainty in future development
activities, we do not update the model using data assimilation. Even when the model remains
unchanged, in many cases the operator may not implement the proposed strategy for future infill
drilling, which is why the uncertainty in drilling schedule has to be considered. To explore the
advantage of using this approach, we solve the multi-stage well placement problem for all 64
possible outcomes to obtain the resulting optimal NPV values. In real applications, only one path
is followed and the decision about the number of wells for the current stage is provided by the
operator (based on a variety of factors, not all of which may be predictable). For instance, if we
choose to follow the same scenario that was achieved by the deterministic (Look-Ahead) field
development strategy, the scenario tree for the second stage will shrink to the tree branch
highlighted in red (Figure 2.24). With the new scenario tree, Eq. (2.24) needs to be resolved for
the second stage information (drilling one infill wells) to find optimal location for that well
(optimization with recourse). This procedure will continue for the third and fourth stages. In the
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

59

last stage, because there is no remaining uncertainty in future drilling plans, the problem becomes
a deterministic well placement problem.

Figure 2.24: Scenario tree for the second stage well placement (highlighted in red) assuming it is decided to drill
one infill well in the second stage

Figure 2.25 shows the NPV values for all 64 drilling scenarios with their corresponding
assigned probabilities (shown in Figure 2.22). On average, we observe an increase in NPV with
increasing probabilities of the scenarios. Note that the probabilities were calculated before solving
for the scenarios and they were assigned based on a subset of un-optimized well configurations
using quality maps and well distance criteria. The highest NPV was achieved for a 4-1-1-2 drilling
schedule (4 wells (2 injectors+ 2 producers) at beginning, 1 well after two years, 1 well after four
years, and 2 wells after 6 year), which confirms our optimal solution from the deterministic (Look-
Ahead) field development strategy in the previous section example. Figure 2.26 shows the results
of sequential well placement for the 4-1-1-2 drilling sequence. The final configuration (Figure
2.26(c) is very similar to Figure 2.18(b)). The final NPV achieved for this particular scenario is
equal to $8.62 𝑀𝑀 + $14.77 𝑀𝑀 + $11.94 𝑀𝑀 + $12.53 𝑀𝑀 = $47.86 𝑀𝑀 , which is close
to the NPV achieved using the deterministic (look-ahead) field development optimization. It is
important to note that the main distinction between the Look-Ahead approach and stochastic
solution is that in the former future drilling events are considered as decision variables while in
the latter they are modeled as uncertain variables. Table 2.11 summarizes the values of NPV,
water injection, water production and oil production for each horizon.
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

60

Figure 2.25: NPV of sequential four-stage well placement for all 64 drilling scenarios vs. their corresponding
probabilities (with incorporating the uncertainy in future drilling possibilties)

(a) Second stage well placement

(b) Third stage well placement

(c) Fourth stage well placement

Figure 2.26: 4-stage well placement for the scheduling 4-1-1-2 (4 wells (2 injectors+ 2 producers) at beginning, 1
well after two years, 1 well after four years and 2 wells after 6 year) (a) left: optimal location for P3 scheduled to be
drilled at the beginning of the second stage; right: Saturation map after a total of 4 years of production (b) left: optimal
location for P4 scheduled to be drilled at the beginning of the third stage; right: Saturation map after a total of 6 years
of production (c) right: optimal locations for P5 and P6 scheduled to be drilled at the beginning of the fourth stage;
left: Saturation map after a total of 10 years of production.

Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

61

Table 2.11: NPV, water injection, oil production and water production results of stochastic (here-and-now) field
development optimization
Horizon NPV ($) Water Injected
(MMSTB)
Oil Produced
(MMSTB)
Water Produced
(MMSTB)
1 8.62 MM 0.4573 0.4347 0.0226
2 14.77 MM 0.4573 0.4369 0.0204
3 11.94 MM 0.4573 0.3873 0.0700
4 12.53 MM 0.9146 0.5673 0.3473
Sum 47.86 MM 2.2865 1.8262 0.4603

Interestingly, comparing Table 2.9 and 2.11 the NPV value achieved in the first stage in
the case where the uncertainty in future drilling is not incorporated is higher than that for the
stochastic (here-and-now) solution. The reason is that when the uncertainty in future infill drilling
is not accounted for, optimization algorithm assumes the initial 4 wells (P1, P2, I1 and I2) are the
only wells for the entire project life (it does not have any knowledge of future drillings) and the
four wells are placed in the best possible locations. However, at later times, particularly at stage 4
(Figure 2.20 (d)), it becomes more difficult to find a location that can optimally contribute to the
production, mainly due to lack of information about addition of the new wells. Comparing the
results for the case where uncertainty is neglected (Table 2.9) with those obtained from the
stochastic solution shows a 12% increase in NPV when the stochastic approach is used. Figure
2.27(a) compares stage-wise normalized NPV values of the four presented approaches. In Figures
2.27(b)-(d), the normalized water production, oil production and recovery factors corresponding
to No Drilling (possibility of future drillings is disregarded), EVP (Expected Value Problem) and
stochastic (recourse problem) and Perfect Information are compared, showing that by accounting
for future drilling uncertainty the recovery factor increased from 39.5% (disregarding uncertainty)
to 44% (stochastic).
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

62

(a) (b) (c)

(d)
Figure 2.27: Comparison of (a) normalized NPV, (b) normalized water production, (c) normalized oil production and
(d) recovery for sequential well placement of for the case with no incorporation of uncertainty in future drilling in
optimization scheme (No Drilling-white bar), Expected value approach (EVP-light gray bar), stochastic approach
where the uncertainty in possible future drillings is accounted for (dark gray bar) and perfect information approach
(black bar)

We note that Figure 2.27 only shows the results for one of the scenarios [4-1-1-2]. To
verify the significance of incorporating uncertainty in future development activities under other
scenarios, we repeated the same approach for the other 63 possible scenarios. Figure 2.28 shows
the percentage-wise increase in the NPV when the uncertainty in future drilling is incorporated
and when it is disregarded. In all the cases, except for the first scenario, the NPV is higher in the
stochastic (recourse) approach, which confirms our earlier statement that considering uncertainty
in future drilling activities will result in improved reservoir performance. The scenario in which
the increase is negative is the case where no infill well is drilled throughout the reservoir’s life-
cycle [4-0-0-0]. In this case, the scenario is identical to no-drilling assumption that was adopted
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

63

when future drilling uncertainty was disregarded. The maximum increase is observed for the last
scenario in which a total of 9 infill wells are drilled (~%27 increase). This scenario represents the
maximum number of infill wells that can be drilled during the last 3 stages of the project. However,
because the No-Drilling approach does not account for future drilling possibilities, finding good
locations for a large number of infill wells becomes increasingly difficult. Figure 2.29 shows the
NPV values of the two methods in a bar plot. The solid and dash lines show the mean of the NPV
for all 64 scenarios in the Stochastic and No-Drilling (i.e., disregarding future drilling
opportunities is neglected) approaches, respectively. As can be seen the mean value of stochastic
(here-and-now) approach ($45.55 𝑀𝑀 ) is about %14 higher than the mean value of field
development approach without incorporating the future drilling uncertainty ($40.09 𝑀𝑀 ). This
difference indicates what is known in the stochastic optimization literature as the Value of
Stochastic Solution (VSS).

Figure 2.28: Comparison of NPV differences (in percentage) between stochastic approach and the case where
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

64

uncertainty in future drilling is disregarded (No Drilling) for all 64 possible drilling scenarios

Figure 2.29: Comparison of normalized NPV vs. the occurrence probability of 64 drilling scenarios between
stochastic (here-and-now) approach and the approach in which the uncertainty in future drillings is disregarded (No
Drilling).

2.6.2.5 Perfect Information (Distribution) Approach
In the perfect information approach, knowledge about the realization of future events is
obtained prior to solving the optimization problem. In this case, for each scenario a deterministic
optimization is solved to find the corresponding optimal NPV values. The distribution of the
optimal NPV values for all the scenarios provides the solution of the Perfect Information (wait-
and-see) approach.
Figure 2.30 shows the NPV values for the Stochastic (recourse) and Perfect Information
approaches sorted based on scenario probabilities for all 64 scenarios. The solid and dash lines
show the mean of NPV for all 64 scenarios in the Perfect Information and Stochastic approaches,
respectively. As can be seen the mean value of the perfect information approach is about %2
higher than the mean value of recourse solution. This difference is also known as the Expected
Value of Perfect Information (EVPI).
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

65

Figure 2.30: Comparison of normalized NPV vs. the occurrence probability of 64 drilling scenarios between Perfect
Information (wait-and-see) and stochastic (here-and-now) approaches

2.7 Conclusion
In this work, we propose, for the first time, a stochastic reservoir optimization approach
under uncertainty in future developments. The motivation behind the proposed framework is that
oilfields undergo development activities throughout their life-cycle and disregarding future
developments when optimizing current well configurations and well control settings is likely to
result in suboptimal solutions over the life of the field. A motivating example showed that
incorporating the knowledge about future developments can have a significant impact on
hydrocarbon recovery performance. We then studied two approaches for incorporating future
developments in reservoir optimization. The first is a deterministic formulation, in which future
development activities are included as decision variables that are optimized upfront. This approach
assumes that the optimized future decisions will be implemented exactly as they are prescribed.
However, in practice, future optimization decisions are not likely to be implemented as obtained
from an optimization problem (for several reasons, including economic, technical, and field
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

66

management constraints). In the second approach, we developed a stochastic formulation, in which
future developments are incorporated as uncertain events that are modeled using random variables.
In this case, the decision variables pertain to the current stage while the future development plans
are incorporated as a stochastic process.
To evaluate the performance of each approach and demonstrate the advantages of
incorporating future development events, we presented two sets of numerical experiments. While
the uncertainty space in future events can involve several aspects, e.g., number, type, and location
of wells as well as their control setting and drilling schedule, we considered the uncertainty in the
number of infill wells and their corresponding drilling time. In the first example, we compared the
solutions from stochastic, Expected Value Problem, Perfect Information and the approach where
the possibility of future drilling is disregarded (No Drilling) for PUNQ-S3 model. It was observed
that the stochastic solution provides an optimal development plan that enables us to hedge against
the uncertainty in future developments, while the other approaches could not yield a solution that
is robust against future development decisions. In the second example, first we modeled infill
drilling as an optimization decision variable based on “perfect implementation” assumption,
meaning that all the development decisions are made at the beginning of the project. We
implemented sequential well scheduling and placement optimization until minimal improvement
to NPV was achieved.
Next, we modeled infill drilling as an uncertain (random) variable. In this case, future
events (infill wells) were considered as random variables whose realizations would be observed at
later stages. To account for this uncertainty, a multistage stochastic optimization formulation was
performed to find a sequence of decisions that minimize an expected cost function (over possible
realizations of the future events). In the presented formulation, the decisions can depend on
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

67

observed realizations as they unfold. The numerical example involved infill drilling uncertainty
that was defined as a stochastic process over development stages as discrete time intervals and the
stochastic process could be represented through a scenario tree. Specifically, we considered well
placement over 4 drilling stages, where at each stage four drilling scenarios could occur: drilling
0, 1, 2 or 3 infill wells. At each stage, the locations of current wells were optimized while
accounting for the included range of possible future drilling scenarios. The resulting optimal policy
from the stochastic approach showed an average of 14% improvement in NPV over the well
placement approach where the uncertainty in future drillings is disregarded, with the maximum
improvement over all scenarios being 27%. The computational results indicate that the solutions
from the stochastic model are very robust to the uncertainty of future drilling activities. The
stochastic solution is obtained by considering 64 scenarios, which were run on a high-performance
computing cluster, where the objective function for all 64 scenarios were calculated in parallel by
executing the simulations over 64 cores. The results were also compared with the Expected Value
Problem (EVP) formulation, where the random variable was replaced with its expected value to
keep the problem deterministic. Although the results indicated a better performance than was
achieved with the no-drilling assumption, the stochastic formulation still outperformed to the
solution of EVP. This can be attributed to the lack of robustness in EVP formulation as the solution
of the latter approach does not account for several alternative future drilling scenarios.
The findings of our research highlight the importance of including the uncertainty in future
field developments in oilfield optimization. While the current work presented sequential well
placement in which uncertain future events consisted of the number of wells at each stage, the
presented method can also be extended to include the uncertainty in the reservoir model (geologic
uncertainty). Generalization to consider geologic uncertainty and a closed-loop formulation with
Chapter 2: Stochastic Oilfield Optimization under Uncertain Future Development Plans

68

dynamic model updating is straightforward and involves repeating the optimization procedure after
each model updating step. In such a closed-loop implementation, updatable decision variables
(those that are not irreversible) can be adjusted after each model updating stage. Another
exogenous factor contributing to drilling decisions is the oil price, in that case the formulation
needs to account for the possible fluctuations in future oil price. However, the goal of this chapter
was to demonstrate that without incorporating future development uncertainty in oilfield
optimization problems, it is highly unlikely to find optimal solutions, as any new development
event can markedly diminish the quality of the optimal solution. On the other hand, formulations
that incorporate the uncertainty in future development plans are robust and offer hedging against
new development activities.

Chapter 3: Optimization under Uncertainty in Future Operations

69

Chapter 3
Optimization under Uncertainty in
Future Operations
3.1 Introduction
In chapter 2 chapter, we only considered the future number of wells as uncertain variables
and optimized the corresponding location and control trajectories, however, in real applications
not only the number of future infill wells are unknown, their future location and operational
settings are uncertain as well. In this chapter, we will extend the formulation to account for future
location and control setting uncertainty.
3.2 Problem Formulation
In field development optimization, at early stages both the number of infill wells (𝜔 ) and
their respective locations (𝑢̃) and control settings (𝑞̃) are typically uncertain. Denoting the vector
of uncertain variables as 𝜃 , where 𝜃 =[𝜔 ,𝑢 , ̃𝑞̃]
𝑇 , the multi-stage stochastic programing can be
expressed as:
min
𝑥 1
𝜖 𝜒 1
𝑓 1
(𝑥 1
)+𝐸 𝜃 2
[ min
𝑥 2
𝜖 𝒳 2
(𝑥 1
,𝜃 2
)
𝑓 2
(𝑥 2
,𝜃 2
)
+𝐸 𝜃 3
|𝜃 2
[…+𝐸 𝜃 𝑇 |𝜃 2
,…,𝜃 𝑇 −1
[ min
𝑥 𝑇 𝜖 𝒳 𝑇 (𝑥 [1:𝑇 −1]
,𝜃 [2:𝑇 ]
)
𝑓 𝑇 (𝑥 𝑇 ,𝜃 𝑇 )]]]

(3.1)

Assuming stage-wise independent drilling decisions, the uncertainty in the number of
future infill wells (𝜔 [𝑡 +1:𝑇 ]
) can be formulated as a sequential decision-making problem with
unconditional expectations in multi-stage formulation. On the other hand, characterizing the
Chapter 3: Optimization under Uncertainty in Future Operations

70

uncertainty in future well locations and controls are more complex because the number of possible
scenarios can be extremely large, and the well locations are stage-wise dependent (the well
locations at stage 𝑡 depend on well locations decided at previous stages). Therefore, incorporating
the uncertainty in future infill well locations and control settings (𝑢̃
[𝑡 +1:𝑇 ]
,𝑞̃
[𝑡 +1:𝑇 ]
) is more
involved.
3.2.1 Modeling the Uncertainty in Future Operations
One way to relax the uncertainty in well locations is to assign potential locations to future
infill wells. For allocating potential locations to future infill wells, a subset of possible well site
combinations based on reservoir quality maps and a set of constraints such as minimum interwell
spacing or other well configuration constraints is generated. However, the potential locations for
future wells must be updated at each iteration as for each iterate (proposed current well locations)
the plausible/promising future well locations are different (and are needed to evaluate the
corresponding realizable NPV values). In optimization with recourse, the updated future locations
are not fixed and can be corrected at each stage using new information and past decisions.
Cruz et al. (1999) introduced the concept of the quality map, which is a two-dimensional
representation of reservoir response and its uncertainties. The quality map, which is a measure of
“how good” a region of the reservoir is for production, can be used for ranking stochastic
realization and selecting well locations with fewer simulation runs. In their work, the data points
necessary to generate the quality map are obtained by running a flow simulator with a single well
and varying the location of the well in each run to have a good coverage of the entire horizontal
grid. The “quality” for each position of the well is the cumulative oil production or NPV value
after a long time of production. However, because in reservoir optimization the decision variables
are optimized iteratively and, therefore, the quality map should be updated at each iteration,
Chapter 3: Optimization under Uncertainty in Future Operations

71

generating the quality map by running a flow simulator with varying the location of the well over
the entire horizontal grid is not computationally feasible. Therefore, in this study the quality map
is generated based on reservoir properties in each grid bock as follows:
∑ 𝑘 𝑙 ℎ
𝑙 𝜑 𝑙 𝑆 𝑜𝑙
𝑛 𝑙=1
𝑛
(3.2)
Here, 𝑛 is total number of reservoir layers in vertical direction. As mentioned before, the
quality map is a two-dimensional representation of the reservoir’s quality, showing the mean of
all the grid block values in the vertical direction for the same horizontal position. Figure 3.1(d)
depicts the quality map for a two-dimensional reservoir obtained as the product of the saturation
(Figure 3.1(a)), normalized permeability (Figure 3.1(b)) and normalized porosity (Figure 3.1(c)).
Note that for this reservoir it is assumed the thickness (ℎ) is uniform for the entire reservoir.

Figure 3.1: (a) Initial saturation map; (b) Normalized permeability map; (c) Normalized porosity map; (d)
Corresponding quality map of the reservoir.

Because saturation is a dynamic property a dynamic quality map that is updated at each
development stage is necessary. We use the example in Figure 3.2 to illustrate a well location
sampling approach based on the quality map. For this example, we assume P1, P2, I1 and I2 are
two producers and the two injectors that are planned to be drilled at the initial stage, and the
location of 4 infill wells that are planned to be drilled in the 3 remaining stages are uncertain.
Figure 3.2(a) illustrates the quality map at the end of the first stage. Note that because minimum
Chapter 3: Optimization under Uncertainty in Future Operations

72

well distance criterion is honored for taking samples for future infill well locations, the quality
value within a distance from the existing wells (P1, P2, I1 and I2) are set to be zero. A simple
thresholding scheme (or clustering algorithm) can be applied to the quality map to keep only
important regions (Figure 3.2(b)). For each potential region (dark red) in Figure 3.2(c), a few
locations are randomly selected and used to generate the average NPV for that region. These
average NPVs can be used to assign probabilities to each region; that is, regions with higher
average NPV values (red regions in Figure 3.2(d)) are more likely to be selected for drilling infill
wells in the earlier stages of the development. To obtain sampling regions for two wells, a
sequential approach is used where for each sample infill well location a new quality map is
generated for the second well and the above procedure is repeated. This process is continued until
the desired number of samples for all stages of field development are obtained.

Figure 3.2: (a) Quality map after drilling two injectors (I1 & I2) and two producers (P1 & P2); (b) Smoothed-out
quality map; (c) Clustered quality map to identify promising drilling locations; (d) Labelled potential drilling regions
based on quality map.

For well control uncertainty, because operational control settings are continuous variables
incorporating the uncertainty in future controls can be accomplished by defining a probabilistic
representation of plausible future controls and sampling from it. In this chapter, we generate the
plausible control strategies for each development scenario (number and locations of wells). To this
end, we define a nominal control trajectory (in this chapter we used the optimized control trajectory
Chapter 3: Optimization under Uncertainty in Future Operations

73

for each scenario) and generate random control perturbations, with specified variance, to account
for uncertainty in future controls.
Based on the above discussion and depending on the type of uncertainty and decision
variables in field development optimization, two variants of Equation (3.1) can be formulated. In
Variant I, the uncertain variables include only the number of infill drilling (𝜔 [𝑡 +1:𝑇 ]
) and the
uncertainty in their locations and controls (𝑢̃
[𝑡 +1:𝑇 ]
,𝑞̃
[𝑡 +1:𝑇 ]
) is removed by finding optimal
location and control for each drilling scenario. In Variant II, the number and locations and controls
of infill drilling for the remaining horizons (𝜔 [𝑡 +1:𝑇 ]
,𝑢̃
[𝑡 +1:𝑇 ]
,𝑞̃
[𝑡 +1:𝑇 ]
) remain uncertain.

3.3 Optimization Solution Approach
In stochastic field development, at the current optimization stage future infill drillings are
modeled as random parameters. Instead of optimizing future decisions, the decision maker focuses
on current decisions given a range of scenarios that might take place in the future. In reservoir
development, the reservoir life is divided into a finite number of horizons (stages). At the
beginning of each horizon, a decision regarding the number of infill wells at that stage is made and
the optimization scheme is implemented to find optimal locations and controls for those wells
while accounting for the uncertainty in drilling and operations in the remaining horizons. For each
stage Equation (3.3) can be used to find optimal controls and locations for the current time
decision variables:
𝑚𝑖𝑛 𝑥 𝑡 {𝑓 𝑡 (𝑥 𝑡 )+∑𝑝 𝑠 𝑆 𝑠 =1
∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
}

(3.3)

where 𝜃 =(𝜔 [𝑡 +1:𝑇 ]
,𝑢̃
[𝑡 +1:𝑇 ]
,𝑞̃
[𝑡 +1:𝑇 ]
) and 𝜃 2
𝑠 ,𝜃 3
𝑠 ,…,𝜃 𝑇 𝑠 are the random data process that
represent the uncertainty in the number of future infill wells and their corresponding locations and
Chapter 3: Optimization under Uncertainty in Future Operations

74

control trajectories for each scenario, 𝑥 𝑡 =[𝑢 𝑡 ,𝑞 𝑡 ]∈ℝ
𝑛 𝑡 ,𝑡 =1,…,𝑇 are the decision variables
(well locations and control settings at the current stage), 𝑓 𝑡 (𝜃 𝑡 ) is the cost function for stage 𝑡 and
depends on the decision variables and random outcomes for that stage. In optimization with
recourse, at each stage once the realization of the random variables (number of new wells) is
known, a recourse decision (well locations and well controls for that stage) will be made based on
the results obtained from past decisions (current reservoir state).
To solve the optimization problem at each stage, we adopt a sequential well location and
control optimization. The framework is initialized with well placement optimization using fixed
control setting. Once converged, the well locations are used to start the well scheduling
optimization solution. The cycle of well placement and drilling time optimizations is repeated until
no improvement in the objective function is observed. We implement the SPSA algorithm (see
Appendix A) for well placement optimization, which can be formulated as follows:
𝑚𝑖𝑛 𝑢 𝑡 𝐽 ={𝑓 𝑡 (𝑢 𝑡 ,𝑞 𝑡 0
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
}
𝑠 .𝑡 . 𝑢 𝑡 ∈𝒰 t

(3.4)
where 𝒰 t
is the feasible set for well locations of the stage 𝑡 . Table 2.4 shows the pseudo-
code used to implement the integer SPSA. Once, the well placement optimization is converged,
well control optimization is performed with the optimal locations from the previous well placement
optimization (𝑢̂
1
). For the well control minimization, we use a quasi-Newton method with
gradients generated with the adjoint method and the approximate Hessian calculated by BFGS
method.
𝑚𝑖𝑛 𝑞 𝑡 𝐽 ={𝑓 𝑡 (𝑢̂
t
,𝑞 𝑡 )+∑𝑝 𝑠 𝑆 𝑠 =1
∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
}
(3.5)
Chapter 3: Optimization under Uncertainty in Future Operations

75

At each iteration, the decision variable 𝑞 t
is updated as follows:
𝑞 𝑡 𝑘 +1
=𝑞 𝑡 𝑘 −𝛼 𝑘 𝐵 𝑘 −1
𝛻 𝑞 𝑡 𝐽 𝑘
𝛻 𝑞 𝑡 𝐽 =𝛻 𝑞 𝑡 {𝑓 𝑡 (𝑢̂
𝑡 ,𝑞 𝑡 )+∑𝑝 𝑠 𝑆 𝑠 =1
∑ 𝑓 𝑡 (𝜃 𝑡 𝑠 )
𝑇 𝜏 =𝑡 +1
}=𝛻 𝑞 𝑡 𝑓 𝑡 (𝑢̂
𝑡 ,𝑞 𝑡 )
(3.6)

For each stage, the cycle of well control and well placement optimization is repeated until
no further improvement in the objective function is observed. The same procedure is repeated to
find optimal 𝑢̂
𝑡 +1
and 𝑞̂
𝑡 +1
in the second stage and thereafter.
Before presenting our numerical results, we reiterate that in optimization with recourse, the
reservoir model is typically updated using incoming observations. The model updating aspect can
be conveniently incorporated to form a closed-loop implementation. However, in this chapter, we
focus on the stochastic optimization problem without including model updating.
3.4 Numerical Experiments
In this section, two sets of numerical examples are presented to compare the significance
of accounting for uncertainty in future development and operation scenarios. The first set of
examples (Case Study I) are based on a single (6
th
) layer of the SPE10 model to investigate the
significance of well location and control uncertainties for an infill well. The second set of examples
(Case Study II) include a two-stage field development optimization in which well placement and
well control optimization are implemented sequentially under different assumptions for future
infill drilling scenarios. The second example is based on PUNQ-S3 reservoir model with 5 layers.
3.4.1 Case Study I: Uncertainty in future infill well location and control settings
In this set of examples, 2 injectors and 4 producers at are considered at the initial phase of
the project and a single infill well is planned after 6 years of production. The goal of these examples
is to evaluate the role of uncertainty in the location and control settings of the single infill well.
Chapter 3: Optimization under Uncertainty in Future Operations

76

3.4.1.1 Base (Nominal) case
For the base case, the location and control settings of 2 injectors and 5 producers are
optimized, where the fifth well (P5) is an infill well that comes online after 6 years of production.
The well control trajectories are divided into 5 evenly spaced increments (control time steps) over
the simulation time of 10 years. The reservoir model description is provided in Table 3.1.
Table 3.1 : Reservoir model paramters for SPE10 model
Number of the grid cells 220×60×1
Grid cell dimensions 30 ft× 15 ft× 50 ft
Oil Price $50/𝑏𝑏𝑙
Water injection and production cost $10/𝑏𝑏𝑙
Fluid Phases Oil-Water
Simulated reservoir life cycle 10 years
Injectors control mode Total water injection rate
Producers control mode Total fluid production rate

Figure 3.1 illustrates the evolution of objective function with respect to optimization
iteration.

Figure 3.3: Evolution of objective function for the base case by implementing sequential well placement and well
control optimization

Figure 3.4 shows the optimal configuration as well as the final saturation map after 10
years of production. Additionally, the optimized control trajectories for producers and injectors
are displayed in Figure 3.5. The optimized NPV value in this case is $123.9MM (base case), and
Chapter 3: Optimization under Uncertainty in Future Operations

77

the cumulative water injection, oil and water production are 9.1, 5.1, and 4.0 MMSTB,
respectively.

Figure 3.4: (left) Optimized well configuration for 10 years of production; the injectors and producers are marked
with (×) and (•), respectively (P5 is an infill well scheduled to be drilled after 6 years); (right) saturation map after
10 years

Figure 3.5: Optimized 5-step control trajectories of the producers and injectors in base case

3.4.1.2 Uncertainty in future well locations
In this case, the location of the future infill well is regarded as a random variable while its
control settings are considered as decision variables. For this problem 50 samples based on the
procedure depicted in Figure 3.2 are taken for the location of P5 to account for its location
uncertainty. Figure 3.6 shows the objective function change at each iteration while Figure 3.7
depicts the optimal well configuration for the initial two injectors and four producers, as well as
possible future locations for P5 and the final saturation map after 10 years of production for one
of the realizations. The optimized control time trajectories for producers and injectors are
displayed in Figure 3.8. The expected NPV over all 50 realizations in this case is 𝐸 𝑢 5
[𝑁𝑃𝑉 ]=
$ 117.5 𝑀𝑀 . To evaluate the robustness of the solutions obtained in the above cases, a new set of
random locations for P5 are generated (Figure 3.9), and the well controls of all the wells are
optimized for years 6-10 in each case. Figure 3.10 shows the resulting NPV (𝑓 1
+𝑓 2
) CDF plots.
The mean value of NPV for the nominal and stochastic solutions are $106.1𝑀𝑀 and $117.0𝑀𝑀 ,
Chapter 3: Optimization under Uncertainty in Future Operations

78

respectively, showing an average improvement of 10% in the stochastic case. For two of the
samples (#34 and #38), the NPV values of the nominal case is larger than their corresponding
stochastic cases. A closer investigation showed that the well locations in these realizations happen
to be very close to the location used in the base case (Figure 3.4).

Figure 3.6: Evolution of objective function for location uncertainty case by implementing sequential well placement
and well control optimization

Figure 3.7: (left) Optimized well configuration for 10 years of production; the injectors and producers are marked
with (×) and (•), respectively, the realizations for possible P5 locations are marked with (o) (P5 is an infill well
scheduled to be drilled after 6 years); (right) saturation map after 10 years for one the realizations.

Figure 3.8: Optimized 5-step control trajectories of the producers and injectors when the location of the infill well
(P5) is an uncertain variable

Chapter 3: Optimization under Uncertainty in Future Operations

79

Figure 3.9: 50 random realizations generated for the location of P5

Figure 3.10: (left) NPV values for each 50 realizations of Figure 19 calculated based on base case (nominal values)
and uncertain location example (stochastic values); (right) empirical cumulative density function for NPV values of
nominal and stochastic cases.

3.4.1.3 Uncertainty in future well controls
For this example, the future (second stage) control trajectories of all the wells (year 6-10)
are regarded as random variables while their locations are optimized. Figure 3.11 shows the
evolution of the objective function during optimization in this case while Figure 3.12 illustrates
the optimal well configurations as well as the final saturation map after 10 years of production, for
one of the control realizations.
Chapter 3: Optimization under Uncertainty in Future Operations

80

Figure 3.11: Evolution of objective function for control uncertainty case by implementing sequential well placement
and well control optimization.

Figure 3.12: (left) Optimized well configuration for 10 years of production (P5 is an infill well scheduled to be
drilled after 6 years) when the second stage control trajectories are uncertain; (right) saturation map after 10 years
for one of the control realizations.

In Figure 3.13, 20 sample realizations out of the final 50 realizations for control scenarios
are shown. Comparing the optimal well configuration with the base case (Figure 3.4 and Figure
3.12), it can be observed that when future controls are uncertain, the infill well (P5) is placed
farther from injectors, possibly to avoid early water breakthrough. The expected NPV over all 50
realizations for this case is 𝐸 𝑞 [𝑁𝑃𝑉 ]=$ 116.7 𝑀𝑀 . To evaluate the robustness of the solution of
base case and the uncertain control example, a new set of random controls are generated (15
realizations are shown in Figure 3.14) and the location of infill well is optimized for each
realization. Figure 3.15(left) shows the resulting NPV (𝑓 1
+𝑓 2
) plots, which shows that on
average the NPV value for the nominal and stochastic cases are $102.6𝑀𝑀 and $115.0𝑀𝑀 ,
showing a 12% improvement.

Chapter 3: Optimization under Uncertainty in Future Operations

81

Figure 3.13: Optimized first stage (T0:T6) and uncertain second stage (T6:T10) control trajectories when the future
control trajectories are posed as random variables.

Figure 3.14: New set of random realizations generated for second stage’s control trajectories for year 6-10 for all 7
wells (5 producers and 2 injectors).

Figure 3.15: (left) total NPV values for each 50 realizations of Figure 24 calculated for base case (nominal values)
and uncertain location example (stochastic values); (right) empirical cumulative density function for NPV values of
nominal and stochastic cases.

3.4.1.4 Uncertainty in future well locations and controls
Lastly, we consider a case in which the second stage control trajectories of all the wells
(year 6-10) as well as the location of the infill well (P5) are regarded as random variables while
the locations of the initial 2 injectors and 4 producers and their control trajectories for years 0-6
are optimized. To keep the computational cost at a reasonable level, 20 scenarios for well location
and 20 scenarios for well control trajectories were generated (leading to a total of 400 scenarios).
Figure 3.16 shows the evolution of the objective function for this example while Figure 3.17
(right) illustrates the optimal configuration for the initial two injectors and four producers as well
Chapter 3: Optimization under Uncertainty in Future Operations

82

as possible future locations for P5. The final saturation map after 10 years of production for one
of the realizations is shown in Figure 3.17 (left).

Figure 3.16: Evolution of objective function for location & control uncertainty case by implementing sequential
well placement and well control optimization.

Figure 3.17: (left) Optimized well configuration; the initial injectors and producers are marked with (×) and (•),
respectively, the 20 realizations for possible P5 locations are marked with (o) (P5 is an infill well scheduled to be
drilled after 6 years); (right) saturation map after 10 years for one the realizations.

The control scenarios for one of the well location realizations are depicted in Figure 3.18.
In this case, the expected NPV over all 400 realizations is 𝐸 𝑢 ,𝑞 [𝑁𝑃𝑉 ]=$ 114.8 𝑀𝑀 . To evaluate
the solution robustness against well controls and location, 20 new random well locations are
generated and for each set another 20 realizations for control settings are considered (a total of 400
scenarios). For each of the location realizations, the future controls are optimized and for each
control realization the location of the infill well is optimized. Figure 3.19 shows the CDF plots for
the nominal and stochastic solutions. The average value of NPV for the nominal case is equal to
$99.2𝑀𝑀 while the mean value for the stochastic case is equal to $112.7𝑀𝑀 , indicating on
average 14% improvement.
Chapter 3: Optimization under Uncertainty in Future Operations

83

Figure 3.18: Optimized first stage (T0:T6) and uncertain second stage (T6:T10) control trajectories when the future
control trajectories and well locations are posed as random variables.

Figure 3.19: empirical cumulative density function for NPV values of nominal and stochastic cases.

3.4.2 Case Study II: PUNQ-S3 Model
In this section, a two-stage well placement and control optimization problem is presented
in order to demonstrate the significance of incorporating the uncertainty in future infill drilling and
operations in formulating the optimization problem. This example is based on the PUNQ-S3
reservoir model. We will compare the results of perfect information, stochastic, and deterministic
solutions with the (No Drilling) assumption. For the stochastic solution, we implement two
variants: Variant I, in which only the number of future infill wells is uncertain and their future
location and controls are optimized; Variant II, where in addition to number of infill wells their
location and operational control settings are also random variables.
For this example, 2 injectors and 3 producers are considered at the beginning and the
number of future infill wells is uncertain with three possible outcomes: 𝜔 1
=0,𝜔 2
=1, 𝜔 3
=2,
Chapter 3: Optimization under Uncertainty in Future Operations

84

each with equal probability of occurrence 𝑝 𝑠 =
1
3
; 𝑠 =1,2,3. General information about the
reservoir model is provided in Table 3.2.
Table 3.2: Reservoir model parameters
Number of the grid cells 19×28×5
Number of active cells 1,761
Grid cell dimensions 100 ft× 100 ft× 50 ft
Initial water saturation 0.05
Oil Price $50/bbl
Water injection and production cost $10/bbl
Annual Discount rate 0
Fluid Phases Oil-Water
Simulated reservoir life cycle 10 years
Injectors control mode Total water injection rate
Producers control mode Total fluid production rate

3.4.2.1 Case 1: Disregarding Future Drilling
In this approach the decision variables corresponding to the current time-step are optimized
without accounting for possible future infill drilling opportunities. Therefore, in the first stage the
decision variables are the locations and controls of the first-stage wells (3 producers and 2
injectors). In the second stage after the revelation of the number of infill wells, a new optimization
is solved to find the optimal location of the new wells as well as the control settings of all wells.
Figure 3.20 shows the optimal configuration and the corresponding final oil saturation plots for
each layer of the reservoir after 10 years of production, and for different number of future infill
well outcomes (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2). Figure 3.21 shows the optimal control
trajectories for each drilling scenario (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2) where the solid lines
represent the initial optimal trajectories (before future infill drilling are known). The dashed and
dotted lines represent the optimal trajectory for the second 5 years of the project when one and two
Chapter 3: Optimization under Uncertainty in Future Operations

85

infill wells are introduced, respectively. Table 3.3 (Case 1) summarizes the performance of the
final solutions for this case.

(a) (b) (c)

Perm. map

Layer 1 Sat.

Layer 2 Sat.

Layer 3 Sat.

Layer 4 Sat.

Layer 5 Sat.

Figure 3.20: Results for case 1 field development (well location and operational control) optimization problem where
the uncertainty in future infill drillings is disregarded; top: optimal well configuration, bottom: saturation distribution
for each individual layer after 10 years of production for (a) No infill well (𝜔 1
=0), (b) one infill well (𝜔 2
=1), (c)
two infill wells (𝜔 3
=2).

Chapter 3: Optimization under Uncertainty in Future Operations

86

Figure 3.21: Optimal control trajectories of the wells for case 1 where the possibility of future infill drillings is
neglected; solid lines are the control trajectories for the 3 producers and 3 injectors obtained at the beginning of the
project without accounting for the future infill well; dashed lines are the optimal control trajectories for the second 5
years of the project when it is decided to drill one infill well; dotted lines are the optimal control trajectories for the
second 5 years of the project when it is decided to drill two infill wells.

Table 3.3: Results of two-stage field development and control optimization
Case 1: Disregarding Uncertainty
NPV

($ MM)
Water Injected
(MMSTB)
Oil Produced
(MMSTB)
Water Produced
(MMSTB)
Scenario 1: ω
1
=0 475.35 26.35 16.71 9.64
Scenario 2: ω
2
=1 478.24 26.35 16.75 9.60
Scenario 3: ω
3
=2 480.25 26.35 16.79 9.56
Case 2: Perfect Information
Scenario 1: ω
1
=0 475.35 26.35 16.71 9.64
Scenario 2: ω
2
=1 536.92 26.35 17.73 8.62
Scenario 3: ω
3
=2 548.21 26.35 17.92 8.43
Case 3: Stochastic Solution I
Scenario 1: ω
1
=0 471.25 26.35 16.64 9.71
Scenario 2: ω
2
=1 531.41 26.35 17.64 8.71
Scenario 3: ω
3
=2 545.60 26.35 17.88 8.47
Case 4: Stochastic Solution II
Scenario 1: ω
1
=0 463.28 26.35 16.50 9.85
Scenario 2: ω
2
=1 522.06 26.35 17.48 8.87
Scenario 3: ω
3
=2 531.84 26.35 17.65 8.70

3.4.2.2 Case 2: Perfect Information (PI)
In this approach, the decision maker does not take any action unless the uncertainty is
resolved, which is not realistic, but severs as an upper bound for the stochastic solution. Figure
3.22 shows the optimal configuration and the final oil saturation plots for each layer after 10 years
of production, and for different infill well outcomes (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2). Figure
3.23 shows the optimized control trajectories for each drilling scenario while Table 3.3 (Case 2)
summarizes the outcomes corresponding to this solution approach.
Chapter 3: Optimization under Uncertainty in Future Operations

87

(a) (b) (c)

Perm. map

Layer 1 Sat.

Layer 2 Sat.

Layer 3 Sat.

Layer 4 Sat.

Layer 5 Sat.

Figure 3.22: Results for perfect information optimization (case II); top: optimal well configuration, bottom:
saturation distribution for each individual layer after 10 years of production for (a) No infill well (𝜔 1
=0), (b) one
infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

Chapter 3: Optimization under Uncertainty in Future Operations

88

(a)
(b)
(c)

Figure 3.23: Optimal control trajectories of Perfect Information optimization (case II) for (a) No infill well (𝜔 1
=
0), (b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

3.4.2.3 Case 3: Stochastic Solution (Variant I)
In variant I of the Stochastic solution only the number of future infill wells (𝜔 ) is
considered as a random variable and future well locations and controls are optimized. Figure 3.24
shows the solution of the stochastic approach for all 3 scenarios (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=
2) while Figure 3.25 depicts the optimal control trajectories for each case (solid lines). The
numerical values for the performance of this case are summarized in Table 3.3 (Case 3).

Chapter 3: Optimization under Uncertainty in Future Operations

89

(a) (b) (c)

Perm.
map

Layer 1
Sat.

Layer 2
Sat.

Layer 3
Sat.

Layer 4
Sat.

Layer 5
Sat.

Figure 3.24: Results for stochastic (here-and-now) optimization (case 3- variant I); top: optimal well configuration,
bottom: saturation distribution for each individual layer after 10 years of production for (a) No infill well (𝜔 1
=0),
(b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

Chapter 3: Optimization under Uncertainty in Future Operations

90

(a)
(b)
(c)

Figure 3.25: Optimal control trajectories of stochastic optimizations (case 3 & case 4) for (a) No infill well (𝜔 1
=
0), (b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).[solid lines for variant I (case 3) and dashed lines for
variant II (case 4) of the stochastic programming].

3.4.2.4 Case 4: Stochastic Solution 2 (Variant II)
In variant II, the number of future infill wells and their corresponding locations and control
settings are considered as random variables. For each of drilling scenarios in this case 20 samples
of future well locations and 20 samples of control settings for each well configuration are used,
leading to a total of 820 (𝑆 =20+2×20×20=820) scenarios. Figure 3.26 shows the top
view of the first layer of the reservoir with the optimal first-stage well locations (Prod1, Prod2,
Prod3, Inj1 and Inj2) and the final 20 future well location samples for Scenario 2 and 3 (𝜔 2
=
1, 𝜔 3
=2), shown with empty circles. Figure 3.27 displays sample control trajectories for years
6-10 for one of the location samples. In Figure 3.28, the results from the Variant II of the stochastic
solution for all 3 scenarios (a: 𝜔 1
=0, b: 𝜔 2
=1, c: 𝜔 3
=2) are shown. Comparing Figure 3.28
and Figure 3.24, the final saturation map shows a better sweep for variant I, which is also
confirmed by the numerical values reported in Table 3.3. This is expected as in Variant I the future
well locations and controls are optimized while in Variant II they are modeled as uncertain
variables. However, Variant I is not robust against changes in future well locations and operations.
Chapter 3: Optimization under Uncertainty in Future Operations

91

Figure 3.25 (dashed lines) depicts the optimal control trajectories for Variant II of the stochastic
solution.

(a) (b) (c)
Figure 3.26: Final location realizations for stochastic (here-and-now) optimization (case 4- variant II) for (a) No infill
well (𝜔 1
=0), (b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2). the injectors and producers are marked with
(×) and (•), respectively, the realizations for possible future well locations are marked with (o).

(a)
(b)
(c)

Figure 3.27: Realizations of control trajectories for second stage (year 6-10) of case 4 for (a) no infill well (𝜔 1
=
0) (b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2) for one of the location realizations.

Chapter 3: Optimization under Uncertainty in Future Operations

92

(a) (b)
(c)
Perm.
map

Layer 1
Sat.

Layer 2
Sat.

Layer 3
Sat.

Layer 4
Sat.

Layer 5
Sat.

Figure 3.28: Results for stochastic (here-and-now) optimization (case 4- variant II); top: optimal well configuration,
bottom: saturation distribution for each individual layer after 10 years of production for (a) No infill well (𝜔 1
=0),
(b) one infill well (𝜔 2
=1), (c) two infill wells (𝜔 3
=2).

Chapter 3: Optimization under Uncertainty in Future Operations

93

Figure 3.29 compares the NPV values for Cases 1-4 relative to the perfect information
solution (being 1 or 100%). For all three scenarios (𝜔 ={0,1,2}), the perfect information
approach provides the best NPV values because it is based on perfect knowledge of the scenarios
(which is ideal but not realistic). The No Drilling case provides the lowest NPV values in all but
the first drilling scenario where no infill well is drilled (i.e., the underlying assumption of the
scenario). The two stochastic formulations have a reasonably high NPV values for all 3 scenarios,
indicating robustness against uncertainty in future drilling activities. For the second drilling
scenario (𝜔 2
=1), the stochastic solution I results in 11% increase in NPV compared to No
Drilling formulation. This difference increases to 13% for the last drilling scenario (𝜔 3
=2). The
same percentages for stochastic solution II formulation compared to No Drilling formulation is
9% for the second scenario (𝜔 3
=1) and 11% for the last scenario (𝜔 3
=2). The results suggest
that as the number of infill wells increases, the NPV value of the No-Drilling solution increasingly
deviates from stochastic solution.

Figure 3.29: Comparing the final normalized NPV values No Drilling (Disregarding the uncertainty), stochastic
(here-and-now) solution I and II and Perfect Information optimization.

To evaluate the robustness of the perfect information and the stochastic solutions to control
and location uncertainties, a new set of random realizations for each of them were generated. For
each new control/location realization, the location/control for the second stage are optimized.
Chapter 3: Optimization under Uncertainty in Future Operations

94

Figure 3.30 displays the new random location samples for the second (𝜔 2
=1) and third scenario
(𝜔 3
=2). The NPV CDF plots for the Perfect Information and Stochastic Solution Variants I and
II are shown in Figure 3.31 The results show that for the second drilling scenario (𝜔 2
=1), the
mean value of NPV for Perfect Information and Stochastic Solution I and II are $415.9MM,
$429.4MM and $463.4MM, respectively. The average NPV of the Stochastic Solution II is 8%
and 11% higher than those for Stochastic Solution I and Perfect Information. For the third drilling
scenario (𝜔 3
=2), the average NPV for Perfect Information and Stochastic Solutions I and II are
$422.9MM, $423.1MM and $466.7MM million dollars, respectively. The average NPV for the
Stochastic Solution II is 10% higher than those of Stochastic Solution I and Perfect Information
approaches, indicating the robustness of the Stochastic Solution II against uncertain future well
locations.

(a) (b)
Figure 3.30: 25 random realizations generated for the location of (a) one infill well (𝜔 2
=1), (b) two infill wells
(𝜔 3
=2).

Chapter 3: Optimization under Uncertainty in Future Operations

95

(a) (b)
Figure 3.31: Empirical cumulative density function for NPV values of perfect information and stochastic solution
I&II for location realizations of Figure 3.30 (a) one infill well (ω
2
=1), (b) two infill wells (ω
3
=2) .

Figure 3.32 shows 15 realizations (out of 50) of new random controls for the second with
𝜔 2
=1 (a) and third scenario with 𝜔 3
=2 (b) while Figure 3.33 shows the NPV CDF plots for
the Perfect Information and Stochastic Solution I and II after optimizing the infill well locations
for each realization. In Figure 3.33, for the second drilling scenario (𝜔 2
=1), the mean value of
NPV for Perfect Information and Stochastic Solution I and II are $415.9MM, $429.4MM and
$463.4MM, respectively, showing on average 8% and 11% higher NPV for Stochastic solution II
relative to the Stochastic Solution I and Perfect Information approaches, respectively. For the third
drilling scenario (𝜔 3
=2), the mean value of NPV for Perfect Information and Stochastic Solution
I and II are $422.9MM, $423.1MM and $466.7MM, respectively, indicating an average NPV
improvement of approximately 10% over Stochastic Solution I and Perfect Information
approaches, which clearly shows the robustness of the Stochastic Solution II against uncertain infill
well control settings.

Chapter 3: Optimization under Uncertainty in Future Operations

96

(a)
(b)
Figure 3.32: Random samples for control trajectories for the second stage (year 6-10) for (a) one infill well (ω
2
=
1), (b) two infill wells (ω
3
=2) .

Figure 3.33: Empirical cumulative density function for NPV values of perfect information and stochastic solution
I&II for control realizations of Figure 3.32 (a) one infill well (ω
2
=1), (b) two infill wells (ω
3
=2) .

3.5 Conclusion
In this chapter, we present a stochastic reservoir optimization approach under uncertainty
in future field operation parameters, namely new well locations and controls. The motivation
behind the proposed framework is that oilfields undergo development and operation activities
throughout their life-cycle that are typically not predictable upfront. Disregarding future
development and operations in optimizing current well configurations and well control settings is
likely to result in suboptimal solutions. To account for future development and operation
uncertainty, we developed a stochastic formulation in which future operations are incorporated as
uncertain variables. We presented two variations of the stochastic formulation. In the first variation
Chapter 3: Optimization under Uncertainty in Future Operations

97

(Variant I), we modelled the number of future infill wells as random variables and optimized their
corresponding future location and operational control settings. In the second variation (Variant II),
in addition to the number of future infill wells, their future location and control trajectories were
represented as random variables. To do so, we discussed frameworks to generate plausible
realizations for future well locations and controls to account for their associated uncertainties.
Two case studies were presented to evaluate the performance of the developed approach
under known (Case Study I) and uncertain (Case Study II) number of future wells. The first case
was used to assess the effect of uncertainty in future operations. The results were compared with
a base (nominal) case where the uncertainty in future development strategies was unaccounted for,
where significant improvements in production performance were observed in the stochastic
approach. The second case considered uncertainty in future drilling and operation parameters. In
this case, the results for two stochastic implementations were compared with those obtained from
Perfect Information and No Drilling approaches. In one stochastic solution only the number of
future wells were included as uncertain (Stochastic Solution I), whereas in the other one
(Stochastic Solution II) the number of future wells as well as their locations and control trajectories
are included as uncertain parameters. The results indicate the robustness of Stochastic Solution II
against changes in both future development and operation activities, whereas Stochastic Solution
I only exhibits robustness against future drilling and is sensitivity to changes in future well
locations and controls.
The findings of our research highlight the importance of treating future field development
and operation parameters as uncertain parameter in oilfield optimization. While the focus of this
chapter was on demonstrating robustness against field development uncertainties, the presented
method can also be readily extended to include the uncertainty in the reservoir model (geologic
Chapter 3: Optimization under Uncertainty in Future Operations

98

uncertainty). Generalization to consider geologic uncertainty and a closed-loop formulation with
dynamic model updating involves repeating the optimization procedure after each model updating
step. In such a closed-loop implementation, updatable decision variables (those that are not
irreversible) can be adjusted after each model updating stage. The goal of this chapter was to
demonstrate that without incorporating future development uncertainty in oilfield optimization
problems it is highly unlikely to find optimal solutions as any new development event can
markedly diminish the quality of past optimization solutions. Hence, a major performance loss is
expected if development and operation plans in the future are different from those used during
optimization. The presented stochastic formulations offers a robust approach that for hedging
against changes in future development and operations decisions.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

99

Chapter 4
Closed-Loop Stochastic Oilfield
Optimization under Uncertainty in
Geologic Description and Future
Development Plans
4.1 Introduction
Oil field development projects involve heavy investments and important decisions about
development and operations can have direct impact on economic performance of the asset over its
life cycle. Therefore, there is a significant interest in optimization of the development and
operational strategies to maximize the profit and minimize the cost. However, a major challenge
for the decision makers is that they have to contend with a great deal of uncertainty from different
sources. Uncertainty is inherent to all hydrocarbon prospects and fields and they can be grouped
in three categories from reservoir engineering prospective: geologic uncertainty, economic
uncertainty, and operation/development uncertainty. Geological uncertainty is related to
incomplete information about reservoir description, including reservoir structure, rock and fluid
properties (e.g., distributions of permeability and porosity). Economic uncertainty is also a
significant contributor to field development and project management. It often involves future oil
prices as one of the main drivers. The last source of uncertainty that was introduced for the first
time by Jahandideh and Jafarpour in 2018 is the uncertainty in future development activities (e.g.,
infill drilling). In real-life field development projects, due to high cost of drilling as well as limited
information at the early stages, a small number of wells are drilled and operated for some time to
acquire more information about the reservoir; the collected information then informs decisions
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

100

regarding drilling new wells (infill wells). Therefore, optimization techniques should always
consider the possibility of changes in field configuration, mainly due to the introduction of new
wells.
Two general approaches can be used to include future decision (events) in optimization
problems. The first approach is to include future events as decision variables in the optimization
problem in which all the decisions regarding future drillings as well as current drillings are
optimized based on the current description of the reservoir. The main drawback of this approach
is that the decisions regarding the planning and operations of the entire reservoir life-cycle are
optimized based on current information, which often involves significant uncertainty. However,
knowledge about the reservoir is constantly evolving as information is collected and the
development plans are consistently updated with the new information. Hence, the initial
development decisions are often suboptimal and can constrain field operations for years. In
practice, the inherent uncertainty about the reservoir conditions significantly complicates the
decision-making process in field development. Hence, a more conservative and robust approach is
to model future drilling events as random variables (as opposed to decision variables), where
multiple plausible drilling scenarios are generated to represent the uncertainty in future drilling
decisions. In this approach, at each decision-making stage the decision variables (e.g., new well
locations, controls, etc., as well as controls for existing wells) are optimized while accounting for
the residual uncertainty in future drilling events.
In Chapter 2, we presented, for the first time, a stochastic optimization approach to
account for the uncertainty in future development plans. To evaluate the performance of the
proposed approach and to demonstrate the advantages of incorporating future development events,
different numerical experiments were presented, by considering the uncertainty in the number of
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

101

infill wells and their corresponding drilling time. We observed that the stochastic solution provides
development strategies that hedge against the uncertainty in future developments, while
approaches that do not incorporate the uncertainty in future development plans did not offer
robustness against changes in future development decisions. Moreover, the stochastic solutions
tend to have small loss compared to the solution obtained from a deterministic approach with
perfect knowledge of future development. In this chapter, we extended the method to include the
uncertainty in the reservoir model (geologic uncertainty) as well as the uncertainty in the number
of future infill wells and their respective location and control settings in a closed-loop
implementation with dynamic model updating. The closed-loop implementation involves
repeating the optimization procedure after each model updating step to update the decision
variables (those that are not irreversible) after each model updating stage.
In recent years, Closed-Loop Reservoir Management (CLRM) has received growing
attention among researchers (Jansen et al. 2005; Sarma et al. 2006; Chen et al. 2010; Peters et al.
2010; Shirangi et al., 2015). The CLRM concept combines model-based optimization and data
integration in reservoir engineering application, where the decision variables related to field
development and reservoir production are optimized (and updated) based on the most current
description(s) of the reservoir. The fundamental idea behind CLRM is to ensure that dynamic data
is seamlessly integrated into reservoir management workflow, most importantly reservoir
description, decision-making for the operation and development of the field, are performed based
on up-to-date models. This approach enables us to adjust operational decisions in real-time. When
field configuration remains unchanged, the reservoir is operated based on optimal solutions for a
period of time while reservoir response data is collected and used to update the reservoir
description. The process of optimization entailing model calibration is repeated as new information
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

102

becomes available. In CLRM, reservoir models are updated continuously by assimilating
production and/or hard data, each history matching step is followed by an optimization step where
the decision variables for the remaining expected life of the reservoir are optimized based on the
most recent updated reservoir models. The loop of optimization and data integration is repeated at
each step that data is collected.
History-matching (also called model calibration, data assimilation and model updating) is
used to update reservoir descriptions to generate geological model realizations that are consistent
with static data and can reproduce the observed performance data when used in flow simulation
results. The main objective of the process is to improve the predictive capability of reservoir
models, based on the presumption that a predictive model should be able to provide reasonable
matches to observed historical data. For the model calibration step a popular and computationally
efficient choice is the ensemble Kalman filter (EnKF) (Evensen 1994; Naevdal et al. 2003;
Aanonsen et al, 2009; Evensen 2009) where incoming dynamic data are integrated into an
ensemble of reservoir models. The EnKF uses a single forward reservoir simulation run per
realization and does not require derivative calculations. Ensemble-based data-to-parameter cross-
correlations are computed and used in EnKF to adjust an ensemble of geological models using
observed production data. The filter involves a forecast step to propagate uncertainty and predict
sample statistics (up to second order) for the quantities of interest, and an update step that integrates
the predicted and observed data using the computed statistical information. Some applications of
EnKF in CLRM include (Brouwer et al., 2004; Chen et al. 2009, Wang et al., 2009; Chen et al,
2010). Brouwer et al. (2004) used EnKF for model updating together with a gradient-based method
for production optimization in a CLRM workflow. Chen et al. (2009) applied an ensemble-based
closed-loop optimization method by combining ensemble-based optimization scheme (EnOpt)
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

103

with EnKF. Wang et al. (2009) solved an optimal control problem to determine the operating
conditions that maximize the project NPV using both adjoint and stochastic gradient methods.
They also applied the EnKF for production data integration and model updating. Chen et al (2010)
used an augmented Lagrangian method for production optimization consisting of NPV and all the
equality and inequality constraints, together with covariance-localized implementation of EnKF
for data assimilation.
In real-life field development projects due to high cost of drilling as well as limited
information at the early stages, a few wells are drilled and operated for some time to acquire more
information about the reservoir and based on the collected information, decisions regarding drilling
new wells (infill wells) are made. Therefore, optimization techniques should always consider the
possibility of changes in field configuration, mainly due to the introduction of new wells. Two
general approaches can be used to include future decision (events) in optimization problems. The
first approach is to include future events as decision variables in the optimization problem in which
all the decision regarding future drillings as well as current drillings are optimized based on the
current information about the reservoir. The main drawback of this approach is that all the
decisions regarding the planning and operations of the entire reservoir life-cycle is optimized based
on current information. However, the knowledge about the reservoir is constantly evolving as
information is collected and the development plan is consistently updated with the new information
making initial development decisions suboptimal and suboptimal decisions made early in the field
life may constrain field operations for years. In practice, it is a complex task to make decisions
about future events in conditions of uncertainty. Hence, a more conservative and robust approach
is to model future drilling events as random variables (as opposed to decision variables), in this
approach multiple plausible drilling scenarios are generated to represent the uncertainty in future
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

104

drilling decisions. at each decision-making stage, the decision variables (e.g., location, control,
etc.) for the current-time wells are optimized while accounting for the residual uncertainty in future
drilling events. In Jahandideh and Jafarpour (2018), we proposed a stochastic reservoir
optimization approach under uncertainty in future development plans. Results from several case
studies in that work were used to demonstrate that stochastic treatment of future drilling activities
provides robust decisions that enable hedging against the uncertainty in future developments.
In this chapter, we extended our previous work to include the uncertainty in the reservoir
model (geologic uncertainty) and the uncertainty in number of future infill wells and their
respective locations and control settings in a closed-loop implementation with dynamic model
updating, where the optimization procedure is repeated after each model updating step. In this
closed-loop implementation, updatable decision variables (those that are not irreversible) can be
adjusted after each model updating stage. The loop of optimization and data integration is repeated
at each step that data becomes available. The optimization is performed over multiple geologic
realizations and future development and operation scenarios to account for their respective
uncertainties.
In the remainder of the chapter, we first present the stochastic formulation for field
development and production optimization while accounting for the uncertainty in future infill
drilling scenarios where the number of infill wells and their respective location and controls are
modelled as uncertain variables. In this formulation, the decision variables for the current time are
optimized while accounting for the residual uncertainty in future drilling decisions. We then extend
the formulation to include geologic uncertainty in a CLRM implementation where the geologic
models are consistently updated in order to be consistent with the observed data. We next present
computational results for 2D and 3D examples to demonstrate the potential in improving reservoir
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

105

performance by accounting for the future infill drilling uncertainty. A similar approach was
implemented in Jahandideh and Jafarpour 2018 where only the number of infill wells were
modelled as uncertain variable and the location and control trajectory of future wells were
optimized. However, in this work in addition to number of infill wells, their potential future
location and control trajectory is also uncertain, and this uncertainty is represented by taking
samples from possible future well locations and control settings.
4.2 Mathematical Formulation
In Chapter 2, we derived the multi-stage stochastic formulation for a stochastic process 𝜔
as:
𝑚𝑖𝑛 𝒙 {𝑓 1
(𝑥 1
)+∑𝐸 𝜔 𝑡 [𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 )]
𝑇 𝑡 =2
}

(4.1)

For a discrete random process (𝜔 ), with finite number of scenarios Ω={𝜔 1
,𝜔 2
,…,𝜔 𝑆 },
the problem in Eq. (4.1) can be restated as:
𝑚𝑖𝑛 𝒙 {𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 𝑠 )
𝑇 𝑡 =2
}
(4.2)

For a T-stage stochastic problem with stage-wise independent parameters 𝜔 1
,𝜔 2
,…,𝜔 𝑇
(i.e., the random variable 𝜔 𝑡 +1
is independent of 𝜔 [1:𝑡 ]
=(𝜔 1
,…,𝜔 𝑡 )), the problem can be
extended to account for uncertainty in reservoir input parameters as:

𝑚𝑖𝑛 𝒙 𝐸 𝑚 [𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑆 𝑠 =1
∑𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 𝑠 )
𝑇 𝑡 =2
]
(4.3)
where 𝑚 represents uncertain reservoir properties.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

106

4.2.1 Closed-loop Optimization Workflow
In this work, the decision variables for each stage include the locations and control settings
of the wells scheduled to be drilled for that stage. The reservoir is initially described by a set of
prior geologic realizations. At each optimization step, the problem is solved to find the optimal
decision variables for the existing wells and new infill wells (if any) while accounting for the
geologic uncertainty as well as the uncertainty in number of future infill drillings and their potential
locations and control trajectories. At each time, the reservoir is operated based on optimal solution
for a period time while reservoir response data is collected. Whenever any kind of data (hard data
and/or production data) becomes available, history matching is performed to update the reservoir
models. Every step of model calibration is followed by another step of optimization to also update
the optimal decision variables with the updated reservoir models. If the optimization step coincides
with a development stage (where infill wells are drilled), optimization is implemented to find
optimal locations and control settings for the infill wells as well as the controls for the existing
wells. The pseudo-code for the closed-loop field development optimization under uncertainty in
model input parameters and future development plans is presented in Table 4.1.
Table 4.1: Closed-loop field development optimization pseudo-code.
𝑓𝑜𝑟 𝑡 =1,…,𝑇 (𝑑𝑒𝑣𝑒𝑙𝑜𝑝𝑚𝑒𝑛𝑡 𝑠𝑡𝑎𝑔𝑒 )
𝑘 =0
𝑤 ℎ𝑖𝑙𝑒 𝑐𝑜𝑛𝑣𝑒𝑟𝑔𝑒𝑛𝑐𝑒 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑎 𝑖𝑠 𝑛𝑜𝑡 𝑠𝑎𝑡𝑖𝑠𝑓𝑖𝑒𝑑
𝑚𝑖𝑛 𝑥 𝑢 𝐸 𝑚 𝑡 0
[𝑓 𝑡 (𝑥 𝑡 𝑢 )+∑ 𝑝 𝑠 ∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
𝑆 𝑠 =1
]
𝑚𝑖 𝑛 𝑥 𝑞 𝐸 𝑚 𝑡 0
[𝑓 𝑡 (𝑥 𝑡 𝑞 )+∑ 𝑝 𝑠 ∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
𝑆 𝑠 =1
]
𝑒𝑛𝑑 (𝑤 ℎ𝑖𝑙𝑒 )
𝑘 =1
𝐺𝑒𝑛𝑒𝑟𝑎𝑡𝑒 𝑛𝑒𝑤 𝑟𝑒𝑎𝑙𝑖𝑧𝑎𝑡𝑖𝑜𝑛𝑠 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛𝑎𝑙 𝑡𝑜 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 𝑎𝑡 𝑑𝑟𝑖𝑙𝑙 𝑒 𝑑 𝑤𝑒𝑙𝑙𝑠 → 𝑚 𝑡 1

𝑚𝑖𝑛 𝑥 𝑞 𝐸 𝑚 𝑡 1
[𝑓 𝑡 (𝑥 𝑡 𝑞 )+∑𝑝 𝑠 ∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
𝑆 𝑠 =1
]
𝑓𝑜𝑟 𝑘 =2,…,𝐾 (data collection steps)
𝐷𝑎𝑡𝑎 𝐴𝑠𝑠𝑖𝑚𝑖𝑙𝑎𝑡𝑖𝑜𝑛 → 𝑢𝑝𝑑𝑎𝑡𝑒 𝑚 𝑡𝑘

𝑚𝑖𝑛 𝑥 𝑞 𝐸 𝑚 𝑡𝑘
[𝑓 𝑡 (𝑥 𝑡 𝑞 )+∑ 𝑝 𝑠 ∑ 𝑓 𝜏 (𝜃 𝜏 𝑠 )
𝑇 𝜏 =𝑡 +1
𝑆 𝑠 =1
]
𝑒𝑛𝑑 (𝑓𝑜𝑟 )
𝑚 (𝑡 +1)0
← 𝑚 𝑡𝑘

𝑒𝑛𝑑 (𝑓𝑜𝑟 )
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

107

In Table 4.1, 𝑡 (𝑡 =1,…,𝑇 ) represents the development stage and 𝑚 𝑡 ,𝑘 refers to reservoir
models at stage 𝑡 of development after 𝑘 steps of model calibration. At each development stage
the locations of the wells (𝑥 𝑡 𝑢 ) and their operational control (𝑥 𝑡 𝑞 ) are optimized by using the latest
updated reservoir models (𝑚 𝑡 0
) and a total of 𝑆 future infill drilling scenarios consisting of the
number of future infills and their potential locations and control trajectories. The random process
𝜃 𝜏 =(𝜔 𝜏 ,𝑢̃
𝜏 ,𝑞̃
𝜏 ) represents the uncertainty in the number of future infill wells (ω
τ
) and their
corresponding locations (𝑢̃
𝜏 ) and control trajectories (𝑞̃
𝜏 ) at future stage of development 𝜏 ∈
[𝑡 +1:𝑇 ]. The sequence of well placement and well control optimization is sequentially repeated
until the convergence criterion (minimum changes in the objective functions) is satisfied
(𝑤 ℎ𝑖𝑙𝑒 𝑙𝑜𝑜𝑝 ). After determining the optimal decision variables, the wells are drilled and hard data
(e.g. measured values of porosity or permeability) are collected at well locations and used to
regenerate new initial geologic realizations conditioned on observations at the drilled well
locations (𝑚 𝑡 1
). These initial realizations are then used to assimilate all dynamic response data up
to the current time step. With the new updated geologic realizations, the well control optimization
must be repeated for the existing wells (including the new well(s)). The reservoir will continue
operating based on the optimal controls (rate here). Meanwhile, performance data (WOR and BHP)
are collected at each data collection step (𝑘 =2,..,𝐾 ), and EnKF is applied to update the models
until the next development stage. The same procedure is repeated at each development stage.
Figure 4.1 shows the same procedure in a flowchart.
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

108

Figure 4.1: Closed-loop oilfield optimization under geologic and future development uncertainties.

4.2.2 Data Assimilation with EnKF
For integration of dynamic data, we use the EnKF data assimilation framework (Evensen
1994). A review of the EnKF application to history matching problem can be found in (Aanonsen
et al, 2009) while details about its formulations, various implementations, and properties can be
found in (Evensen, 2009). In this paper, for brevity, a brief overview of the EnKF equations is
presented, deferring additional details and in-depth treatment of this topic to the cited references
above.
The EnKF was introduced by Evensen (1994) as an efficient Monte-Carlo-based
approximation of the Kalman filter (KF) for nonlinear problems For completeness, we present the
update equations of the EnKF formulation in this section. The predicted observations that are
derived from reservoir simulation can be expressed as:
𝑑 𝑘 𝑖 =ℎ(𝑚 𝑘 −1
𝑖 )+𝑣 𝑘 𝑖 (4.4)
where 𝑚 𝑘 −1
𝑖 refers to the realization 𝑖 of reservoir parameters at time step 𝑘 −1, i.e., before
assimilation of the data at the current time step (k), ℎ
𝑘 is the non-linear measurement operator with
uncorrelated additive measurement error 𝑣 𝑘 with a diagonal covariance 𝐶 𝑣𝑣
, and 𝑑 𝑘 𝑖 is the 𝑖 th
measurement vector at time step 𝑘 . The EnKF update equation for N realizations of the initial
model parameter {𝑚 1
,𝑚 2
,…,𝑚 𝑁 } can be expressed as (Evensen, 2009):
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

109

𝑚 𝑘 𝑖 =𝑚 𝑘 −1
𝑖 +𝐶 𝑚𝑑
𝐶 𝑑𝑑
−1
[𝑑 𝑜𝑏𝑠 ,𝑘 𝑖 −𝑑 𝑘 𝑖 ], (4.5)
where 𝑑 𝑜𝑏𝑠 ,𝑖 (𝑡 𝑘 ) represents the i
th
realization of the perturbed measurements at time 𝑡 𝑘 , and
𝐶 𝑚 𝑑 𝐶 𝑑𝑑
−1
is the Kalman gain matrix, with 𝐶 𝑚𝑑
and 𝐶 𝑑𝑑
defined as the cross-covariance between
data and model parameters and the measurement covariance matrices, respectively. Various
implementation of the EnKF update equation have been proposed in the literature. In this paper,
we use the square root filter (SRF) that was proposed by [Evensen, 2004].

4.2.3 Handling Uncertainty in future Well Locations and Controls
In field development optimization, at early stages both the number of infill wells (𝜔 ) and
their respective locations (𝑢̃) and control settings (𝑞̃) are typically uncertain. Denoting the vector
of uncertain variables as 𝜃 , where 𝜃 =[𝜔 ,𝑢 , ̃𝑞̃]
𝑇 , the multi-stage stochastic programing can be
expressed as:
min
𝑥 1
𝜖 𝜒 1
𝑓 1
(𝑥 1
)+𝐸 𝜃 2
[ min
𝑥 2
𝜖 𝒳 2
(𝑥 1
,𝜃 2
)
𝑓 2
(𝑥 2
,𝜃 2
)
+𝐸 𝜃 3
|𝜃 2
[…+𝐸 𝜃 𝑇 |𝜃 2
,…,𝜃 𝑇 −1
[ min
𝑥 𝑇 𝜖 𝒳 𝑇 (𝑥 [1:𝑇 −1]
,𝜃 [2:𝑇 ]
)
𝑓 𝑇 (𝑥 𝑇 ,𝜃 𝑇 )]]]

(4.6)
Assuming stage-wise independent drilling decisions, the uncertainty in the number of
future infill wells (𝜔 [𝑡 +1:𝑇 ]
) can be formulated as a sequential decision-making problem with
unconditional expectations in multi-stage formulation. On the other hand, characterizing the
uncertainty in future well locations and controls is more complex because the number of possible
scenarios can be extremely large, and the well locations are stage-wise dependent (the well
locations at stage 𝑡 depend on well locations decided at previous stages). Therefore, incorporating
the uncertainty in future infill well locations and control settings (𝑢̃
[𝑡 +1:𝑇 ]
,𝑞̃
[𝑡 +1:𝑇 ]
) is more
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

110

involved. One way to incorporate the uncertainty in well locations is to assign potential locations
to future infill wells. For allocating potential locations to future infill wells, a subset of possible
well site combinations based on reservoir quality maps and a set of constraints such as minimum
interwell spacing or other well configuration constraints can be generated. Note that the potential
locations for future wells must be updated at each iteration as for each iterate (proposed current
well locations) the plausible/promising future well locations are different (and are needed to
evaluate the corresponding realizable NPV values). In optimization with recourse, the updated
future locations are not fixed and can be corrected at each stage using new information and past
decisions.
Cruz et al. (1999) introduced the concept of the quality map, which is a two-dimensional
representation of reservoir response and its uncertainties. The quality map, which is a measure of
“how good” a region of the reservoir is for production, can be used for ranking stochastic
realizations and selecting well locations with fewer simulation runs. In this study, the quality map
is generated based on reservoir properties in each grid bock as follows:

∑ 𝑘 𝑙 ℎ
𝑙 𝜑 𝑙 𝑆 𝑜𝑙
𝑛 𝑙 =1
𝑛
(4.7)
Where 𝑛 is total number of reservoir layers in vertical direction. As mentioned before, the
quality map is a two-dimensional representation that provides an aggregate index of the reservoir
quality over the grid blocks in the vertical direction. While porosity and permeability are static
reservoir properties, saturation is a dynamic quantity and undergoes drastic changes during the
project life, making the quality map time-variant. For multi-stage field development, once a well
is placed in a gridblock with relatively high-quality index value, the entire quality map for the
subsequent stages will change. Therefore, at each iteration, when the locations of the wells at the
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

111

current stage are updated, multiple plausible future well location samples are automatedly
generated for the remaining stages of the development (by simulating the corresponding saturation
profile over the life-cycle of the reservoir). Because well control settings are continuous variables,
their feasible set is infinite. Furthermore, well trajectories cannot be completely arbitrary and
should be constrained based on the production response of the reservoir. Hence, incorporating the
uncertainty in future well trajectories is complicated. One way to simplify the problem is to
represent the uncertainty in well controls based on perturbations around a nominal trajectory,
which is the approach we have adopted in this work.
4.3 Numerical Experiments
In this section, numerical examples are presented to compare the performance of proposed
closed-loop stochastic formulation by accounting for future development plans. The first example
is based on the PUNQ-S3 reservoir model with 5 layers and are designed to demonstrate the
significance of incorporating the uncertainty in future number of wells and geologic uncertainty in
formulating the optimization problem. The second example resembles one layer of SPE10
benchmark model that is discretized into a uniform 60×220×1 grid. A two-stage field development
optimization is implemented for this example in which a closed-loop well placement and well
control optimization are applied sequentially under different assumptions in future infill drilling.
The third example is based on Norne reservoir model with 22 layers, which is derived from a large
hydrocarbon reservoir located in the Norwegian Sea, 200 km west of Norway coast. The model
contains 146×112×22 grid blocks, of which 44,915 blocks are active. A closed-loop multi-stage
field development optimization is performed for the second example, in which well locations and
controls are optimized at each stage while accounting for uncertainty in future drilling decisions.
A two-phase incompressible waterflooding experiment is used in both cases.
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

112

4.3.1 Case Study 1: Two-stage Closed-loop Optimization under uncertainty in
future number of wells
For this example, the reservoir model description was provided in Table 4.2. The
Stochastic Solution (SS) approach is compared with the case in which uncertainty in future infill
drillings is disregarded, i.e., No Drilling (ND), to assess their performance and robustness. The
problem is to find the configuration and control settings for 2 injectors and 2 producers at the
current stage while assuming three future drilling scenarios: 𝜔 1
=0,𝜔 2
=1, 𝜔 3
=2 (0, 1 or 2
infill wells after 5 years), each with equal probability of occurrence 𝑝 𝑠 =
1
3
; 𝑠 =1,2,3.
Table 4.2: Reservoir model parameters for PUNQ-S3 model
Number of the grid cells 19×28×5
Number of active cells 1,761
Grid cell dimensions 100 ft× 100 ft× 50 ft
Initial water saturation 0.05
Oil Price $50/𝑏𝑏𝑙
Water injection and production cost $10/𝑏 𝑏 𝑙
Annual Discount rate 0.1
Fluid Phases Oil-Water
Simulated reservoir life cycle 10 years
Injectors control mode Total water injection rate
Producers control mode Total fluid production rate

4.3.1.1 No Drilling (ND)
In the ND approach, the decision variables corresponding to the current time-step are
optimized (under geologic uncertainty) without accounting for possible future infill drilling
opportunities. In the first stage, the following optimization problem is solved:

𝑥̂
1
=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 1
∈𝒳 1
𝐸 𝑚 1,0
[𝑓 (𝑥 1
)]
(4.8)

where 𝑥 1
=[𝑥 1
𝑢 ,𝑥 1
𝑞 ] represent the decision variables (location and control) of the first-
stage wells (2 producers and 2 injectors); 𝑚 𝑡 ,𝑘 represent the input parameters at development stage
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

113

𝑡 (𝑡 =1 here) after 𝑘 (𝑘 =0 here) steps of model calibration. After drilling the wells, geologic
realizations are calibrated using the hard data collected from the wells and the optimization is
repeated to determine control settings using the new realizations. The reservoir is operated with
the optimal strategy and performance data is collected to update the reservoir models. The updated
models are, in turn, used to optimize the control settings. After step 𝑘 of model calibration
(resulting in 𝑚 1,𝑘 ), the controls of the first stage wells are optimized as following:

𝑥̂
1
𝑞 =𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 1
𝑞 𝐸 𝑚 1,𝑘 [𝑓 (𝑥 1
𝑞 )]
(4.9)

In the second stage, once the drilling scenario is known, an optimization problem is solved
to find the optimal decision variables (𝑥 2
=[𝑥 2
𝑢 ,𝑥 2
𝑞 ]: location and control) for the second stage.
𝑥̂
2
=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 2
∈𝒳 2
𝐸 𝑚 2,0
[𝑓 (𝑥 2
)] (4.10)

where 𝑚 2,0
refers to the latest updated model parameters at second stage of development
(with 0 model calibration steps in the second stage). The same procedure for closed-loop field
development optimization will be implemented in the second stage, where after each step (𝑘 ) of
data collection and model calibration, the control settings of the existing wells for the remainder
of the reservoir’s life is optimized:

𝑥̂
2
𝑞 =𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 2
𝑞 𝐸 𝑚 2,𝑘 [𝑓 (𝑥 2
𝑞 )]
(4.11)

Figure 4.2 shows the (a) logarithmic horizontal permeability and (b) porosity distribution
for the reference PUNQ-S3 model. An ensemble of 100 geologic realizations are generated for the
horizontal permeability and porosity. The realizations use published variogram information for
different layers and are generated using Sequential Gaussian Simulation technique in the SGeMS
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

114

geostatistical software. Figure 4.3 shows the mean of initial ensemble and two realizations for
log-permeability and porosity.

(a) (b)

Figure 4.2: (a) Log-horizontal permeability map (millidarcy), and (b) porosity map for the reference PUNQ-S3
model.

𝐿𝑜𝑔 (𝑘 )

𝜑

(a) mean (b) Sample # 1 (c) Sample #2
Figure 4.3: (a) Initial ensemble mean, and (b) & (c) two realizations from the initial ensemble for 𝐿𝑜𝑔 (𝑘 ) (first
row) and 𝜑 (second row).

Figure 4.4 shows the optimal configuration (shown on reference model) for the two initial
producers (Prod1 & Prod2) and injectors (Inj1 & Inj2). Figure 4.5 illustrates the optimal control
trajectories and predicted reservoir response (BHP for all wells and the watercut in the producers)
for each realization. After drilling the initial 4 wells, hard data will become available and can be
used to update the geologic realizations. This step is implemented by first generating a new
conditional ensemble of models and repeating the model calibration steps. With the new ensemble,
the optimal control trajectories of the wells are updated and applied to the reservoir to collect the
performance data for the next model calibration step. The sequence of model calibration and
optimization is repeated until the end of year 5. Figure 4.6 shows the updated optimal trajectories
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

115

after 6 steps of model calibration at the end of year 5. In this figure, the dashed lines represent the
initial trajectories shown in Figure 4.5 (top).

(a) (b)

Figure 4.4: Optimal well configurations for the initial wells (Prod1, Prod2, Inj1 and Inj2) under No Drilling
assumption shown on reference (a) permeability (b) porosity map.

Figure 4.5: (top) Optimal control trajectories for the initial wells (Prod1, Prod2, Inj1 and Inj2) in the No Drilling
case; (bottom) predicted reservoir performance of the optimal solution over the initial ensemble of 100 geologic
realizations.

Figure 4.6: Optimal control trajectory for the No Drilling case after 5 years with 6 steps of model calibration (solid
line) and initial optimal control trajectory (dashed line).

After 5 years, the decision is made regarding the number of infill wells (we solve the
problem under all three possible scenarios: (1) No Infill well (𝜔 1
=0) (2) one Infill well (𝜔 2
=1)
or (3) two infill wells (𝜔 3
=2). Depending on what scenario is selected, the optimal decision
variables for the remainder of the project (remaining 5 years) are found and the sequence of model-
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

116

updating and control is repeated. Figure 4.7 shows the final mean of permeability and porosity
ensemble at the end of the project as well as the optimal configuration for each drilling scenario.

𝐿𝑜𝑔 (𝑘 )

𝜑

(a) No infill well (b) One Infill Well (c) Two Infill wells
Figure 4.7: (top) final configuration and the mean of final updated (after 10 years) ensemble of geologic realizations
representing logarithm of permeability; (bottom) the mean of final updated (after 10 years) ensemble of geologic
realizations representing porosity for (a) No infill drilling, (b) one infill well, (c) two infill wells scenarios.

Figure 4.8 illustrates the response of the final geologic realizations (cyan color) and the
observed response (red color) from the wells for each drilling scenario. Table 4.3 summarizes the
final solutions for a two-stage closed-loop field development for the ND case.

Figure 4.8: Reservoir response (BHP and WOR) of the wells for each final geologic realization (cyan) and the
observations from the reference model (red) for (a) no infill drilling, (b) one infill well, (c) two infill well scenarios.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

117

Table 4.3: Results of two-stage closed-loop field development optimization
Disregarding Uncertainty
NPV
($ MM)
Water Injected (MMSTB) Oil Produced (MMSTB) Water Produced (MMSTB)
Scenario 1: 𝜔 1
=0 225.7 15.3 10.1 5.2
Scenario 2: 𝜔 2
=1 228.3 15.3 10.1 5.2
Scenario 3: 𝜔 3
=2 230.1 15.3 10.2 5.1
Stochastic solution
Scenario 1: 𝜔 1
=0 220.0 15.3 10.0 5.3
Scenario 2: 𝜔 2
=1 249.6 15.3 10.6 4.6
Scenario 3: 𝜔 3
=2 256.8 15.3 10.8
4.5

4.3.1.2 Stochastic Solution (SS)
In the SS approach, the optimization in the first stage can be expressed as:

𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 1
,𝑥 2
𝐸 𝑚 1,0
[𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑓 2
(𝑥 2
,𝜔 𝑠 )
3
𝑠 =1
]
(4.12)

where 𝑥 1
,𝑥 2
represent the decision variables (well locations and controls) of the first and
second stage, respectively, and 𝑚 𝑡 ,𝑘 denotes the input parameters at development stage 𝑡 after 𝑘
steps of model calibration. Figure 4.9 shows the optimal configuration for the initial wells (shown
on reference model) while Figure 4.10 displays the optimal control trajectories and predicted well
responses for each realization. The same procedure as in the last example is repeated for this case,
where a model calibration step is performed once hard data or performance data become available,
followed by an optimization problem based on the updated models (as before). The main difference
is that in this case all the optimization problems are formulated by accounting for the three infill
drilling scenarios.

(a) (b)

Figure 4.9: Optimal well configurations for the initial wells (Prod1, Prod2, Inj1 and Inj2) and for the Stochastic
Solution approach shown on the reference (a) permeability (b) porosity maps.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

118

Figure 4.10: (top) Optimal control trajectories of the initial wells (Prod1, Prod2, Inj1 and Inj2) for stochastic
solution; (bottom) predicted reservoir performance of the optimal solution over the initial ensemble of 100 geologic
realizations.

After 5 years, when the uncertainty regarding the number of infill wells is resolved, the
optimal location(s) of the new wells as well as the control settings for all existing wells are
optimized for the remainder of the remaining 5 years. Once the optimal well locations are
identified, the model realizations are updated, and the sequence of control optimization and model
updating steps is repeated. Figure 4.11 shows the final mean of permeability and porosity
ensemble at the end of the project as well as the optimal configuration for each drilling scenario
obtained from the SS approach.
𝐿𝑜𝑔 (𝑘 )

𝜑

(a) No infill well (b) One Infill Well (c) Two Infill wells

Figure 4.11: (top) Final well configurations and the updated ensemble mean (after 10 years) of model realizations
representing log-permeability; (bottom) mean of final (after 10 years) ensemble of geologic realizations representing
porosity for (a) no infill drilling, (b) one infill well, (c) two infill well scenarios for Stochastic Solution.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

119

Figure 4.12: Reservoir response (BHP and WOR) of the wells for each final geologic realization (cyan) and the
observations from the reference model (red) for (a) no infill drilling, (b) one infill well, (c) two infill well scenarios.

Figure 4.12 illustrates the response of the final geologic realizations (cyan color) and the
observed response (red color) from the wells for each drilling scenario while Table 4.3 summarizes
the final solutions for a two-stage closed-loop field development using the SS approach. In Figure
4.13, bar plots are to illustrate the normalized NPV values for the ND and SS approaches (for each
scenario, NPVs are normalized w.r.t. highest NPV value for that scenario). Except for the first
scenario, where no infill well is drilled (i.e., the very assumption used by the ND approach), the
NPV values for the ND case are consistently lower that those resulting from the stochastic solution.
Interestingly, the NPV (performance) loss by including future development plans (Figure 4.13,
left) is much smaller than the NPV gain by including robustness (Figure 4.13, middle and right).
For the second drilling scenario (𝜔 2
=1), the SS approach results in 9.3% increase in NPV
compared to the ND formulation. This difference increases to 11.6% for the last drilling scenario
(𝜔 3
=2).

Figure 4.13: NPV performance for No Drilling and Stochastic Solution under each drilling scenario.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

120

4.3.2 Case study 2: Two-stage Closed-loop Optimization under uncertainty in
future number of wells, locations and controls
In this section, a closed-loop two-stage well placement and control optimization problem
is presented in order to demonstrate the significance of incorporating the uncertainty of future infill
drillings in formulating the optimization problem. This example is based on the SPE10 reservoir
model. Figure 4.14(a) shows the logarithmic horizontal permeability for the reference model. A
prior ensemble of 50 geologic realizations are generated for the horizontal permeability and
porosity. The realizations are generated using Sequential Gaussian Simulation technique in the
SGeMS geostatistical software. Figure 4.14(b-d) shows the mean of prior ensemble and two
realizations for horizontal log-permeability. The reservoir model description and parameters used
for computing the NPV objective function is provided in Table 4.4. For this example, the
simulation is performed for 10 years and 2 injectors and 3 producers are planned to be drilled
initially while 14 infill drilling scenarios are used to represent the uncertainty in future
development plans in the second half of the reservoir life (last 5 years). These 14 infill scenarios
are generated based on the assumption that a maximum of 2 injector wells can be drilled in the
second stage and the maximum number of total infill wells in the second stage is set to be equal to
5. Figure 4.15 shows the decision tree for this example where each node represent one possible
drilling scenario in the second stage and the number of producing and injecting infill wells for each
node (drilling possibility) is denoted as [
+𝑛 𝑝𝑟𝑜𝑑 −𝑛 𝑖𝑛𝑗 ] where +𝑛 𝑝𝑟𝑜𝑑 ,−𝑛 𝑖𝑛𝑗 represent the number of
producers and injectors for second stage, respectively. For instance, the scenario [
+3
−2
] represents a
drilling scenario in which 3 producers and 2 injectors are drilled at the end of year 5. The simulator
used in this work is the Matlab Reservoir Simulation Toolbox (MRST), which is developed by the
Computational Geosciences group in the Department of Applied Mathematics at SINTEF ICT
(MRST, 2018a). We will compare the results of proposed stochastic formulation and the
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

121

traditional approach where the possibility of future drilling is disregarded (No Drilling), both under
geologic uncertainty, to assess the performance of them in field development optimization and
their robustness to uncertainty in future drilling events.

(a) (b) (c) (d)

Figure 4.14: (a) Reference Log-horizontal permeability map (millidarcy); (b) mean of prior ensemble for Log-
horizontal permeability maps; (c) sample #1 of prior and (d) sample #2 of prior.

Table 4.4: Reservoir model parameters for second case study
Number of the grid cells 220×60×1
Grid cell dimensions 30 ft× 20 ft× 50 ft
Fluid Phases Oil-Water
Simulated reservoir life cycle 10 years
Injectors control mode Total water injection rate
Producers control mode Total fluid production rate
Oil price 50 $/𝑏𝑏𝑙
Water injection cost 6 $/𝑏𝑏𝑙
Water disposal/recycling cost 6 $/𝑏𝑏𝑙
Discount rate %10
Drilling cost $2×10
6
𝑀𝑀

Figure 4.15: decision tree for a two-stage field development optimization considering 14 future development
possibilities.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

122

4.3.2.1 Disregarding Future Drilling (No Drilling)
Before presenting the solution of the optimization in stochastic framework, in this section
the problem is solved in a closed-loop implementation using the traditional approach in industry
where the goal is to optimize the decision variables corresponding to the current time-step are
optimized (under geologic uncertainty) without accounting for possible future infill drilling
opportunities. In the first stage, the following optimization problem is solved:

𝑥̂
1
=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 1
∈𝒳 1
𝐸 𝑚 1,0
[𝑓 (𝑥 1
)]
(4.13)

where 𝑥 1
=[𝑥 1
𝑢 ,𝑥 1
𝑞 ] represent the decision variables (location and control) of the first-
stage wells (3 producers and 2 injectors in this example); 𝑚 𝑡 ,𝑘 represent the input parameters at
development stage 𝑡 (𝑡 =1 here) after 𝑘 (𝑘 =0 here) steps of model calibration. After drilling the
wells, geologic realizations are calibrated using the hard data collected from the wells and the
optimization is repeated to determine control settings using the new realizations. The reservoir is
operated with the optimal strategy and performance data is collected to update the reservoir
models. The updated models are, in turn, used to optimize the control settings. After step 𝑘 of
model calibration (resulting in 𝑚 1,𝑘 ), the controls of the first stage wells are optimized as
following:

𝑥̂
1
𝑞 =𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 1
𝑞 𝐸 𝑚 1,𝑘 [𝑓 (𝑥 1
𝑞 )]
(4.14)

Figure 4.16 shows the optimal configuration for the three initial producers (P1, P2 & P3)
and two injectors (I1 & I2) shown on the mean of initial ensemble. Note that for model updating
BHP values for all the wells and the WOR in producers are used as dynamic data. After drilling
the initial 5 wells, hard data will become available and can be used to update the geologic
realizations. This step is implemented by first generating a new conditional ensemble of models
and repeating the model calibration steps. With the new ensemble, the optimal control trajectories
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

123

of the wells are updated and applied to the reservoir to collect the performance data for the next
model calibration step. The sequence of model calibration and optimization is repeated until the
end of year 5. Figure 4.17 illustrates the optimal control trajectories at the end of year 5.

Figure 4.16: Optimal well configurations for the initial wells (P1, P2, P3, I1 and I2) under No Drilling assumption
shown on mean of prior ensemble.

Figure 4.17: Optimal control trajectories for the initial wells (P1, P2, P3, I1 and I2) at the end of year 5 after 6 steps
of model updating in the No Drilling case.

In the second stage, once the drilling scenario is known, an optimization problem is solved
to find the optimal decision variables (𝑥 2
=[𝑥 2
𝑢 ,𝑥 2
𝑞 ]: location and control) for the second stage.
𝑥̂
2
=𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 2
∈𝒳 2
𝐸 𝑚 2,0
[𝑓 (𝑥 2
)] (4.15)

where 𝑚 2,0
refers to the latest updated model parameters at second stage of development
(with 0 model calibration steps in the second stage). The same procedure for closed-loop field
development optimization will be implemented in the second stage, where after each step (𝑘 ) of
data collection and model calibration, the control settings of the existing wells for the remainder
of the reservoir’s life is optimized:

𝑥̂
2
𝑞 =𝑎𝑟𝑔𝑚𝑖𝑛 𝑥 2
𝑞 𝐸 𝑚 2,𝑘 [𝑓 (𝑥 2
𝑞 )]
(4.16)

Depending on what scenario is selected, the optimal decision variables for the remainder
of the project (remaining 5 years) are found and the sequence of model-updating and control is
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

124

repeated. Top rows of Figure 4.18 (No Drilling) show the final updated mean of ensemble at the
end of the project as well as the final saturation map and their corresponding normalized Net
Present Value (w.r.t maximum NPV values) for four of the drilling scenarios when the possibility
of the future drilling decisions is disregarded.
(a) 𝑁𝑜 𝐼𝑛𝑓𝑖𝑙𝑙 𝑤𝑒𝑙𝑙 (𝑏 )1 𝑝𝑟𝑜𝑑 +1 𝑖𝑛𝑗 (𝑐 ) 2 𝑝𝑟𝑜𝑑 +2 𝑖𝑛𝑗 (𝑐 ) 4 𝑝𝑟𝑜𝑑 +1 𝑖𝑛𝑗
No Drilling

Ens.
Mean

sat
NPV
0.7470 0.7745 0.8054 0.8181

Stochastic

Ens.
Mean

sat
NPV
0.7183 0.8129 0.9601 1

Figure 4.18: (top) final optimal configuration and the mean of final updated (after 10 years) ensemble of geologic
realizations representing logarithm of permeability and their corresponding saturation map (at year 10) for four drilling
scenarios (a) No Infill well, (b) 1 𝑝𝑟𝑜𝑑 +1 𝑖𝑛𝑗 , (c) 2 𝑝𝑟𝑜𝑑 +2 𝑖𝑛𝑗 and (d) 4 𝑝𝑟𝑜𝑑 +1 𝑖𝑛𝑗 and their corresponding
saturation map (at year 10) for the case where the possibility of future drilling opportunities are unaccounted for (No
Drilling); (bottom) final optimal configuration and the mean of final updated (after 10 years) ensemble of geologic
realizations representing logarithm of permeability for the same four drilling scenarios and their corresponding
saturation map (at year 10) for the case where future drilling possibilities are modelled as uncertain variable (stochastic
approach).

4.3.2.2 Stochastic Solution
In the stochastic approach, the optimization in the first stage can be expressed as:

𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 1
𝐸 𝑚 1,0
[𝑓 1
(𝑥 1
)+∑𝑝 𝑠 𝑓 2
(𝜃 2
𝑠 )
𝑆 𝑠 =1
] (4.17)

where 𝑥 1
represent the decision variables (well locations and controls) of the first stage,
𝑚 𝑡 ,𝑘 denotes the input parameters at development stage 𝑡 (𝑡 =1 here) after 𝑘 (𝑘 =0 here) steps
of model calibration, 𝜃 2
𝑠 =(𝜔 2
𝑠 ,𝑢̃
2
𝑠 ,𝑞̃
2
𝑠 ) is the vector of uncertain variables representing the
uncertainty in the number of infill wells (𝜔 2
𝑠 ) and their corresponding location (𝑢̃
2
𝑠 ) and control
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

125

trajectories (𝑞̃
2
𝑠 )in the second stage. 𝑆 denotes the total number of scenarios for number, location
and control of the future wells. Because it is impractical to account for all possible combination of
future wells and their respective location and control settings, different number of samples were
taken to account for the uncertainty in geologic realization, future well drilling scenarios and their
location and control. Figure 4.19(a) shows the evolution of the objective function in Eq. (4.17)
with different number of samples (50, 100, 300, 500, 1000, 2000 samples), the plot shows that the
optimal final value starts to stabilize at around 500 samples. Figure 4.19(b) illustrates the mean
of coefficient of variation over all iterations for each case. The coefficient of variation becomes
stables at 1000 samples. Figure 4.20(a) shows the initial configuration for the initial 3 producers
(P1, P2 & P3) and 2 injectors (I1 & I2) shown on the mean of initial ensemble and Figure 4.20(b-
d) depict the optimal configuration for 300, 500 and 2000 samples. Note that other initializations
were tested and the depicted one provided the highest value for the objective function. Figure 4.21
illustrates the optimal control trajectories for the first 5 years for 300, 500 and 2000 samples, the
optimal control trajectories of 500 samples is the closest to the 2000 samples case. To implement
a closed-loop sequential well placement and control optimization while accounting for 14 possible
future drilling decisions, the solution from the 2000 samples case is selected to continue the
optimization and model updating for the next 10 years of the project. To do so, starting from the
solution of 2000 samples (Figure 4.20(d) and Figure 4.21 (solid lines)), the same procedure as in
the last example is repeated, where a model calibration step is performed once hard data or
performance data become available, followed by an optimization problem based on the updated
models (as before). The main difference is that in this case all the optimization problems are
formulated by accounting for the future development plans (uncertainty in number of future wells
and their location and control settings).

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

126

(a) (b)
Figure 4.19: (a) Evolution of Objective function with different number of samples for geologic realizations, future
drilling scenarios, future well locations and future control trajectories (b) Mean of coefficient of variations over all
iterations for different number of samples.

Initial config. Optimal config. for 300 samples Optimal config. for 500 samples Optimal config. for 2000 samples

(a) (b) (c) (d)
Figure 4.20: (a) Initial configuration for the stochastic case; Optimal configuration for the stochastic case with (b)
300 samples, (c) 500 samples and (d) 2000 samples for geologic realizations, future drilling scenarios, future well
locations and future control trajectories.

Figure 4.21: Optimal control trajectories of the initial wells (P1, P2, P3, I1 and I2) of the first five years for stochastic
solution for 300 (dashed lines), 500 (dotted lines) and 2000 (solid lines) samples for geologic realizations, future
drilling scenarios, future well locations and future control trajectories.

After 5 years, when the uncertainty regarding the number of infill wells is resolved, the
optimal location(s) of the new wells as well as the control settings for all existing wells are
optimized for the remaining 5 years. Once the optimal well locations are identified, the model
realizations are updated, and the sequence of control optimization and model updating steps is
repeated. Bottom rows of Figure 4.18 (stochastic) show the final mean of permeability ensemble
at the end of the project and the optimal configuration as well as the final saturation map and their
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

127

corresponding normalized Net Present Value (w.r.t maximum NPV) for four of the drilling
scenarios obtained from the stochastic approach.
In Figure 4.22, bar plots are to illustrate the normalized NPV values for the No Drilling
(disregarding the possibility of future infill drilling) and stochastic approaches (for each scenario,
NPVs are normalized w.r.t. highest NPV achieved) for all 14 drilling scenarios. Except for the first
scenario, where no infill well is drilled (i.e., the very assumption used by the No Drilling
approach), the NPV values for the ND case are consistently lower that those resulting from the
stochastic solution. Interestingly, the NPV (performance) loss by including future development
plans (Figure 4.22, left and Figure 4.18, left) is much smaller than the NPV gain by including
robustness (Figure 4.22 and Figure 4.18). The maximum increase is observed for the last scenario
(Figure 4.22, right and Figure 4.18, right) in which 4 producing and one injecting infill wells are
drilled (~%22 increase). Because the No-Drilling approach does not account for future drilling
possibilities, finding good locations for a large number of infill wells becomes increasingly
difficult. The solid and dash lines in Figure 4.22 show the mean of the NPV for all 14 scenarios
in the Stochastic and No-Drilling (i.e., disregarding future drilling opportunities is neglected)
approaches, respectively. As can be seen the mean value of stochastic (here-and-now) approach is
more than %11 higher than the mean value of field development approach without incorporating
the future drilling uncertainty. This difference indicates what is known in the stochastic
optimization literature as the Value of Stochastic Solution (VSS).
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

128

Figure 4.22: NPV performance for No Drilling and Stochastic Solution under each drilling scenario.

4.3.3 Case study 3: Multi-stage Closed-loop Optimization under uncertainty in
future number of wells, locations and controls
In this section, a closed-loop multi-stage well placement and control optimization problem
is presented. This example resembles the Norne field which is 200 km offshore in Norwegian
Continental Shelf and was discovered in Dec. 1991. The reference map and an ensemble of 50
geologic realizations are generated for the horizontal permeability and porosity. The realizations
are generated based on variogram information in (Suman et al., 2011) using Sequential Gaussian
Simulation technique in the SGeMS geostatistics software. Figure 4.23 shows the logarithmic
horizontal permeability for the reference model (Figure 4.23(a)) as well as the initial saturation
map (Figure 4.23(b)). The mean of prior is also depicted in Figure 4.23(c). The reservoir model
description and parameters used for computing the NPV objective function is provided in Table
4.5. For this example, the fluid flow simulation is performed for 8 years and the reservoir life is
divided in three stages for drilling; 2 injectors and 3 producers are planned to be drilled at the first
stage while infill wells can be drilled at the end of year 2 (second stage) and at the end of year 4
(third stage) which results in a total of 75 drilling scenarios for the infill well drillings. These 75
infill scenarios are generated based on the assumption that a maximum of 3 injector infill wells
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

129

can be drilled collectively in the second and third stage and the maximum number of total infill
wells in the second and third stage is set to be equal to 5. Figure 4.24 shows the decision tree for
this example where the inner nodes represent possible drilling scenarios for the second stage where
number of wells (drilling possibility) is denoted as [
+𝑛 𝑝𝑟𝑜𝑑 −𝑛 𝑖𝑛𝑗 ] where +𝑛 𝑝𝑟𝑜𝑑 ,−𝑛 𝑖𝑛𝑗 represent the
number of producers and injectors for second stage, respectively. The leaves indicate the drilling
possibilities for the third stage where
+𝑛 𝑝𝑟𝑜𝑑 −𝑛 𝑖𝑛𝑗 indicates the number of producers and injectors for
the third stage. We will compare the results of proposed stochastic formulation and the traditional
approach where the possibility of future drilling is disregarded (No Drilling), both under geologic
uncertainty, to assess the performance of them in field development optimization and their
robustness to uncertainty in future drilling events.
(a) (b) (c)

Figure 4.23: (a) Reference Log-horizontal permeability map (millidarcy); (b) initial saturation map; (c) mean of
initial ensemble for Log-horizontal permeability maps.
Table 4.5: Reservoir parameters for the Norne model
Number of the grid cells 146×112×22
Number of active cells 44,915
Fluid Phases Oil-Water
Simulated reservoir life cycle 8 years
Injectors control mode Total water injection rate
Producers control mode Total fluid production rate
Oil price 50 $/𝑏𝑏𝑙
Water injection cost 6 $/𝑏𝑏𝑙
Water disposal/recycling cost 6 $/𝑏𝑏𝑙
Discount rate %10
Drilling cost $3×10
6
𝑀𝑀
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

130

Figure 4.24: decision tree for a multi-stage field development optimization considering 75 future development
possibilities
4.3.3.1 Disregarding Future Drilling (No Drilling)
In this approach, similar to the two-stage optimization example, the decision variables
corresponding to the current time-step are optimized (under geologic uncertainty) without
accounting for possible future infill drilling opportunities. Figure 4.25(a) shows the optimal
configuration for the three initial producers (Prod1, Prod2 & Prod3) and two injectors (Inj1 & Inj2)
shown on the mean of prior ensemble. After drilling the wells, geologic realizations are calibrated
using the hard data collected from the wells and the optimization is repeated to determine control
settings using the new realizations. The reservoir is operated with the optimal strategy and
performance data is collected to update the reservoir models. The updated models are, in turn, used
to optimize the control settings. Note that for model updating BHP values for all the wells and the
WOR in producers are used as dynamic data.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

131

(a) Disregarding Uncertainty (b) Stochastic

Figure 4.25: (a) Optimal well configurations for the initial wells (Prod1, Prod2, Prod3, Inj1 and Inj2) under No
Drilling assumption shown on initial mean of ensemble; (b) Optimal configuration for the stochastic case with 500
samples for geologic realizations, future drilling scenarios, future well locations and future control trajectories.

In the second stage and third stage, once the drilling scenario is known, an optimization
problem is solved to find the optimal decision variables. Depending on what scenario is selected,
the optimal decision variables for the remainder of the project are found and the sequence of
model-updating and optimization is repeated. Top rows of Figure 4.26 and Figure 4.27 (No
Drilling) show the mean of final updated ensemble at the end of the project as well as the final
saturation map and their corresponding normalized Net Present Value (w.r.t maximum NPV) for
six of the drilling scenarios for the case where the possibility of future infill drillings are not
included in the optimization formulation.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

132

(a) No Infill well [
0
0
],[
0
0
], scen 1 (a) [
0
0
],[
+1
0
], scen 5 (b) [
+1
−1
],[
+1
0
] , scen 38

No Drilling

Ens. Mean

Sat
NPV
0.6868 0.7064 0.7641

Stochastic

Ens. Mean

Sat
NPV
0.6659 0.7265 0.8716

Figure 4.26: (top) final optimal configuration and the mean of final updated (after 8 years) ensemble of geologic
realizations representing logarithm of permeability for three drilling scenarios (a) No Infill well [
0
0
],[
0
0
]; (b) [
0
0
],[
+1
0
] (1
producing infill well in third stage) and (c) [
+1
−1
],[
+1
0
] (one producing and one injecting infill wells in second stage and
an additional producer in third stage) and their corresponding saturation map (at year 8) for the case where the
possibility of future drilling opportunities are unaccounted for (No Drilling); (bottom) final optimal configuration and
the mean of final updated (after 8 years) ensemble of geologic realizations representing logarithm of permeability for
the same three drilling scenarios and their corresponding saturation map (at year 8) for the case where future drilling
possibilities are modelled as uncertain variable (stochastic approach)

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

133

(c) [
0
−2
],[
+2
−1
], scen 51 (b) [
+2
0
],[
+1
−1
], scen 59 (c) [
+2
0
],[
+2
−1
], scen 75

No Drilling

Ens. Mean

Sat
NPV
0.7936 0.8057 0.8365

Stochastic

Ens. Mean

Sat
NPV
0.9052 0.9329 1

Figure 4.27: (top) final optimal configuration and the mean of final updated (after 8 years) ensemble of geologic
realizations representing logarithm of permeability for three drilling scenarios (a) [
0
−2
],[
+2
−1
] (2 injectors in second stage
and 2 producers and 1 injector in third stage); (b) [
+2
0
],[
+1
−1
], (2 producers in second stage and 1 producer and 1 injector
in third stage) and (c) [
+2
0
],[
+2
−1
] (2 producers in second stage and 2 producers and 1 injector in third stage) and their
corresponding saturation map (at year 8) for the case where the possibility of future drilling opportunities are
unaccounted for (No Drilling); (bottom) final optimal configuration and the mean of final updated (after 8 years)
ensemble of geologic realizations representing logarithm of permeability for the same three drilling scenarios and their
corresponding saturation map (at year 8) for the case where future drilling possibilities are modelled as uncertain
variable (stochastic approach)

4.3.3.2 Stochastic Solution
In the stochastic approach, the optimization in the first stage can be expressed as:

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

134

𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 1
𝐸 𝑚 1,0
[𝑓 1
(𝑥 1
)+∑𝑝 𝑠 ∑𝑓 𝜏 (𝜃 𝜏 𝑠 )
3
𝜏 =2
𝑆 𝑠 =1
]
(4.18)

where 𝑥 1
represent the decision variables (well locations and controls) of the first stage,
𝑚 𝑡 ,𝑘 denotes the input parameters at development stage 𝑡 after 𝑘 steps of model calibration; 𝜃 𝜏 𝑠 =
(𝜔 𝜏 𝑠 ,𝑢̃
𝜏 𝑠 ,𝑞̃
𝜏 𝑠 ) is the vector of uncertain variables representing the uncertainty in the number of infill
wells (𝜔 𝜏 𝑠 ) and their corresponding location (𝑢̃
𝜏 𝑠 ) and control trajectories (𝑞̃
𝜏 𝑠 )in the 𝜏 stage of field
development problem. 𝑆 denotes the total number of scenarios for number, location and control of
the future wells. To solve for this problem, at each iteration of optimization 500 samples are taken
to represent the uncertainty in geologic description, future drilling scenario and the infill wells
potential locations and controls. To reduce the computational demand of solving Eq. 4.16, becuase
the objective function includes a linear operation over all the scenarios (expected value), the cost
and the gradients (if required) are calculated independently by parallel execution over multi-core
cluster nodes. Therefore, we were able to expedite optimization process by using multiple
processing core at a time.
Figure 4.28 shows the evolution of the objective function in Eq. (4.18). Figure 4.25(b)
shows the optimal configuration for the initial 3 producers and 2 injectors (shown on mean of prior
ensemble) obtained from solving the optimization problem in (4.18). The same procedure as in the
last example is repeated, where a model calibration step is performed once hard data or
performance data become available, followed by an optimization problem based on the updated
models (as before).

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

135

Figure 4.28: Evolution of the objective function in multi-stage optimization case where the possibility of future
drillings is accounted for upfront.

In the second stage, when the uncertainty regarding the number of infill wells for the second
stage is resolved, the optimization problem for the second stage can be formulated as:

𝑚𝑖𝑛𝑖𝑚𝑖𝑧 𝑒 𝑥 2
𝐸 𝑚 2,0
[𝑓 2
(𝑥 2
)+∑𝑝 𝑠 𝑓 3
(𝜃 3
𝑠 )
𝑆 𝑠 =1
]
(4.19)

where 𝑥 2
represent the decision variables (well locations and controls) of the second stage,
𝑚 2,0
refers to the latest updated model parameters at second stage of development (with 0 model
calibration steps in the second stage) and 𝜃 3
𝑠 =(𝜔 3
𝑠 ,𝑢̃
3
𝑠 ,𝑞̃
3
𝑠 ) is the vector of uncertain variables
representing the uncertainty in the number of infill wells (𝜔 3
𝑠 ) and their corresponding location
(𝑢̃
3
𝑠 ) and control trajectories (𝑞̃
3
𝑠 )in the third stage of field development problem. After solving
problem (4.18) and drilling the wells, the same procedure for closed-loop field development
optimization will be implemented in the second stage, where after each step (𝑘 ) of data collection
and model calibration, the control settings of the existing wells for the remainder of the reservoir’s
life is optimized.
After 4 years, when the uncertainty regarding the number of infill wells of the third stage
is revealed, the optimal location(s) of the new wells as well as the control settings for all existing
wells are optimized for the remainder of the project. Once the optimal well locations are identified,
the model realizations are updated, and the sequence of control optimization and model updating
Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

136

steps is repeated. Bottom rows of Figure 4.26 and Figure 4.27 (stochastic) show the mean of final
updated permeability ensemble at the end of the project and the optimal configuration as well as
the final saturation map and their corresponding Net Present Value (NPV) for six of the drilling
scenarios obtained from the stochastic approach.
In Figure 4.29, the normalized NPV values for all the 75 scenarios for the No Drilling
(disregarding the possibility of future infill drilling) and stochastic approaches are shown in a bar
plot. The scenarios are assorted based on their final NPV value from Stochastic Approach. Similar
to the previous example, except for the first scenario, where no infill well is drilled, the NPV values
for the ND (No-Drilling) case are consistently lower that those resulting from the stochastic
solution and the maximum increase is about %26. The solid and dash lines in Figure 4.29 show
the mean of the NPV for all 75 scenarios in the Stochastic and No-Drilling (i.e., disregarding future
drilling opportunities is neglected) approaches, respectively. As can be seen the mean value of
stochastic (here-and-now) approach is more than %13 higher than the mean value of field
development approach without incorporating the future drilling uncertainty (Value of Stochastic
Solution).

Figure 4.29: NPV performance for No Drilling and Stochastic Solution under each drilling scenario in multi-stage
optimization.

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

137

To evaluate the robustness of stochastic solution w.r.t location and control uncertainty. For
the last drilling scenario (#75), 50 new realizations for control and 50 new realizations for the
location of the wells for the last stage of field development are generated. For each 50 control
realizations, for both case studies (Stochastic and No Drilling) well placement optimization is
performed to find the optimal well locations for the last stage. Similarly, for each 50 well locations,
well control optimization is implemented to obtain optimal control trajectory for each
configuration for both No drilling and stochastic case.
Figure 4.30(a) shows the CDF of total NPV for No Drilling and Stochastic cases by
optimizing the controls for the last stage over 50 new realizations for possible locations of infill
wells. The expected normalized NPV for No Drilling and Stochastic cases are 0.7783 and 0.9491,
respectively. Similarly, for the 50 new control realizations, well locations of the last stage infill
wells were optimized, and Figure 4.30 (b) depicts the CDF plot for this case where the expected
normalized NPV value for No Drilling and Stochastic cases are 0.8004 and 0.9700, respectively.
The results from these experiments demonstrate the importance of including the uncertainty in
infill well control settings to ensure solution robustness under uncertain future development plans.

(a) (b)
Figure 4.30: (a) Empirical CDF for NPV values of No Drilling and stochastic based on 50 well location realizations
(b) Empirical CDF for NPV values of No Drilling and stochastic based on 50 control realizations

Chapter 4: Closed-Loop Optimization Under Geologic and future Development Uncertainty

138

4.4 Conclusion
In this chapter, we develop a closed-loop stochastic field development optimization
approach under uncertainty in future development plans (number of future infill wells and their
locations and control settings) and geologic reservoir description. The motivation behind the
developed framework is that oilfields undergo, often unpredictable, development activities
throughout their life-cycle that are not accounted for in traditional optimization workflows.
Disregarding future developments when optimizing current well configurations and well control
settings is likely to result in suboptimal solutions over the life of the field. While future
development plans can be considered as decision variables for optimization, in practice future
development events are very likely to deviate from those obtained from past optimization
solutions.
We develop a closed-loop stochastic formulation, where in addition to uncertain reservoir
model parameters, future developments are incorporated as uncertain events that are modeled
using random variables. In this approach, the decision variables pertaining to the current stage are
optimized while the future development plans are incorporated as a stochastic process. To evaluate
the performance of the proposed approach and demonstrate the advantages of incorporating future
development events as uncertain variables, we presented results from numerical experiments for
closed-loop field development optimization. We compared the solutions from the stochastic and
no-drilling assumption approaches and observed that the former offers an opportunity to hedge
against the uncertainty in future developments, while the latter suffers from lack of robustness to
changes in future development activities. The results indicate that the robustness of stochastic
approach against uncertainty in future drilling plans is an important consideration in realistic field
development problems where future decisions are highly uncertain.
Chapter 5: Summary, Conclusions and Future Work

139

Chapter 5
Summary, Conclusions and Future
Work
In this chapter, we first briefly summarize the main research topics covered in this
dissertation. We then present the main conclusions of our findings in this work. At the end, we
provide some future research directions based on our conclusions.
5.1 Summary
In this dissertation, we focused on life-cycle oilfield development and production
optimization while considering the possibility of future drilling events. The main idea here is, in
real-life field development projects due to high cost of drilling as well as limited information at
the early stages, a few number of wells are drilled and operated for some time to acquire more
information about the reservoir and based on the collected information, decisions regarding drilling
new wells (infill wells) are made. Therefore, optimization techniques should always consider the
possibility of changes in field configuration, mainly due to the introduction of new wells.
In reservoir development where time and uncertainty play an important role, the decision
model can be designed to allow the decision maker to adopt a decision that can respond to
observations as they unfold. Therefore, the decision maker does not have to make all the decision
at once. Instead, decisions can be delayed until the existing uncertainties are resolved by additional
data and information. However, in anticipation of changes in reservoir configurations due to
development plans, the decision maker must consider possible future events with their associated
uncertainties.
Chapter 5: Summary, Conclusions and Future Work

140

In chapter 2, we introduced the idea of stochastic field development and production
optimization under uncertainty in future development plans where in the “stochastic” (here-and-
now) approach, the uncertainty in future drilling events is modeled with random variables. As the
uncertain variable, i.e., number of wells to drill, is realized (based on new information from the
field and other external factors), future decisions can be adapted according to the realized uncertain
parameters and the information available about the current state of the field. This formulation leads
to a sequential decision-making problem where the decision variables pertaining to the current
stage are optimized while the future development plans are incorporated as a stochastic process.
In Chapter 3, we extended the uncertain parameter to include the uncertainty in future well
locations and operational settings, where we used quality maps and set of constraints such as
minimum interwell spacing for allocating potential locations to future infill wells.
In Chapter 4, we generalized the formulation to consider geologic uncertainty in a closed-
loop implementation with dynamic model updating which involved repeating the optimization
procedure after each model updating step. In such a closed-loop implementation, updatable
decision variables (those that are not irreversible) can be adjusted after each model updating stage.
5.2 Conclusions
The key findings from this study are as follows:
• Without incorporating future development uncertainty in oilfield optimization problems, it
is highly unlikely to find optimal solutions, as any new development event can markedly
diminish the quality of the optimal solution. Initial suboptimal solution can constrain the field
operations for years.
Chapter 5: Summary, Conclusions and Future Work

141

• The proposed stochastic formulation in this work by incorporating the uncertainty in future
development plans leads to solutions that are robust and offer hedging against new
development activities.
• By extending the uncertain parameter to include the uncertainty in future well location and
control strategies (in addition to number of future infill drillings), we could demonstrate the
importance of including the uncertainty in infill well control settings and location to ensure
solution robustness under uncertain future development plans.
• By generalizing the formulations to account for the uncertainty in geologic description and
comparing the solutions from the stochastic and no-drilling assumption approaches and
observed that the former offers an opportunity to hedge against the uncertainty in future
developments, while the latter suffers from lack of robustness to changes in future development
activities. The results indicate that the robustness of stochastic approach against uncertainty in
future drilling plans is an important consideration in realistic field development problems
where future decisions are highly uncertain.
In closed-loop implementation, the only computational burden is that the stochastic
optimization must be repeated at each data assimilation step to modify the decision variables
for the remainder of the project. However, because the objective function includes a linear
operation over all the scenarios (expected value), the cost and the gradients (if required) are
calculated independently by parallel execution over multi-core cluster nodes. Therefore, we
were able to expedite optimization process by using multiple processing core at a time.
5.3 Discussion
• In the stochastic formulation, we assumed the stochastic process representing the
uncertainty in number of future infill wells is stage-wise independent (meaning that the random
Chapter 5: Summary, Conclusions and Future Work

142

variable 𝜔 𝑡 +1
is independent of 𝜔 [1:𝑡 ]
=(𝜔 1
,…,𝜔 𝑡 )).We note that the number of infill wells
at each stage may depend on the number of infill wells in other stages. In that case, the decision
tree will show dependence between the number of wells at different stages of each scenario.
However, the exact form of the dependence among the drilling decision will be problem-
specific. In the presented examples we accounted for this dependency by constraining the
maximum number of producers and injectors that can be drilled throughout the life-cycle of
the reservoir under each drilling scenario.
• Throughout this manuscript, Net Present Value was the objective function used for all the
examples. However, the presented formulation can be used for any other measure of
performance (e.g. maximizing total oil production or minimizing the total water production).
• For most of the presented examples, equi-probable drilling scenarios were used to represent
the uncertainty in future number of the wells. We agree that the final solution is dependent on
the choice of the probabilities assigned to each scenario. For the multi-stage stochastic field
development optimization of section 2.6.2.4 we assumed different probability of occurrence
for each drilling scenario and we updated these probabilities through the reservoir’s production
life.
• For all the examples in this dissertation, we implemented SPSA algorithm for well
placement and Quasi-Newton method for control optimization. Because both of these methods
are gradient-based methods (SPSA provides stochastic gradient at each iteration), the solution
obtained from them is a local solution. That is why for all the examples, we repeated the
sequential well placement and control optimization for multiple different initializations and
reported the ones that achieved the highest performance measure.
Chapter 5: Summary, Conclusions and Future Work

143

5.4 Future Research Directions
There are many directions that could be pursued in areas relevant to the elements and
objective of this dissertation. Our suggestions for future research are as follows:
• In this study, we only considered optimizing the location and operational strategies of the
wells at each stage while accounting for future drilling activities. The decision vector can be
augmented to find optimal number of infill wells at each stage.
• Pruning strategies can be used to reduce the size of decision tree. Because the decision tree
grows exponentially with the number of drilling scenarios, it is important to remove drilling
scenarios that have smaller importance and impact on our solution to keep the computational
burden at an acceptable level. However, at the same time the decision tree should capture the
uncertainty in future drilling events.
• Depending on the size of the reservoir and its complexities (e.g. existence of faults and
barriers), the computational burden of the stochastic framework can be demanding. In that
case, we suggest using computationally efficient proxy models to replace some of the required
reservoir simulations. The proxy models have been used as computationally efficient ways to
approximate reservoir and well responses in literature. The idea behind using surrogate model
is that parts of the underlying physics can be modified in terms of simpler equations, or through
the use of data-driven methods, thereby sacrificing some accuracy for a significant decrease in
the computational loads. Although proxy models have been developed to predict well and
reservoir response for a fixed well configuration, forecasting the response for different well
configurations and number of the wells have not been investigated widely in the literature.
• Because the proposed formulation is quite general, we highly encourage using other
optimization methods suitable for MINL programs.
Chapter 5: Summary, Conclusions and Future Work

144

• In this work, we used the expected value of the random function as optimization objective
function. We suggest using other probabilistic representation (e.g. coefficient of variation,
variance and fractiles) of the objective function and constraints for stochastic (here-and-now)
approach. We also suggest using chance constrained programming where the objective
function is expressed in terms of expected value, while the constraints are expressed in terms
of fractiles (probability of constraint violation) and are interpreted probabilistically, and
inequality constraints can be violated with a pre-defined probability (𝛼 ).
• In this dissertation we only studied the application of the proposed stochastic framework
in waterflooding operations and used a black-oil simulator for predictions. We suggest
applying the proposed framework for other recovery process (SAGD, gas injection).
• We suggest applying the proposed framework on a real field applications so we can identify
the areas that we can improve.
• Another interesting extension to investigate is to include the value of information (VoI) in
Closed-loop Reservoir Management (CLOREM) where the value of future measurements is
assessed in the field development planning phase and it enables us to take into account how
the additional information can be used to make better decisions regarding developing the
reservoir. In that case, in addition to going after maximizing the performance of reservoir by
making optimal infill drilling decisions, objective function should quantify how the
information collected from future infill drilling can help us improve our knowledge about
reservoir (through data-assimilation) and how it impacts the drilling decisions.
Nomenclature

145

Nomenclature
𝑁𝑃𝑉 Net present value ($)
𝑓 (𝑥 ) Objective function
𝐶 Cost of fracturing ($/𝑓𝑡 )
𝑟 𝑔 Gas price ($/MSCF)
𝑟 𝑤 Water disposal cost ($/𝑏𝑏𝑙 )
𝑄 𝑔 Gas production rate (𝑀𝑆𝐶𝐹 /𝐷 )
𝑄 𝑤 Water production rate (𝑏𝑏𝑙 /𝑑𝑎𝑦 )
𝑛 Number of fracture stages
𝑢 Decision variable denoting the location of the well
𝑞 (𝑢 ,𝑡 ) Decision variable denoting the well control setting as a function of location and time
𝑏 Discount factor
𝐶 Cost of drilling a well
𝑟 𝑜 Price of oil per barrel
𝑐 𝑤𝑝
Cost of disposing/recycling of produced water per barrel
𝑐 𝑤𝑖
Cost of injecting water per barrel
𝜏 Time (days)
𝑁 𝑝 Total number of producers
𝑁 𝐼 Total number of injectors
𝑄 𝑜 ,𝑗 𝑘
Oil produced from well j at time step k (bbl)
𝑄 𝑤 ,𝑖
𝑘
Water injected/produced from well i at time step k (bbl)
∆𝑡 𝑘 𝑘 𝑡 ℎ
(time) step-size
𝑚 Model parameter input parameters (e.g. petrophysical and fluid properties)
𝑢 𝑡 Well location decision variable at stage 𝑡
𝜔 𝑡 Random variable observed just before taking decision 𝑢 𝑡 (number of the wells)
𝜔 [1:𝑡 ]
The history of random variable observation from stage 1 to 𝑡
𝑓 𝑡 (∙) Cost function at stage 𝑡
𝑥 [1:𝑡 ]
The collection of decision policies from stage 1 to 𝑡
𝑇 Total number of decision-making stages
𝒰 t
Feasibility set for 𝑢 𝑡 (well locations)
𝑝 𝑘 Probability assigned to scenario 𝑘
𝑛 Number of scenarios
𝐶 𝑤 𝑡 Discounted cost of drilling well 𝑤 in horizon 𝑡
𝑥 𝑤 𝑡 Binary variable that determines if well 𝑤 is drilled at stage 𝑡
𝑊 Maximum number of infill wells at each stage
Nomenclature

146

𝜃 Random variable denoting the uncertainty in location and number of future infill
drillings
𝐾 Permeability
𝜑 Porosity
𝑆 𝑜 Oil saturation
ℎ Reservoir thickness

Bibliography

147

Bibliography
[1] Aanonsen, S. I., Nævdal, G., Oliver, D. S., Reynolds, A. C., & Vallès, B. (2009). The
ensemble Kalman filter in reservoir engineering--a review. SPE Journal, 14(03), 393-412.
[2] Badru, O., & Kabir, C. S. (2003, January). Well placement optimization in field development.
In SPE Annual Technical Conference and Exhibition. Society of Petroleum Engineers.
[3] Bangerth W., Klie H., Wheeler, M.F., Stoffa, P. L., & Sen, M. K. (2006): "On Optimization
Algorithms for the Reservoir Oil Well Placement Problem", Computational Geosciences,
10(3), 303-319.
[4] Beckner, B. L., & Song, X. (1995, January). Field development planning using simulated
annealing-optimal economic well scheduling and placement. In SPE annual technical
conference and exhibition. Society of Petroleum Engineers.
[5] Bellman, R. (1961). Adaptive control processes: A guided tour. Princeton, NJ: Princeton
University Press.
[6] Brouwer, D. R., & Jansen, J. D. (2002, January). Dynamic optimization of water flooding
with smart wells using optimal control theory. In European Petroleum Conference. Society
of Petroleum Engineers.
[7] Brouwer, D. R., Nævdal, G., Jansen, J. D., Vefring, E. H., & Van Kruijsdijk, C. P. J. W.
(2004, January). Improved reservoir management through optimal control and continuous
model updating. In SPE Annual Technical Conference and Exhibition. Society of Petroleum
Engineers.
[8] Centilmen, A., Ertekin, T., & Grader, A. S. (1999, January). Applications of neural networks
in multiwell field development. In SPE Annual Technical Conference and Exhibition.
Society of Petroleum Engineers.
[9] Chen, C., Wang, Y., Li, G., & Reynolds, A. C. (2010). Closed-loop reservoir management
on the Brugge test case. Computational Geosciences, 14(4), 691-703.
[10] Chen, Y., Oliver, D. S., & Zhang, D. (2009). Efficient ensemble-based closed-loop production
optimization. SPE Journal, 14(04), 634-645.
[11] Cullick, S., Heath, D., Narayanan, K., April, J., & Kelly, J. (2004). Optimizing multiple-field
scheduling and production strategy with reduced risk. Journal of Petroleum
Technology, 56(11), 77-83.
Bibliography

148

[12] da Cruz, P. S., Horne, R. N., & Deutsch, C. V. [1999] The quality map: a tool for reservoir
uncertainty quantification and decision making. In SPE Annual Technical Conference and
Exhibition. Society of Petroleum Engineers.
[13] Emerick, A. A., Silva, E., Messer, B., Almeida, L. F., Szwarcman, D., Pacheco, M. A. C., &
Vellasco, M. M. B. R. (2009, January). Well placement optimization using a genetic algorithm
with nonlinear constraints. In SPE reservoir simulation symposium. Society of Petroleum
Engineers.
[14] Evensen, G. (1994). Sequential data assimilation with a nonlinear quasi‐geostrophic model
using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research:
Oceans, 99(C5), 10143-10162.
[15] Evensen, G. (2009). Data assimilation: the ensemble Kalman filter. Springer Science &
Business Media.
[16] Forouzanfar, F., & Reynolds, A. C. (2014). Joint optimization of number of wells, well
locations and controls using a gradient-based algorithm. Chemical Engineering Research and
Design, 92(7), 1315-1328.
[17] Geir, N., Johnsen, L. M., Aanonsen, S. I., & Vefring, E. H. (2003, January). Reservoir
monitoring and continuous model updating using ensemble Kalman filter. In SPE Annual
Technical Conference and Exhibition. Society of Petroleum Engineers.
[18] Gupta, V., & Grossmann, I. E. (2011). Solution strategies for multistage stochastic
programming with endogenous uncertainties. Computers & Chemical Engineering, 35(11),
2235-2247.
[19] Guyaguler, B., & Horne, R. N. (2001, January). Uncertainty assessment of well placement
optimization. In SPE annual technical conference and exhibition. Society of Petroleum
Engineers.
[20] Isebor, O. J., Echeverría Ciaurri, D., & Durlofsky, L. J. (2014). Generalized field-
development optimization with derivative-free procedures. SPE Journal, 19(05), 891-908.
[21] Jahandideh, A., & Jafarpour, B. (2016). Optimization of hydraulic fracturing design under
spatially variable shale fracability. Journal of Petroleum Science and Engineering, 138, 174-
188.
[22] Jaikumar, R., & Bohn, R. E. (1992). A dynamic approach to operations management: An
alternative to static optimization. International Journal of Production Economics, 27(3), 265-
282.
Bibliography

149

[23] Jansen, J. D., Brouwer, D. R., Naevdal, G., & Van Kruijsdijk, C. P. J. W. (2005). Closed-loop
reservoir management. First Break, 23(1), 43-48.
[24] Jansen, J.D. (2013). A systems description of flow through porous media. Springer.
[25] Humphries, T. D., Haynes, R. D., & James, L. A. (2013). Simultaneous and sequential
approaches to joint optimization of well placement and control. Computational
Geosciences, 1-16.
[26] Humphries, T. D., Haynes, R. D., & James, L. A. (2014). Simultaneous and sequential
approaches to joint optimization of well placement and control. Computational
Geosciences, 18(3-4), 433-448.
[27] Li L., Jafarpour B. (2012): "A Variable-control Well Placement Optimization for Improved
Reservoir Development ". Computational Geosciences, 16 (4), pp:871-889.
[28] Li L., Jafarpour B., Mohammad-Khaninezhad M.R. (2013): "A Simultaneous Perturbation
Stochastic Approximation Algorithm for Coupled Well Placement and Control Optimization
under Geologic Uncertainty". Computational Geosciences 17.1 (2013): 167-188.
[29] Montes, G., Bartolome, P., & Udias, A. L. (2001, January). The use of genetic algorithms in
well placement optimization. In SPE Latin American and Caribbean Petroleum Engineering
Conference. Society of Petroleum Engineers.
[30] MRST: The MATLAB Reservoir Simulation Toolbox. www.sintef.no/MRST (2016b &
2018a)
[31] Navabi, S., Khaninezhad, R., & Jafarpour, B. (2016). A unified formulation for generalized
oilfield development optimization. Computational Geosciences, 1-28.
[32] Peaceman, D.W. Fundamentals of Numerical Reservoir Simulation, 1977.
[33] Peters, L., Arts, R., Brouwer, G., Geel, C., Cullick, S., Lorentzen, R. J., ... & Sarma, P. (2010).
Results of the Brugge benchmark study for flooding optimization and history matching. SPE
Reservoir Evaluation & Engineering, 13(03), 391-405.
[34] Sarma, P., Durlofsky, L. J., Aziz, K., & Chen, W. H. (2006). Efficient real-time reservoir
management using adjoint-based optimal control and model updating. Computational
Geosciences, 10(1), 3-36.
[35] Scott, A. J. (1971). Dynamic location-allocation systems: some basic planning
strategies. Environment and Planning A, 3(1), 73-82.
Bibliography

150

[36] Shapiro, A. (2011). Topics in stochastic programming. CORE Lecture Series, Universite
Catholique de Louvain.
[37] Shirangi, M. G., & Durlofsky, L. J. (2015). Closed-loop field development under uncertainty
by use of optimization with sample validation. SPE Journal, 20(05), 908-922.
[38] Shmoys D.B., Tardos E., Aardal K. (1997). Approximation algorithms for facility location
problems (extended abstract). In Proceedings of the twenty-ninth annual ACM symposium on
Theory of computing (STOC '97). ACM, New York, NY, USA, 265-274.
[39] Shu, T., Krunz, M., & Vrudhula, S. (2006). Joint optimization of transmit power-time and bit
energy efficiency in CDMA wireless sensor networks. IEEE Transactions on Wireless
Communications, 5(11).
[40] Siraj, M. M., Van den Hof, P. M., & Jansen, J. D. (2015, December). Risk management in oil
reservoir water-flooding under economic uncertainty. In Decision and Control (CDC), 2015
IEEE 54th Annual Conference on (pp. 7542-7547). IEEE.
[41] Siraj, M. M., Van den Hof, P. M., & Jansen, J. D. (2016). Robust optimization of water-
flooding in oil reservoirs using risk management tools. IFAC-PapersOnLine, 49(7), 133-138.
[42] Siraj, M.M., Van den Hof, P.M.J. and Jansen, J.D. (2017). Handling geological and economic
uncertainties in balancing short-term and long-term objectives in water-flooding optimization.
SPE-173285-MS (Accepted for publication in SPE Journal).
[43] Snyder, L. V. (2006). Facility location under uncertainty: a review. IIE transactions, 38(7),
547-564.
[44] Spall, J. C. (1998): "An Overview of the Simultaneous Perturbation Method for Efficient
Optimization". Johns Hopkins APL Technical Digest, 19(4), 482-492.
[45] Spall, J.C. (2000): "Adaptive Stochastic Approximation by the Simultaneous Perturbation
Method". IEEE Trans. Autom. Contr. 45, 1839–853.
[46] Spall J.C. (2003): "Introduction to Stochastic Search and Optimization: Estimation,
Simulation and Control". Wiley, New Jersey.
[47] Suman, A., Fernández-Martínez, J. L., & Mukerji, T. [2011] Joint inversion of time-lapse
seismic and production data for Norne field. In 2011 SEG Annual Meeting. Society of
Exploration Geophysicists.
[48] van Essen, G., Zandvliet, M., Van den Hof, P., Bosgra, O., & Jansen, J. D. (2009). Robust
waterflooding optimization of multiple geological scenarios. SPE Journal, 14(01), 202-210.
Bibliography

151

[49] Vygen, J. (2005). Approximation algorithms for facility location problems. Lecture Notes
Research Institute for Discrete Mathematics, University of Bonn, Bonn, Germany.
[50] Wang, C., Li, G., & Reynolds, A. C. (2009). Production optimization in closed-loop reservoir
management. SPE journal, 14(03), 506-523.

Appendix A: SPSA Algorithm

152

Appendix A: SPSA Algorithm
SPSA algorithm (Simultaneous Perturbation Stochastic Approximation) was first
introduced by Spall (1992) and later extended to several variants (Spall, 2000,2003) has also been
applied to well placement optimization in conventional reservoirs (Bangerth et al., 2006; Li and
Jafarpour 2012; Li et al., 2013). A main advantage of the SPSA algorithm is its stochastic gradient
approximation, which is quite efficient for large dimensional problems where analytical gradient
calculation is not possible. For the objective function 𝑓 (𝑥 ) (-NPV in our case), where 𝑥 is a m-
dimensional vector of unknown decision variables over the feasible set of well location or well
schedules, a minimizer 𝑥̂
is found by satisfying the first-order optimality condition 𝑔 (𝑥̂)=
𝜕𝑓 (𝑥 )
𝜕𝑥
|
𝑥 =𝑥̂
=0, using an appropriate line-search method. The following standard recursion
algorithm can be implemented to search for a solution:
𝑥 𝑘 +1
=𝑥 𝑘 −𝑎 𝑘 𝑔̂
𝑘 (𝑥 𝑘 ) (A-1)

where 𝑔̂
𝑘 (𝑥 𝑘 ) is a stochastic approximation of 𝛻𝑓 (𝑥 𝑘 ) at 𝑘 th iteration, and 𝑎 𝑘 is the update
step size. Gradient approximation can be computed using a finite difference stochastic
approximation (FDSA) or the SPSA algorithm. For the finite difference approximation, the 𝑖 th
component of the gradient vector 𝑔 𝑘 (𝑥 𝑘 ) at the 𝑘 th iteration can be written as:
𝑔̂
𝑘𝑖
(𝑥 𝑘 )=
𝑓 (𝑥 𝑘 +𝑐 𝑘 𝑒 𝑖 )−𝑓 (𝑥 𝑘 −𝑐 𝑘 𝑒 𝑖)
2𝑐 𝑘 (𝑖 =1,2,…,𝑚 ) (A-2)

in which 𝑒 𝑖 denotes the standard unit vector (with one as its 𝑖 th component and zero
elsewhere), and 𝑐 𝑘 represents a small positive number. Considering the dimensionality of 𝑥 , the
Appendix A: SPSA Algorithm

153

FDSA approximation of the gradient is implemented one dimension at a time, requiring a total of
2𝑚 function evaluations at each iteration, which is computationally inefficient for large problems.
When the gradient is approximated using simultaneous perturbations, all the 𝑚 components of the
gradient vector are randomly perturbed at once to obtain two measurements of 𝑓 (𝑥 ) to compute:
𝑔̂
𝑘 (𝑥 𝑘 )=
𝑓 (𝑥 𝑘 +𝑐 𝑘 ∆
𝑘 )−𝑓 (𝑥 𝑘 −𝑐 𝑘 ∆
𝑘 )
2𝑐 𝑘 ∆
𝑘 (A-3)

where the user-generated 𝑚 -dimensional random perturbation vector ∆
𝑘 =
[∆
𝑘 1
,∆
𝑘 2
,…,∆
𝑘𝑚
]
𝑇 contains independent and symmetrically distributed (about 0) members with
finite inverse moment 𝐸 (|∆
𝑘𝑖
|
−1
). One particular distribution for ∆
k
that satisfies the
aforementioned conditions is the symmetric Bernoulli distribution with random values ±1.
The advantage of the SPSA over the FDSA is its efficiency in estimating the approximate
gradient, especially in large dimensional problems or when the objective function is noisy. To
approximate the gradient, the SPSA requires two function calls in each iteration, regardless of the
size of 𝑚 . On the other hand, the FDSA gradient approximation is achieved through 2𝑚 cost
function evaluations. Thus, the SPSA uses 𝑚 times fewer function evaluations than the FDSA.
However, because the SPSA does not follow the gradient path exactly, its convergence to the
solution may require more iterations than is needed with the FDSA algorithm. Nonetheless, it has
been shown that because the approximated gradient with the SPSA is an unbiased estimator of the
gradient, on average it nearly tracks the gradient. Hence, if implemented properly, the SPSA
algorithm offers a 𝑚 -fold savings per iteration over the FDSA algorithm.
An important point to consider in well placement and well scheduling is that the domain
of optimization is discrete, implying that the feasible sets 𝛩 𝑢 and 𝛩 𝑡 are finite. Although the SPSA
algorithm originally was developed for continuous optimization problems, it is generally
applicable to both integer and mixed-integer optimization problems. Examples of its application
Appendix A: SPSA Algorithm

154

to well placement and general field development optimization can be found in (Bangerth et al.,
2006, Li and Jafarpour 2012, Li et al., 2013).
Appendix B: Multi-Stage Stochastic Optimization

155

Appendix B: Multi-Stage Stochastic
Optimization
In its generic form, a T-stage stochastic problem can be written as (Shapiro, 2011):

𝑚𝑖𝑛 𝑥 1
,𝑥 2
,…,𝑥 𝑇 𝐸 𝜔 [ 𝑓 1
(𝑥 1
)+𝑓 2
(𝑥 2
(𝜔 2
),𝜔 2
)+⋯+𝑓 𝑇 (𝑥 𝑇 (𝜔 [2:𝑇 ]
),𝜔 𝑇 )]
(B-1)
where 𝜔 2
,𝜔 3
,…,𝜔 𝑇 represent the uncertain variables (i.e., random data process), 𝑥 𝑡 ∈
ℝ
𝑛 𝑡 ,𝑡 =1,…,𝑇 are the decision variables and 𝑓 𝑡 (𝑥 𝑡 (𝜔 [2:𝑡 ]
),𝜔 𝑡 ) is the cost function for stage 𝑡
that depends on the decision variables and random outcomes for that stage and the history of the
observations in the past stages. Note that 𝑓 1
(𝑥 1
):ℝ
𝑛 1
→ℝ is deterministic. The expectation in
(B-1) is calculated based on the following definition:

𝐸 𝑥 1
,𝑥 2
,…,𝑥 𝑇 [𝐺 (𝒙 ,𝝎 )]=∫ ∫ …∫ 𝐺 (𝒙 ,𝝎 )
∞
−∞
∞
−∞
∞
−∞
ℎ(𝜔 2
,…,𝜔 𝑇 ) 𝑑 𝜔 𝑇 …𝑑 𝜔 2

(B-2)
where ℎ(𝜔 1
,𝜔 2
,…,𝜔 𝑇 ) is the joint probability distribution of 𝝎 , and the function under
the expectation is:

𝐺 (𝒙 ,𝝎 )=𝑓 1
(𝑥 1
)+𝑓 2
(𝑥 2
(𝜔 2
),𝜔 2
)+⋯+𝑓 𝑇 (𝑥 𝑇 (𝜔 [2:𝑇 ]
),𝜔 𝑇 )
(B-3)
In the above equations, 𝒙 =[𝑥 1
,𝑥 2
,…,𝑥 𝑇 ] and 𝝎 =[𝜔 2
,𝜔 3
,…,𝜔 𝑇 ]. Using the
multiplication rule of probability, the multi-variate joint probability distribution of ℎ can be
decomposed into a set of univariate conditional densities:
ℎ(𝜔 2
,…,𝜔 𝑇 )=ℎ(𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
)ℎ(𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
)…ℎ(𝜔 3
|𝜔 2
)ℎ(𝜔 2
) (B-4)
which can be plugged into Eq. (B-2) to yield:
Appendix B: Multi-Stage Stochastic Optimization

156

𝑚𝑖𝑛 𝒙 ∫ ⋯∫ 𝐺 (𝒙 ,𝝎 )
∞
−∞
∞
−∞
ℎ(𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
)ℎ(𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
) …
…ℎ(𝜔 3
|𝜔 2
)ℎ(𝜔 2
)𝑑 𝜔 𝑇 …𝑑 𝜔 𝑇

(B-5)
𝑚𝑖𝑛 𝒙 ∫ ⋯{∫ 𝐺 (𝒙 ,𝝎 )ℎ(𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
)𝑑 𝜔 𝑇 ∞
−∞
}
∞
−∞
ℎ(𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
)…
…ℎ(𝜔 2
)𝑑 𝜔 𝑇 −1
…𝑑 𝜔 2

(B-6)
The term inside the curly brackets is 𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[𝐺 (𝒙 ,𝝎 )] , which is a function of
𝜔 [2:𝑇 −1]
. After plugging the solution for the inner integral, the result can be expressed as:
𝑚𝑖𝑛 𝒙 ∫ ⋯{∫ 𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[𝐺 (𝒙 ,𝝎 )]
∞
−∞
ℎ(𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
)𝑑 𝜔 𝑇 −1
}
∞
−∞
…
… ℎ(𝜔 𝑇 −2
|𝜔 2
,…,𝜔 𝑇 −3
)… ℎ(𝜔 2
) 𝑑 𝜔 𝑇 −2
…𝑑 𝜔 2

(B-7)

where now the term inside the brackets is 𝐸 𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
[𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[𝐺 (𝒙 ,𝝎 )]] and the
resulting problem is expressed as
𝑚𝑖𝑛 𝒙 ∫ ⋯
∞
−∞
∫ 𝐸 𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
[𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[𝐺 (𝒙 ,𝝎 )]]ℎ(𝜔 𝑇 −2
|𝜔 2
,…,𝜔 𝑇 −3
)…
∞
−∞

…ℎ(𝜔 2
)𝑑 𝜔 𝑇 −2
…𝑑 𝜔 2

(B-8)

continuing this process for 𝑇 −3, 𝑇 −4,…,2 , we can write Eq. (B-1) in the following
iterated expectation form:
𝑚𝑖𝑛 𝒙 𝐸 𝜔 2
[… 𝐸 𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
[𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[𝑓 1
(𝑥 1
)+𝑓 2
(𝑥 2
(𝜔 2
),𝜔 2
)+⋯
+𝑓 𝑇 (𝑥 𝑇 (𝜔 [1:𝑇 ]
),𝜔 𝑇 )]]]

(B-9)

which can be simplified further to:
Appendix B: Multi-Stage Stochastic Optimization

157

𝑚𝑖𝑛 𝒙 𝑓 1
(𝑥 1
)+𝐸 𝜔 2
[𝑓 2
(𝑥 2
(𝜔 2
),𝜔 2
)… 𝐸 𝜔 𝑇 −1
|𝜔 2
,…,𝜔 𝑇 −2
[𝑓 𝑇 −1
(𝑥 𝑇 −1
(𝜔 [2:𝑇 −1]
),𝜔 𝑇 −1
)
+𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[𝑓 𝑇 (𝑥 𝑇 (𝜔 [2:𝑇 ]
),𝜔 𝑇 )]]]
(B-10)

The decomposition property of the expectation operator and the interchangeability property
of the expectation and minimization operators lead to the following equivalent (nested)
formulation of the multistage problem:
min
𝑥 1
𝜖 𝜒 1
𝑓 1
(𝑥 1
)+𝐸 𝜔 2
[ min
𝑥 2
𝜖 𝒳 2
(𝑥 1
,𝜔 2
)
𝑓 2
(𝑥 2
,𝜔 2
)
+𝐸 𝜔 3
|𝜔 2
[…+𝐸 𝜔 𝑇 |𝜔 2
,…,𝜔 𝑇 −1
[ min
𝑥 𝑇 𝜖 𝒳 𝑇 (𝑥 [1:𝑇 −1]
,𝜔 [2:𝑇 ]
)
𝑓 𝑇 (𝑥 𝑇 ,𝜔 𝑇 )]]]
(B-11)

where 𝒳 𝑡 (𝑥 [1:𝑡 −1]
,𝜔 [2:𝑡 ]
) is the feasible set for decision variable of stage 𝑡 that depends of
all decisions and random outcomes between stage 1 to 𝑡 , and 𝑓 𝑡 (𝑥 𝑡 ,𝜔 𝑡 ) is the cost function for
stage 𝑡 that depends on the decision and random outcomes for that stage.

Abstract (if available)

Abstract Reservoir simulation is a valuable tool for model-based field development and production performance optimization. In recent years, significant progress has been made in developing automated workflows for production optimization and field development by combining reservoir simulation forecasts with numerical optimization schemes. While field development optimization under geologic uncertainty has received considerable attention, to-date future developments and their associated uncertainties have not been considered explicitly in field development optimization. In practice, reservoirs undergo extensive development activities throughout their life-cycle. Disregarding the possibility of future developments can lead to field performance predictions and optimization results that may be far from optimal. A complexity in accounting for future developments is related to the uncertainty in development plans. ❧ In this work, we develop a stochastic field development optimization formulation to account for the uncertainty in future infill drilling scenarios. The proposed approach optimizes the decision variables for current stage of planning (e.g. well locations and operational settings) while accounting for future development uncertainties, where the uncertainty is represented through drilling scenario trees and probabilistic description of future drilling events/parameters. In the developed method, for sequential field development under uncertainty in future decisions, multi-stage stochastic programming is implemented, in which the decision-maker selects an optimal strategy (e.g. well locations and operational controls) for current stage while accounting for the residual uncertainty in future development activities. Using a multi-stage stochastic optimization workflow this process is repeated after each decision stage. Several numerical experiments are presented to discuss various aspects of the proposed stochastic optimization formulation and to compare the solutions from different methods adopted for treatment of future development plans. The results indicate that stochastic treatment of future development events (1) can hedge against uncertain future development activities by obtaining optimization solutions that are robust against changes in future decisions, and (2) considerably reduces the performance losses that can result from field development when uncertainty is disregarded. ❧ The proposed approach offers a fresh perspective on formulating and solving production optimization problems to explicitly incorporate the uncertainty in future development activities.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

$Optimization of CO2 storage efficiency under geomechanical risks using coupled flow-geomechanics-fracturing model$

PDF

Optimization of CO2 storage efficiency under geomechanical risks using coupled flow-geomechanics-fracturing model

PDF

Application of data-driven modeling in basin-wide analysis of unconventional resources, including domain expertise

PDF

Efficient control optimization in subsurface flow systems with machine learning surrogate models

PDF

Accurate and efficient uncertainty quantification of subsurface fluid flow via the probabilistic collocation method

PDF

Design optimization under uncertainty for rotor blades of horizontal axis wind turbines

PDF

Uncertainty quantification and data assimilation via transform process for strongly nonlinear problems

PDF

Deep learning architectures for characterization and forecasting of fluid flow in subsurface systems

PDF

Feature learning for imaging and prior model selection

PDF

Real-time reservoir characterization and optimization during immiscible displacement processes

$Coarse-scale simulation of heterogeneous reservoirs and multi-fractured horizontal wells$

PDF

Coarse-scale simulation of heterogeneous reservoirs and multi-fractured horizontal wells

PDF

Subsurface model calibration for complex facies models

PDF

Optimization of coupled CO₂ sequestration and enhanced oil recovery

PDF

I. Asynchronous optimization over weakly coupled renewal systems

PDF

Comprehensive uncertainty quantification in composites manufacturing processes

PDF

Integration of multi-physics data into dynamic reservoir model with ensemble Kalman filter

PDF

The fusion of predictive and prescriptive analytics via stochastic programming

PDF

The next generation of power-system operations: modeling and optimization innovations to mitigate renewable uncertainty

PDF

Waterflood induced formation particle transport and evolution of thief zones in unconsolidated geologic layers

PDF

Computational stochastic programming with stochastic decomposition

PDF

Asphaltene deposition during co₂ injection in enhanced oil recovery applications

Asset Metadata

Creator Jahandideh, Atefeh (author)

Core Title Stochastic oilfield optimization under uncertainty in future development plans

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Petroleum Engineering

Publication Date 04/26/2019

Defense Date 02/15/2019

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag OAI-PMH Harvest,oilfield development optimization,optimization under uncertainty,production optimization,stochastic programming

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Jafarpour, Behnam (committee chair), Ershaghi, Iraj (committee member), Ghanem, Roger (committee member)

Creator Email atefeh.jahandideh@gmail.com,jahandid@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-152041

Unique identifier UC11660745

Identifier etd-Jahandideh-7295.pdf (filename),usctheses-c89-152041 (legacy record id)

Legacy Identifier etd-Jahandideh-7295.pdf

Dmrecord 152041

Document Type Dissertation

Format application/pdf (imt)

Rights Jahandideh, Atefeh

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA