Fundamental Engineering Optimization Methods

This book is addressed to students in fields of engineering and technology as well as practicing engineers. It covers the fundamentals of commonly used optimization methods used in engineering design. Optimization methods fall among the mathematical tools typically used to solve engineering problems. It is therefore desirable that graduating students and practicing engineers are equipped with these tools and are trained to apply them to specific problems encountered in engineering practice.

Optimization is an integral part of the engineering design process. It focuses on discovering optimum solutions to a design problem through systematic consideration of alternatives, while satisfying resource and cost constraints. Many engineering problems are open-ended and complex. The overall design objective in these problems may be to minimize cost, to maximize profit, to streamline production, to increase process efficiency, etc. Finding an optimum solution requires a careful consideration of several alternatives that are often compared on multiple criteria.

Mathematically, the engineering design optimization problem is formulated by identifying a cost function of several optimization variables whose optimal combination results in the minimal cost. The resource and other constraints are similarly translated into mathematical relations. Once the cost function and the constraints have been correctly formulated, analytical, computational, or graphical methods may be employed to find an optimum. The challenge in complex optimization problems is finding a global minimum, which may be elusive due to the complexity and nonlinearity of the problem.

This book covers the fundamentals of optimization methods for solving engineering problems. Written by an engineer, it introduces fundamentals of mathematical optimization methods in a manner that engineers can easily understand. The treatment of the topics presented here is both selective and concise. The material is presented roughly at senior undergraduate level. Readers are expected to have familiarity with linear algebra and multivariable calculus. Background material has been reviewed in Chapter 2.

The methods covered in this book include a) analytical methods that are based on calculus of variations; b) graphical methods that are useful when minimizing functions involving a small number of variables; and c) iterative methods that are computer friendly, yet require a good understanding of the problem. Both linear and nonlinear methods are covered. Where necessary, engineering examples have been used to build an understanding of how these methods can be applied. Though not written as text, it may be used as text if supplemented by additional reading and exercise problems from the references.

There are many good references available on the topic of optimization methods. A short list of prominent books and internet resources appears in the reference section. The following references are main sources for this manuscript and the topics covered therein Arora (2012); Belegundu and Chandrupatla (2012); Chong and Zak (2013); and, Griva, Nash & Sofer (2009). In addition, lecture notes of eminent professors who have regularly taught optimization classes are available on the internet. For details, the interested reader may refer to these references or other web resources on the topic.

• Preface

1. Engineering Design Optimization
   1. Introduction
   2. Optimization Examples in Science and Engineering
   3. Notation
2. Mathematical Preliminaries
   1. Set Definitions
   2. Function Definitions
   3. Taylor Series Approximation
   4. Gradient Vector and Hessian Matrix
   5. Convex Optimization Problems
   6. Vector and Matrix Norms
   7. Matrix Eigenvalues and Singular Values
   8. Quadratic Function Forms
   9. Linear Systems of Equations
   10. Linear Diophantine System of Equations
   11. Condition Number and Convergence Rates
   12. Conjugate-Gradient Method for Linear Equations
   13. Newton’s Method for Nonlinear Equations
3. Graphical Optimization
   1. Functional Minimization in One-Dimension
   2. Graphical Optimization in Two-Dimensions
4. Mathematical Optimization
   1. The Optimization Problem
   2. Optimality criteria for the Unconstrained Problems
   3. Optimality Criteria for the Constrained Problems
   4. Optimality Criteria for General Optimization Problems
   5. Postoptimality Analysis
   6. Lagrangian Duality
5. Linear Programming Methods
   1. The Standard LP Problem
   2. The Basic Solution to the LP Problem
   3. The Simplex Method
   4. Postoptimality Analysis
   5. Duality Theory for the LP Problems
   6. Non-Simplex Methods for Solving LP Problems
   7. Optimality Conditions for LP Problems
   8. The Quadratic Programming Problem
   9. The Linear Complementary Problem
6. Discrete Optimization
   1. Discrete Optimization Problems
   2. Solution Approaches to Discrete Problems
   3. Linear Programming Problems with Integral Coefficients
   4. Integer Programming Problems
7. Numerical Optimization Methods
   1. The Iterative Method
   2. Computer Methods for Solving the Line Search Problem
   3. Computer Methods for Finding the Search Direction
   4. Computer Methods for Solving the Constrained Problems
   5. Sequential Linear Programming
   6. Sequential Quadratic Programming

• References