{"id":1061,"date":"2023-05-30T08:40:30","date_gmt":"2023-05-30T08:40:30","guid":{"rendered":"https:\/\/nag.com\/?page_id=1061"},"modified":"2023-07-14T13:39:23","modified_gmt":"2023-07-14T13:39:23","slug":"derivative-free-optimization-dfo","status":"publish","type":"page","link":"https:\/\/nag.com\/derivative-free-optimization-dfo\/","title":{"rendered":"Derivative-free Optimization (DFO)"},"content":{"rendered":"\n<div class=\"gbc-title-banner ta ta-lg ta-xl\" style='background-color: #082d48ff; color: #ffffffff; border-radius: 0px; '>\n    <div class=\"container\" style='border-radius: 0px; '>\n        <div class=\"row justify-content--center\" style='color: #ffffffff;'>\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-4 \" style=\"0px\">\n                                <div class=\"col-12 col-md-12 col-lg-10 col-xl-8  banner-content\"  >\n    \n                                             <h1>Derivative-free Optimization (DFO)<\/h1>\n                    \n                    <div class=\"mt-1 mb-1 content\"><\/div>\n\n                    \n                                    <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-12 col-lg-10 col-xl-8\">\n            <p>Optimizing complex numerical models is one of the most common problems found in the industry (finance, multi-physics simulations, engineering, etc.). To solve these optimization problems with a standard optimization algorithm such as Gauss\u2013Newton (for problems with a nonlinear least squares structure) or CG (for unstructured nonlinear objective) requires good estimates of the model&#8217;s derivatives. They can be computed by:<\/p>\n<ul>\n<li>explicitly written derivatives<\/li>\n<li>algorithmic differentiation (<a href=\"https:\/\/nag.com\/automatic-differentiation\">see <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG AD tools<\/a>)<\/li>\n<li>finite differences (bumping),\u00a0<span class=\"math inline\">\\(\\frac{\\partial \\phi}{\\partial x_i} \\approx \\frac{\\phi(x+he_i) &#8211; \\phi(x)}{h}\\)<\/span><\/li>\n<\/ul>\n<p>If exact derivatives are easy to compute then using derivative-based methods is preferable. However, explicitly writing the derivatives or applying AD methods might be impossible if the model is a black box. The alternative, estimating derivatives via finite differences, can quickly become impractical or too computationally expensive as it presents several issues:<\/p>\n<ul>\n<li><b>Expensive<\/b>, one gradient evaluation requires at least\u00a0<span class=\"math inline\">\\(n+1\\)<\/span>\u00a0model evaluations;<\/li>\n<li><b>Inaccurate<\/b>, the size of the model perturbations\u00a0<span class=\"math inline\">\\(h\\)<\/span>\u00a0influences greatly the quality of the derivative estimations and is not easy to choose;<\/li>\n<li><b>Sensitive to noise<\/b>, if the model is subject to some randomness (e.g. Monte Carlo simulations) or is computed to low accuracy to save computing time, then finite differences estimations will be highly inaccurate;<\/li>\n<li><b>Poor utilization of model evaluations<\/b>, each evaluation is only used for one element of one gradient and the information is discarded as soon as that gradient is no longer useful to the solver.<\/li>\n<\/ul>\n<p>These issues can greatly slow down the convergence of the optimization solver or even completely prevent it. Conversely, DFO solvers are designed to get good improvements of the objective in these situations. They are able to reach convergence with a lot fewer function evaluations and are naturally quite robust to noise in the model evaluations.<\/p>\n<h3><span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG Library Mark 27<\/h3>\n<p><span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG introduces, at Mark 27, a complete update of its model-based DFO solvers for nonlinear least squares problems,\u00a0<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04ffc.html\">nag_opt_handle_solve_dfls<\/a>\u00a0(<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04ffc.html\">e04ff<\/a>) and\u00a0<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04fgc.html\">nag_opt_handle_solve_dfls_rcomm<\/a>\u00a0(<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04fgc.html\">e04fg<\/a>), and unstructured nonlinear problems,\u00a0<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04jdc.html\">nag_opt_handle_solve_dfno<\/a>\u00a0(<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04jdc.html\">e04jd<\/a>) and\u00a0<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04jec.html\">nag_opt_handle_solve_dfno_rcomm<\/a>\u00a0(<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04jec.html\">e04je<\/a>). They present a number of attractive features:<\/p>\n<ul>\n<li>Improved performance;<\/li>\n<li>Integrated to the\u00a0<i><span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG Optimization Modeling Suite<\/i>\u00a0with simple interfaces for the solvers and related routines;<\/li>\n<li>Optional reverse communication interface (<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04fgc.html\">e04fg<\/a>,\u00a0<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04jec.html\">e04je<\/a>);<\/li>\n<li>Able to start making progress with as few as two objective evaluations;<\/li>\n<li>Improved resilience to noise.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<div class=\"gbc-title-banner ta ta-lg ta-xl\" style='border-radius: 0px; '>\n    <div class=\"container\" style='border-radius: 0px; '>\n        <div class=\"row justify-content--center\" >\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-4 \" style=\"0pxbackground-color: \">\n                                <div class=\"col-12 col-md-12 col-lg-10 col-xl-8  banner-content\"  >\n    \n                    \n                    <div class=\"mt-1 mb-1 content\"><\/div>\n\n                    \n                    <a href='https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04ffc.html' style='background-color: #ff7d21ff; color: #ffffffff; border-radius: 30px; font-weight: 600; ' class='btn mr-1  ' >DFO Documentation <i class='fas fa-angle-right'><\/i><\/a>                <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-12 col-lg-10 col-xl-8\">\n            <h2>Problems structure<\/h2>\n<p>One frequent problem in practice is tuning a model&#8217;s parameters to fit real world observations as well as possible. Let us consider a process that is observed at times\u00a0<span class=\"math inline\">\\(t_i\\)<\/span>\u00a0and measured with results\u00a0<span class=\"math inline\">\\(y_i\\)<\/span>, for\u00a0<span class=\"math inline\">\\(i=1,2,\\dots,m\\)<\/span>. Furthermore, the process is assumed to behave according to a numerical model\u00a0<span class=\"math inline\">\\(\\phi(t,x)\\)<\/span>\u00a0where\u00a0<span class=\"math inline\">\\(x\\)<\/span>\u00a0are parameters of the model. Given the fact that the measurements might be inaccurate and the process might not exactly follow the model, it is beneficial to find model parameters\u00a0<span class=\"math inline\">\\(x\\)<\/span>\u00a0so that the error of the fit of the model to the measurements is minimized. This can be formulated as an optimization problem in which\u00a0<span class=\"math inline\">\\(x\\)<\/span>\u00a0is the decision variables and the objective function is the sum of squared errors of the fit at each individual measurement, thus:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n\n<math class=\"equation\" display=\"block\" xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"> <mtable columnalign=\"left\"> <mtr> <mtd> <munder> <mi mathvariant=\"normal\">minimize<\/mi> <mrow> <mi>x<\/mi> <mo>\u2208<\/mo> <msup><mo>\u211d<\/mo><mi>n<\/mi><\/msup> <\/mrow> <\/munder><mspace width=\"0.25em\"><\/mspace> <\/mtd> <mtd> <munderover> <mo>\u2211<\/mo> <mrow><mi>i<\/mi><mo>=<\/mo><mn>1<\/mn><\/mrow> <mi>m<\/mi> <\/munderover> <msup> <mrow><mo>(<\/mo> <mi>\u03d5<\/mi><mo>(<\/mo><msub><mi>t<\/mi><mi>i<\/mi><\/msub><mo>,<\/mo><mi>x<\/mi><mo>)<\/mo><mo>&#8211;<\/mo><msub><mi>y<\/mi><mi>i<\/mi><\/msub> <mo>)<\/mo><\/mrow> <mn>2<\/mn> <\/msup> <\/mtd> <\/mtr> <mtr> <mtd><mtext>subject to<\/mtext><\/mtd> <mtd> <mi>l<\/mi> <mo>\u2264<\/mo> <mi>x<\/mi> <mo>\u2264<\/mo> <mi>u<\/mi> <\/mtd> <\/mtr> <\/mtable> <\/math>\n\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-12 col-lg-10 col-xl-8\">\n            <p>When the optimization problem cannot be written as nonlinear least squares, a more general formulation has to be used:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n\n<math class=\"equation\" display=\"block\" xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"> <mtable columnalign=\"left\"> <mtr> <mtd> <munder> <mi mathvariant=\"normal\">minimize<\/mi> <mrow><mi>x<\/mi><mo>\u2208<\/mo><msup><mo>\u211d<\/mo><mi>n<\/mi><\/msup><\/mrow> <\/munder><mspace width=\"0.25em\"><\/mspace> <\/mtd> <mtd> <mi>f<\/mi> <mo>(<\/mo><mi>x<\/mi><mo>)<\/mo> <\/mtd> <\/mtr> <mtr> <mtd><mtext>subject to<\/mtext><\/mtd> <mtd> <mi>l<\/mi> <mo>\u2264<\/mo> <mi>x<\/mi> <mo>\u2264<\/mo> <mi>u<\/mi> <\/mtd> <\/mtr> <\/mtable> <\/math>\n\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-12 col-lg-10 col-xl-8\">\n            <div class=\"paragraph--color--transparent paragraph--alignment--left paragraph paragraph--type--text paragraph--view-mode--default\">\n<div class=\"field field--name-field-paragraph-text field--type-text-long field--label-hidden field--item\">\n<div class=\"tex2jax_process\">\n<p>The <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG solvers accommodate both of these formulations.<\/p>\n<\/div>\n<\/div>\n<\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-12 col-lg-10 col-xl-8\">\n            <h2>Noise robustness: illustration<\/h2>\n<p>Consider the following unbounded problem where\u00a0<span class=\"math inline\">\\(\\epsilon\\)<\/span>\u00a0is some random uniform noise in the interval\u00a0<span class=\"math inline\">\\(\\left[-\\nu,\\nu\\right]\\)<\/span>\u00a0and\u00a0<span class=\"math inline\">\\(r_i\\)<\/span>\u00a0are the residuals of the Rosenbrock test function.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n\n<math class=\"equation\" display=\"block\" xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"> <munder> <mi mathvariant=\"normal\">minimize<\/mi> <mrow> <mi>x<\/mi> <mo>\u2208<\/mo> <msup><mo>\u211d<\/mo><mi>n<\/mi><\/msup> <\/mrow> <\/munder> <mrow><mi>f<\/mi> <mo>(<\/mo><mi>x<\/mi><mo>)<\/mo><\/mrow> <mo>=<\/mo> <munderover> <mo>\u2211<\/mo> <mrow><mi>i<\/mi><mo>=<\/mo><mn>1<\/mn><\/mrow> <mi>m<\/mi> <\/munderover> <msup> <mrow><mo>(<\/mo> <msub><mi>r<\/mi><mi>i<\/mi><\/msub> <mo>(<\/mo><mi>x<\/mi><mo>)<\/mo><mo>+<\/mo><mi>\u03b5<\/mi> <mo>)<\/mo><\/mrow> <mn>2<\/mn> <\/msup> <\/math>\n\n\n<!-- Spacer -->\n<div class=\"pt-4 pt-lg-4 pt-xl-4\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-12 col-lg-10 col-xl-8\">\n            <div class=\"paragraph--color--transparent paragraph--alignment--left paragraph paragraph--type--text paragraph--view-mode--default\">\n<div class=\"field field--name-field-paragraph-text field--type-text-long field--label-hidden field--item\">\n<div class=\"tex2jax_process\">\n<p>Let us solve this problem with a Gauss\u2013Newton method combined with finite differences,\u00a0<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04fcc.html\">nag_opt_lsq_uncon_mod_func_comp<\/a>\u00a0(<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04fcc.html\">e04fc<\/a>) and the corresponding derivative-free solver (<a href=\"https:\/\/support.nag.com\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04ffc.html\">e04ff<\/a>). For various noise level\u00a0<span class=\"math inline\">\\(\\nu\\)<\/span>, we present in the following table the number of model evaluations needed to find a solution with an &#8216;achievable&#8217; precision with respect to the noise level:\u00a0<\/p>\n<p><span class=\"math display\">\\[ f(x) &lt; \\max (10^{-8}, 10 \\times \\nu^2) \\]<\/span><\/p>\n<p>The figures in Table 1 show the average (on 50 runs) of number objective evaluations to solve the problem. The number in brackets represent the number of failed runs out of 50.<\/p>\n<table id=\"TBL-1\" class=\"tabular\" style=\"border: thin solid #000000;\" border=\"thin solid rgb(0, 0, 0)\" rules=\"groups\" cellspacing=\"0\" cellpadding=\"0\">\n<caption>\u00a0<\/caption>\n<tbody>\n<tr>\n<td style=\"width: 7.93358%; border: thin solid #000000;\">noise level\u00a0<span class=\"math inline\">\\(\\nu\\)<\/span><\/td>\n<td style=\"width: 4.61255%; border: thin solid #000000;\"><span class=\"math inline\">\\(0\\)<\/span><\/td>\n<td style=\"width: 13.8376%; border: thin solid #000000;\"><span class=\"math inline\">\\(10^{-10}\\)<\/span><\/td>\n<td style=\"width: 12.1771%; border: thin solid #000000;\"><span class=\"math inline\">\\(10^{-8}\\)<\/span><\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(10^{-6}\\)<\/span><\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(10^{-4}\\)<\/span><\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(10^{-2}\\)<\/span><\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(10^{-1}\\)<\/span><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 7.93358%; border: thin solid #000000;\"><a href=\"https:\/\/www.nag.co.uk\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04fcc.html\">e04fc<\/a><\/td>\n<td style=\"width: 4.61255%; border: thin solid #000000;\">90 (0)<\/td>\n<td style=\"width: 13.8376%; border: thin solid #000000;\">93 (0)<\/td>\n<td style=\"width: 12.1771%; border: thin solid #000000;\">240 (23)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(\\infty\\)<\/span>\u00a0(50)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(\\infty\\)<\/span>\u00a0(50)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(\\infty\\)<\/span>\u00a0(50)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\"><span class=\"math inline\">\\(\\infty\\)<\/span>\u00a0(50)<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 7.93358%; border: thin solid #000000;\"><a href=\"https:\/\/www.nag.co.uk\/numeric\/nl\/nagdoc_latest\/clhtml\/e04\/e04ffc.html\">e04ff<\/a><\/td>\n<td style=\"width: 4.61255%; border: thin solid #000000;\">22 (0)<\/td>\n<td style=\"width: 13.8376%; border: thin solid #000000;\">22 (0)<\/td>\n<td style=\"width: 12.1771%; border: thin solid #000000;\">22 (0)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\">22 (0)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\">22 (0)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\">17 (0)<\/td>\n<td style=\"width: 15.3137%; border: thin solid #000000;\">15 (0)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p class=\"para-sm\"><span class=\"id\">Table\u00a01:\u00a0<\/span><span class=\"content\">Number of objective evaluations required to reach the desired accuracy. Numbers in the brackets represent amount of failed runs.<\/span><\/p>\n<p>On this example, the new derivative-free solver is both cheaper in terms of model evaluations and far more robust with respect to noise.<\/p>\n<\/div>\n<\/div>\n<\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<div class=\"gbc-title-banner ta ta-lg ta-xl\" style='border-radius: 0px; '>\n    <div class=\"container\" style='border-radius: 0px; '>\n        <div class=\"row justify-content--center\" >\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-4 \" style=\"0pxbackground-color: \">\n                                <div class=\"col-12 col-md-12 col-lg-10 col-xl-8  banner-content\"  >\n    \n                    \n                    <div class=\"mt-1 mb-1 content\"><\/div>\n\n                    \n                    <a href='https:\/\/support.nag.com\/market\/posters\/derivative_free_optimization_solver_calibration_problems.pdf' style='background-color: #ff7d21ff; color: #ffffffff; border-radius: 30px; font-weight: 600; ' class='btn mr-1  ' >DFO Poster <i class='fas fa-angle-right'><\/i><\/a>                <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n\n<div class=\"gbc-title-banner ta ta-lg ta-xl\" style='color: #ffffffff; border-radius: 20px; '>\n    <div class=\"container\" style='border-radius: 20px; '>\n        <div class=\"row justify-content--center\" style='color: #ffffffff;'>\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-4 \" style=\"border-radius: 20px;background-color: #082d48ff\">\n                                <div class=\"col-12 col-md-12 col-lg-10 col-xl-8  banner-content\"  >\n    \n                                             <h2>References<\/h2>\n                    \n                    <div class=\"mt-1 mb-1 content\"><p>[1]\u00a0\u00a0\u00a0C. Cartis, J. Fiala, B. Marteau, and L. Roberts\u00a0Improving the Flexibility and robustness of model-based derivative-free optimization solvers\u00a0ACM Transactions On Numerical Software. 2019.<\/p>\n<p>[2]\u00a0\u00a0\u00a0C. Cartis and L. Roberts\u00a0A derivative-free Gauss\u2013Newton method\u00a0Mathematical Programming Computation. 2017.<\/p>\n<p>[3]\u00a0\u00a0\u00a0Powell M. J. D.\u00a0The BOBYQA algorithm for bound constrained optimization without derivatives\u00a0Report DAMTP 2009\/NA06 University of Cambridge. 2009.<\/p>\n<p>\u00a0<\/p>\n<\/div>\n\n                    \n                    <a href='https:\/\/support.nag.com\/form\/watch-webinar' style='background-color: #ff7d21ff; border-radius: 30px; font-weight: 600; ' class='btn mr-1  ' >Watch the Webinar <i class='fas fa-angle-right'><\/i><\/a>                <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>minimize x \u2208 \u211dn \u2211 i=1 m ( \u03d5(ti,x)&#8211;yi ) 2 subject to l \u2264 x \u2264 u minimize x\u2208\u211dn f (x) subject to l \u2264 x \u2264 u minimize x \u2208 \u211dn f (x) = \u2211 i=1 m ( ri (x)+\u03b5 ) 2<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"content-type":"","footnotes":""},"class_list":["post-1061","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Derivative-free Optimization (DFO) - nAG<\/title>\n<meta name=\"description\" content=\"Optimizing complex numerical models is one of the most common problems found in the industry.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nag.com\/derivative-free-optimization-dfo\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Derivative-free Optimization (DFO) - nAG\" \/>\n<meta property=\"og:description\" content=\"Optimizing complex numerical models is one of the most common problems found in the industry.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nag.com\/derivative-free-optimization-dfo\/\" \/>\n<meta property=\"og:site_name\" content=\"nAG\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-14T13:39:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/nag.com\/wp-content\/uploads\/2024\/02\/NAG-Logo-White-On-Blue.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"12770\" \/>\n\t<meta property=\"og:image:height\" content=\"4353\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@NAGTalk\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/nag.com\/derivative-free-optimization-dfo\/\",\"url\":\"https:\/\/nag.com\/derivative-free-optimization-dfo\/\",\"name\":\"Derivative-free Optimization (DFO) - nAG\",\"isPartOf\":{\"@id\":\"https:\/\/nag.com\/#website\"},\"datePublished\":\"2023-05-30T08:40:30+00:00\",\"dateModified\":\"2023-07-14T13:39:23+00:00\",\"description\":\"Optimizing complex numerical models is one of the most common problems found in the industry.\",\"breadcrumb\":{\"@id\":\"https:\/\/nag.com\/derivative-free-optimization-dfo\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/nag.com\/derivative-free-optimization-dfo\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/nag.com\/derivative-free-optimization-dfo\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/nag.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Derivative-free Optimization (DFO)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/nag.com\/#website\",\"url\":\"https:\/\/nag.com\/\",\"name\":\"NAG\",\"description\":\"Robust, trusted numerical software and computational expertise.\",\"publisher\":{\"@id\":\"https:\/\/nag.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/nag.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/nag.com\/#organization\",\"name\":\"Numerical Algorithms Group\",\"alternateName\":\"NAG\",\"url\":\"https:\/\/nag.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"width\":1244,\"height\":397,\"caption\":\"Numerical Algorithms Group\"},\"image\":{\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/NAGTalk\",\"https:\/\/www.linkedin.com\/company\/nag\/\",\"https:\/\/www.youtube.com\/user\/NumericalAlgorithms\",\"https:\/\/github.com\/numericalalgorithmsgroup\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Derivative-free Optimization (DFO) - nAG","description":"Optimizing complex numerical models is one of the most common problems found in the industry.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nag.com\/derivative-free-optimization-dfo\/","og_locale":"en_US","og_type":"article","og_title":"Derivative-free Optimization (DFO) - nAG","og_description":"Optimizing complex numerical models is one of the most common problems found in the industry.","og_url":"https:\/\/nag.com\/derivative-free-optimization-dfo\/","og_site_name":"nAG","article_modified_time":"2023-07-14T13:39:23+00:00","og_image":[{"width":12770,"height":4353,"url":"https:\/\/nag.com\/wp-content\/uploads\/2024\/02\/NAG-Logo-White-On-Blue.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@NAGTalk","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/nag.com\/derivative-free-optimization-dfo\/","url":"https:\/\/nag.com\/derivative-free-optimization-dfo\/","name":"Derivative-free Optimization (DFO) - nAG","isPartOf":{"@id":"https:\/\/nag.com\/#website"},"datePublished":"2023-05-30T08:40:30+00:00","dateModified":"2023-07-14T13:39:23+00:00","description":"Optimizing complex numerical models is one of the most common problems found in the industry.","breadcrumb":{"@id":"https:\/\/nag.com\/derivative-free-optimization-dfo\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nag.com\/derivative-free-optimization-dfo\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/nag.com\/derivative-free-optimization-dfo\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nag.com\/"},{"@type":"ListItem","position":2,"name":"Derivative-free Optimization (DFO)"}]},{"@type":"WebSite","@id":"https:\/\/nag.com\/#website","url":"https:\/\/nag.com\/","name":"NAG","description":"Robust, trusted numerical software and computational expertise.","publisher":{"@id":"https:\/\/nag.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nag.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/nag.com\/#organization","name":"Numerical Algorithms Group","alternateName":"NAG","url":"https:\/\/nag.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","width":1244,"height":397,"caption":"Numerical Algorithms Group"},"image":{"@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/NAGTalk","https:\/\/www.linkedin.com\/company\/nag\/","https:\/\/www.youtube.com\/user\/NumericalAlgorithms","https:\/\/github.com\/numericalalgorithmsgroup"]}]}},"_links":{"self":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages\/1061","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/comments?post=1061"}],"version-history":[{"count":5,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages\/1061\/revisions"}],"predecessor-version":[{"id":3246,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages\/1061\/revisions\/3246"}],"wp:attachment":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media?parent=1061"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}