{"id":2086,"date":"2023-06-21T11:22:14","date_gmt":"2023-06-21T11:22:14","guid":{"rendered":"https:\/\/nag.com\/?page_id=2086"},"modified":"2024-03-07T14:49:01","modified_gmt":"2024-03-07T14:49:01","slug":"optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares","status":"publish","type":"page","link":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/","title":{"rendered":"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares)"},"content":{"rendered":"\n
\n
\n
\n
\n
\n
\n \n

Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares)<\/h1>\n \n
<\/div>\n\n \n <\/div>\n <\/div>\n <\/div>\n <\/div>\n <\/div>\n<\/div>\n\n\n\n
<\/div>\n\n
\n
\n
\n

A search on the web will quickly reveal numerous applications for a routine which finds the best fit vector\u00a0x<\/em>\u00a0to a system of linear equations where the components of\u00a0x<\/em>\u00a0are constrained to be non-negative. For example, statisticians may wish to fit a linear regression model:<\/p>\n

y<\/em>=Xc<\/em>\u00a0+\u00a0e<\/em><\/strong><\/p>\n

where\u00a0y<\/em>\u00a0is a vector of observations,\u00a0X<\/em>\u00a0a design matrix and\u00a0e<\/em>\u00a0a vector of noise. If\u00a0c<\/em>\u00a0represents inherently non-negative quantities, such as prices, growth rates or pixel intensity perhaps, then a constraint might be that the elements of\u00a0c<\/em>\u00a0be non-negative. Such a problem is termed a non-negative (linear) least squares problem.<\/p>\n

We may similarly find situations where, additionally, the \u2018solution\u2019 should not have components exceeding a given value. Such a requirement defines a bounded-variable (linear) least squares problem.<\/p>\n

The n<\/i><\/span>AG optimization chapters contain very powerful and general routines, so such problems could always be cast into a form where the problems could be addressed by these routines. When we did this at the request of a user we found, not unsurprisingly, that these non-specialist routines were not as efficient as a routine designed specifically for such problems.<\/p>\n

Consequently, we decided to add to the functionality of the n<\/i><\/span>AG Library by including a specialist routine for bounded-variable least squares problems. This routine is e04pc and is designed for problem sizes that are not too big. (Dense matrices and techniques are employed throughout. Depending upon user-demand, we will consider specialist sparse algorithms for larger problems at future marks.)<\/p>\n

One feature of these types of problems is that the solution may not be unique. This occurs when the matrix (X<\/em>\u00a0above) is not of full rank. This may, or may not, be a problem, since the actual minimum will be the same, merely achieved by different solution values,\u00a0c<\/em>. If uniqueness is required, then n<\/i><\/span>AG provides a facility to return that solution which minimizes the Euclidean norm of\u00a0c,<\/em>\u00a0i.e. the minimal length solution. This facility distinguishes the n<\/i><\/span>AG routine from others on the market, but, for efficiency reasons, the option should not be employed unless a unique solution is really necessary.<\/p>\n <\/div>\n <\/div>\n<\/div>\n\n\n

\n
\n
\n
\n
\n
\n \n \n
<\/div>\n\n \n What’s new in the n<\/i><\/span>AG Library <\/i><\/a> <\/div>\n <\/div>\n <\/div>\n <\/div>\n <\/div>\n<\/div>\n\n\n\n
<\/div>","protected":false},"excerpt":{"rendered":"

Quadratic functions are a powerful modelling construct in mathematical programming and appear in various disciplines such as statistics, machine learning (Lasso regression), finance (portfolio optimization), engineering (OPF) and control theory. At Mark 27.1 of the n<\/i><\/span>AG Library, n<\/i><\/span>AG introduced two new additions to the n<\/i><\/span>AG Optimization Modelling Suite to help users easily define quadratic objective functions and\/or constraints, seamlessly integrate them with other constraints and solve the resulting problems using compatible solvers without the need of a reformulation or any extra effort.<\/p>\n","protected":false},"author":3,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"content-type":"","footnotes":""},"class_list":["post-2086","page","type-page","status-publish","hentry"],"yoast_head":"\nOptimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares) - nAG<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares) - nAG\" \/>\n<meta property=\"og:description\" content=\"Quadratic functions are a powerful modelling construct in mathematical programming and appear in various disciplines such as statistics, machine learning (Lasso regression), finance (portfolio optimization), engineering (OPF) and control theory. At Mark 27.1 of the NAG Library, NAG introduced two new additions to the NAG Optimization Modelling Suite to help users easily define quadratic objective functions and\/or constraints, seamlessly integrate them with other constraints and solve the resulting problems using compatible solvers without the need of a reformulation or any extra effort.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/\" \/>\n<meta property=\"og:site_name\" content=\"nAG\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-07T14:49:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/nag.com\/wp-content\/uploads\/2024\/02\/NAG-Logo-White-On-Blue.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"12770\" \/>\n\t<meta property=\"og:image:height\" content=\"4353\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@NAGTalk\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/\",\"url\":\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/\",\"name\":\"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares) - nAG\",\"isPartOf\":{\"@id\":\"https:\/\/nag.com\/#website\"},\"datePublished\":\"2023-06-21T11:22:14+00:00\",\"dateModified\":\"2024-03-07T14:49:01+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/nag.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/nag.com\/#website\",\"url\":\"https:\/\/nag.com\/\",\"name\":\"NAG\",\"description\":\"Robust, trusted numerical software and computational expertise.\",\"publisher\":{\"@id\":\"https:\/\/nag.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/nag.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/nag.com\/#organization\",\"name\":\"Numerical Algorithms Group\",\"alternateName\":\"NAG\",\"url\":\"https:\/\/nag.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"width\":1244,\"height\":397,\"caption\":\"Numerical Algorithms Group\"},\"image\":{\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/NAGTalk\",\"https:\/\/www.linkedin.com\/company\/nag\/\",\"https:\/\/www.youtube.com\/user\/NumericalAlgorithms\",\"https:\/\/github.com\/numericalalgorithmsgroup\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares) - nAG","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/","og_locale":"en_US","og_type":"article","og_title":"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares) - nAG","og_description":"Quadratic functions are a powerful modelling construct in mathematical programming and appear in various disciplines such as statistics, machine learning (Lasso regression), finance (portfolio optimization), engineering (OPF) and control theory. At Mark 27.1 of the NAG Library, NAG introduced two new additions to the NAG Optimization Modelling Suite to help users easily define quadratic objective functions and\/or constraints, seamlessly integrate them with other constraints and solve the resulting problems using compatible solvers without the need of a reformulation or any extra effort.","og_url":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/","og_site_name":"nAG","article_modified_time":"2024-03-07T14:49:01+00:00","og_image":[{"width":12770,"height":4353,"url":"https:\/\/nag.com\/wp-content\/uploads\/2024\/02\/NAG-Logo-White-On-Blue.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@NAGTalk","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/","url":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/","name":"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares) - nAG","isPartOf":{"@id":"https:\/\/nag.com\/#website"},"datePublished":"2023-06-21T11:22:14+00:00","dateModified":"2024-03-07T14:49:01+00:00","breadcrumb":{"@id":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/nag.com\/optimization-solutions-for-non-negative-least-squares-problems-bounded-variable-least-squares\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nag.com\/"},{"@type":"ListItem","position":2,"name":"Optimization solutions for Non-negative Least Squares problems (Bounded \u2013 Variable Least Squares)"}]},{"@type":"WebSite","@id":"https:\/\/nag.com\/#website","url":"https:\/\/nag.com\/","name":"NAG","description":"Robust, trusted numerical software and computational expertise.","publisher":{"@id":"https:\/\/nag.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nag.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/nag.com\/#organization","name":"Numerical Algorithms Group","alternateName":"NAG","url":"https:\/\/nag.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","width":1244,"height":397,"caption":"Numerical Algorithms Group"},"image":{"@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/NAGTalk","https:\/\/www.linkedin.com\/company\/nag\/","https:\/\/www.youtube.com\/user\/NumericalAlgorithms","https:\/\/github.com\/numericalalgorithmsgroup"]}]}},"_links":{"self":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages\/2086","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/comments?post=2086"}],"version-history":[{"count":3,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages\/2086\/revisions"}],"predecessor-version":[{"id":5862,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/pages\/2086\/revisions\/5862"}],"wp:attachment":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media?parent=2086"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}