{"id":64044,"date":"2025-06-12T09:23:58","date_gmt":"2025-06-12T09:23:58","guid":{"rendered":"https:\/\/nag.com\/?post_type=insights&#038;p=64044"},"modified":"2025-06-12T10:37:41","modified_gmt":"2025-06-12T10:37:41","slug":"non-convex-optimization-and-machine-learning","status":"publish","type":"insights","link":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/","title":{"rendered":"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward"},"content":{"rendered":"<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h2>Non-Convex and Stochastic Optimization in 2025: The Engines of Real-World Intelligence<\/h2>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h3>1\u00a0 The Shape of Complexity in Modern Systems<\/h3>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>Historically, optimization was largely confined to well-structured, convex problems \u2014 settings where theoretical guarantees and algorithmic efficiency aligned neatly. This made sense: algorithms for large-scale linear and convex programs, capable of handling millions of variables and constraints, have matured over decades. In contrast, non-convex problems \u2014 including those involving discrete or combinatorial structures \u2014 remained computationally intractable at scale. But by 2025, the landscape has shifted. Modern commercial systems are increasingly defined by complexity, scale, and uncertainty. As industries move from deterministic, rule-based frameworks to data-driven architectures infused with randomness, two methodological pillars have emerged as essential: <strong>non-convexity<\/strong> and <strong>stochasticity<\/strong>. These form the mathematical foundation for robust, adaptive optimization in the real world.\u00a0<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li><strong>Non-convexity<\/strong> refers to problems where the objective function or constraints exhibit multiple local minima, flat plateaus, or discontinuities. Solving these problems requires escaping local optima and exploring globally.<\/li>\n<li><strong>Stochasticity<\/strong> involves modeling randomness explicitly. This is crucial when data, inputs, or environments are noisy, incomplete, or changing\u2014conditions that prevail in nearly all large-scale operational contexts. The marriage of optimization and stochastic processes is one of the most fruitful in all of applied mathematics.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>These two paradigms \u2014 used alone or together \u2014 form the computational backbone of modern decision-making systems.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h3>2\u00a0 Where It Matters \u2014 Six Fronts of Transformation<\/h3>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>2.1\u00a0 AI and Machine Learning<\/h4>\n<p>Deep learning, the foundation of modern AI, inherently involves non-convex optimization landscapes. Training a neural network means minimizing a high-dimensional loss function riddled with saddle points and local minima. Gradient-based methods such as stochastic gradient descent (SGD) work well in practice, but newer methods like evolutionary strategies and Bayesian optimization are gaining ground for model tuning and hyperparameter search.<\/p>\n<p>Additionally, neural architecture search (NAS) \u2014 where the architecture of the model is itself learned \u2014 requires solving a combinatorial, non-convex, and stochastic problem that blends learning with optimization. Reinforcement learning algorithms also depend heavily on stochasticity to explore state spaces and improve policies over time.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>2.2\u00a0 Distributed Systems<\/h4>\n<p>Modern distributed computing environments \u2014 from federated learning on edge devices to massive cloud clusters \u2014 face dynamic conditions that make optimization difficult. In federated learning, each client device has its own data distribution, leading to non-identical local losses. The global model must minimize a weighted sum of these heterogeneous objectives:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>\\[<br \/>\\min_{w} \\sum_{i=1}^N p_i \\mathcal{L}_i(w)<br \/>\\]<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>where \\(p_i\\) reflects the client importance or data volume. In cloud systems, tasks must be scheduled to optimize for latency, cost, and resource utilization\u2014often with uncertain workloads and shifting resource availability. These conditions naturally introduce both non-convex costs and stochastic inputs.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>2.3\u00a0 Energy and Sustainability<\/h4>\n<p>Power systems are increasingly complex, integrating intermittent sources like wind and solar. Optimization in this domain often involves non-convex unit commitment problems and stochastic forecasts of supply and demand. Operators must ensure balance and stability while minimizing carbon emissions and cost.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>2.4\u00a0 Cloud Cost Management<\/h4>\n<p>Cloud computing introduces varied pricing models:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>On-demand: linear cost<\/li>\n<li>Reserved instances: fixed rate for long-term commitment<\/li>\n<li>Spot pricing: fluctuates with supply\/demand<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>The total cost landscape is non-smooth and time-varying. AI-driven FinOps systems optimize over stochastic forecasts of future demand, taking advantage of market conditions while ensuring reliability.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>2.5\u00a0 Supply Chains &amp; Logistics<\/h4>\n<p>Modern supply chain and logistics networks operate in highly dynamic environments shaped by real-time disruptions, demand variability, uncertain lead times, and complex geopolitical constraints. Traditional optimization approaches \u2014 such as shortest-path algorithms or deterministic linear programs \u2014 fall short when cost functions are non-linear, information is incomplete, or actions must be adapted sequentially over time.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>Let us consider a canonical example: motion planning for autonomous logistics systems, such as drones, autonomous delivery vehicles, or robotic warehouse agents. These systems must determine a trajectory \\( \\{x_t\\}_{t=1}^T \\) over a planning horizon \\( T \\), where each \\( x_t \\in \\mathbb{R}^d \\) represents the system\u2019s state (e.g., location, velocity) at time step \\( t \\). A general trajectory optimization problem can be formulated as:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>Where:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>\\( c(x_t, x_{t+1}) \\) is a cost function capturing travel time, energy consumption, or risk exposure between consecutive states,<\/li>\n<li>\\( \\mathcal{F}_t \\subset \\mathbb{R}^d \\) denotes the time-varying feasible region, accounting for traffic conditions, no-fly zones, terrain restrictions, or regulatory limits,<\/li>\n<li>The constraints may also include dynamic collision avoidance and resource usage limits (e.g., battery levels).<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>This formulation is inherently non-convex, due to:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>Non-linear dynamics or kinematic constraints (e.g., vehicle turning radii, acceleration limits),<\/li>\n<li>Piecewise or discontinuous cost structures (e.g., congestion pricing or toll thresholds),<\/li>\n<li>Obstacle avoidance modeled via non-convex spatial exclusions.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>Due to the presence of uncertainty, factors such as stochastic travel times, real-time weather updates, or unexpected demand spikes lead to:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>Stochastic planning formulations, where travel cost or availability is scenario dependent,<\/li>\n<li>Online or receding-horizon control, continuously updating plans based on live sensor and market data.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>In practice, solution methods include:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>Sampling-based motion planners (e.g., RRT*, PRM) for high-dimensional feasibility search,<\/li>\n<li>Mixed-integer nonlinear programming (MINLP) for incorporating discrete control logic (e.g., path segment selection),<\/li>\n<li>Reinforcement learning to learn adaptive policies under uncertainty,<\/li>\n<li>Heuristic or metaheuristic search (e.g., simulated annealing, genetic algorithms) for real-time route generation.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>As logistics infrastructures scale and autonomy increases, non-convex and stochastic optimization frameworks become indispensable for enabling resilient, efficient, and real-time decision-making across global supply chains.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>2.6\u00a0 Modern Optimization Methods<\/h4>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>To tackle such challenges, hybrid and heuristic approaches are common:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>Metaheuristics like genetic algorithms and particle swarm optimization explore rugged\u00a0solution spaces.<\/li>\n<li>Reinforcement learning models problems as sequential decisions under uncertainty.<\/li>\n<li>Neural combinatorial optimization learns to solve optimization problems using\u00a0neural networks.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>These methods bypass assumptions of convexity or determinism, enabling robust optimization under real-world complexity.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n\n<div class=\"gbc-title-banner tac tac-lg tac-xl\" style='color: #ffffffff; border-radius: 20px; '>\n    <div class=\"container\" style='border-radius: 20px; '>\n        <div class=\"row justify-content--center\" style='color: #ffffffff;'>\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-2 \" style=\"border-radius: 20px;\">\n                                <div class=\"col-12 col-md-11 col-lg-8 col-xl-6  banner-content\"  >\n    \n                    \n                    <div class=\"mt-1 mb-1 content\"><h4 style=\"text-align: center;\"><span style=\"color: #ff7d21;\">Enjoying the blog? Get insights direct to your inbox.<\/span><\/h4>\n<\/div>\n\n                    \n                    <a href='https:\/\/nag.com\/optimization-machine-learning-insights\/' style='background-color: #ff7d21ff; border-radius: 30px; font-weight: 600; ' class='btn mr-1  ' >Insights Sign-Up <i class='fas fa-angle-right'><\/i><\/a>                <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h3>3\u00a0 Case Study \u2014 Modern Portfolio Optimization<\/h3>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>3.1\u00a0 Definitions and Setup<\/h4>\n<p>Let:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>\\( \\mathbf{w} \\in \\mathbb{R}^n \\) be the portfolio weight vector, where each \\( w_i \\) is the fraction of capital allocated to asset \\( i \\)<\/li>\n<li>\\( R_s \\in \\mathbb{R}^n \\) be the vector of asset returns under scenario \\( s \\)<\/li>\n<li>\\( p_s \\in [0,1] \\) be the probability associated with scenario \\( s \\), where \\( \\sum_s p_s = 1 \\)<\/li>\n<li>\\( L(\\mathbf{w}, s) \\) be the portfolio loss under scenario \\( s \\), defined as the negative return<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>3.2\u00a0 Limitations of Classical Models<\/h4>\n<p>Traditional portfolio optimization uses the mean-variance framework:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>\\[<br \/>\\min_{\\mathbf{w}} \\mathbf{w}^\\top \\Sigma \\mathbf{w} \\quad \\text{subject to } \\mathbf{w}^\\top \\mu \\geq R, \\quad \\mathbf{w}^\\top \\mathbf{1} = 1, \\quad \\mathbf{w} \\geq 0<br \/>\\]<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>where:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>\\( \\mu \\in \\mathbb{R}^n \\) is the vector of expected returns<\/li>\n<li>\\item \\( \\Sigma \\in \\mathbb{R}^{n \\times n} \\) is the covariance matrix of returns<\/li>\n<li>\\item \\( R \\) is the required minimum portfolio return<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>This formulation assumes normally distributed returns, convexity, and linear constraints \u2014 rarely satisfied in real markets, where returns usually follow leptokurtic and asymmetric distributions.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>3.3\u00a0 Tail Risk and CVaR Optimization<\/h4>\n<p>To handle asymmetric, heavy-tailed risks, we define:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>\\[<br \/>L(\\mathbf{w}, s) = -\\mathbf{w}^\\top R_s<br \/>\\]<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>and minimize the Conditional Value-at-Risk (CVaR) at level \\( \\alpha \\in (0,1) \\):<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>\\[<br \/>\\min_{\\mathbf{w}, \\nu, \\xi_s} \\quad \\nu + \\frac{1}{1 &#8211; \\alpha} \\sum_s p_s \\xi_s<br \/>\\]<br \/>\\[<br \/>\\text{subject to } \\xi_s \\geq L(\\mathbf{w}, s) &#8211; \\nu, \\quad \\xi_s \\geq 0 \\quad \\forall s<br \/>\\]<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>where:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li>\\( \\nu \\in \\mathbb{R} \\) is an auxiliary variable representing the Value-at-Risk (VaR), ie the quantile<\/li>\n<li>\\( \\xi_s \\in \\mathbb{R}_{\\geq 0} \\) captures the excess loss beyond \\( \\nu \\) in each scenario<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>3.4\u00a0 Non-Convex Constraints and Market Realism<\/h4>\n<p>The real-world feasible region includes:<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <ul>\n<li><strong>Transaction costs<\/strong>: Modeled as piecewise linear or nonlinear functions based on trade volume<\/li>\n<li><strong>Liquidity constraints<\/strong>: \\( w_i \\leq \\text{max tradable volume}_i \\)<\/li>\n<li><strong>Regulatory\/ESG<\/strong>: Sector exposure bounds, carbon scores, or exclusion zones<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-1 pt-lg-1 pt-xl-1\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>These constraints often introduce non-convexities into the problem.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>3.5\u00a0 Solvers, Algorithms, and Deployment<\/h4>\n<p>Practical optimization pipelines often blend global and local strategies \u2014 genetic algorithms, simulated annealing, reinforcement learning, and hybrid methods \u2014 to navigate complex, nonconvex landscapes. Financial institutions operationalize these solutions using cloud compute for parallelism, GPUs for Monte Carlo acceleration, real-time data feeds for adaptive reoptimization, and rigorous stress testing for robustness.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h3>4\u00a0 Conclusion<\/h3>\n<p>Optimization today mirrors the complexity of the systems it governs. Non-convexity allows us to model the nonlinear, constraint-laden, and irregular nature of real systems, while stochastic methods embrace randomness as intrinsic to decision-making. Together, they form a unified framework for adaptive, scalable, and robust operations across AI, finance, energy, and logistics. In a world increasingly shaped by uncertainty and scale, these tools are no longer optional \u2014 they are foundational.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n\n<div class=\"gbc-title-banner tac tac-lg tac-xl\" style='color: #082d48ff; border-radius: 20px; '>\n    <div class=\"container\" style='border-radius: 20px; '>\n        <div class=\"row justify-content--center\" style='color: #082d48ff;'>\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-2 \" style=\"border-radius: 20px;\">\n                                <div class=\"col-12 col-md-11 col-lg-8 col-xl-6  banner-content\"  >\n    \n                    \n                    <div class=\"mt-1 mb-1 content\"><h4><span style=\"color: #000000;\">More optimization learning with insights straight to your inbox.\u00a0<\/span><span style=\"color: #000000;\"><a style=\"color: #000000;\" href=\"https:\/\/nag.com\/optimization-machine-learning-insights\/\">Sign-up here<\/a>.<\/span><\/h4>\n<\/div>\n\n                    \n                    <a href='https:\/\/nag.com\/optimization-machine-learning-insights\/' style='background-color: #ff7d21ff; color: #252a2fff; border-radius: 30px; font-weight: 600; ' class='btn mr-1  ' >Insights Sign-Up <i class='fas fa-angle-right'><\/i><\/a><a href='https:\/\/nag.com\/insights\/balancing-competing-objectives-in-multi-objective-optimization\/' style='border: 2px solid #082d48ff; border-radius: 30px; font-weight: 600; ' class='btn mr-1 outline ' >Opt &#038; ML Blog <i class='fas fa-angle-right'><\/i><\/a>                <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<!-- Spacer -->\n<div class=\"pt-2 pt-lg-2 pt-xl-2\" ><\/div>\n\n<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <h4>References<\/h4>\n<p><strong>Shapiro, A., Dentcheva, D., &amp; Ruszczynski, A. (2014).\u00a0<\/strong><em>Lectures on Stochastic Programming: Modeling and Theory (2nd ed.)<\/em>. Society for Industrial\u00a0and Applied Mathematics (SIAM). A comprehensive reference on stochastic programming, including CVaR, scenario modeling, and theoretical underpinnings.<\/p>\n<p><strong>Boyd, S., &amp; Vandenberghe, L. (2004).\u00a0<\/strong><em>Convex Optimization.<\/em> Cambridge University Press. The foundational text on convex optimization, often cited to highlight where convex assumptions break down.<\/p>\n        <\/div>\n    <\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>Optimization isn\u2019t just convex anymore. Non-convex &#038; stochastic methods now drive AI, logistics, finance, energy &#038; more\u2014adapting to noise, uncertainty &#038; complexity in real-time. They&#8217;re not niche\u2014they\u2019re foundational.<\/p>\n","protected":false},"author":16,"featured_media":64045,"parent":0,"menu_order":0,"template":"","meta":{"content-type":"","footnotes":""},"post-tag":[28,45,29,30,21],"class_list":["post-64044","insights","type-insights","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward - nAG<\/title>\n<meta name=\"description\" content=\"Optimization isn\u2019t just convex anymore. Non-convex &amp; stochastic methods now drive AI, logistics, finance, energy &amp; more\u2014adapting to noise, uncertainty &amp; complexity in real-time. They&#039;re not niche\u2014they\u2019re foundational.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward - nAG\" \/>\n<meta property=\"og:description\" content=\"Optimization isn\u2019t just convex anymore. Non-convex &amp; stochastic methods now drive AI, logistics, finance, energy &amp; more\u2014adapting to noise, uncertainty &amp; complexity in real-time. They&#039;re not niche\u2014they\u2019re foundational.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/\" \/>\n<meta property=\"og:site_name\" content=\"nAG\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-12T10:37:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"667\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@NAGTalk\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/\",\"url\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/\",\"name\":\"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward - nAG\",\"isPartOf\":{\"@id\":\"https:\/\/nag.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg\",\"datePublished\":\"2025-06-12T09:23:58+00:00\",\"dateModified\":\"2025-06-12T10:37:41+00:00\",\"description\":\"Optimization isn\u2019t just convex anymore. Non-convex & stochastic methods now drive AI, logistics, finance, energy & more\u2014adapting to noise, uncertainty & complexity in real-time. They're not niche\u2014they\u2019re foundational.\",\"breadcrumb\":{\"@id\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#primaryimage\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg\",\"width\":1000,\"height\":667},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/nag.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Insights\",\"item\":\"https:\/\/nag.com\/insights\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/nag.com\/#website\",\"url\":\"https:\/\/nag.com\/\",\"name\":\"NAG\",\"description\":\"Robust, trusted numerical software and computational expertise.\",\"publisher\":{\"@id\":\"https:\/\/nag.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/nag.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/nag.com\/#organization\",\"name\":\"Numerical Algorithms Group\",\"alternateName\":\"NAG\",\"url\":\"https:\/\/nag.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"width\":1244,\"height\":397,\"caption\":\"Numerical Algorithms Group\"},\"image\":{\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/NAGTalk\",\"https:\/\/www.linkedin.com\/company\/nag\/\",\"https:\/\/www.youtube.com\/user\/NumericalAlgorithms\",\"https:\/\/github.com\/numericalalgorithmsgroup\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward - nAG","description":"Optimization isn\u2019t just convex anymore. Non-convex & stochastic methods now drive AI, logistics, finance, energy & more\u2014adapting to noise, uncertainty & complexity in real-time. They're not niche\u2014they\u2019re foundational.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/","og_locale":"en_US","og_type":"article","og_title":"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward - nAG","og_description":"Optimization isn\u2019t just convex anymore. Non-convex & stochastic methods now drive AI, logistics, finance, energy & more\u2014adapting to noise, uncertainty & complexity in real-time. They're not niche\u2014they\u2019re foundational.","og_url":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/","og_site_name":"nAG","article_modified_time":"2025-06-12T10:37:41+00:00","og_image":[{"width":1000,"height":667,"url":"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@NAGTalk","twitter_misc":{"Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/","url":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/","name":"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward - nAG","isPartOf":{"@id":"https:\/\/nag.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#primaryimage"},"image":{"@id":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#primaryimage"},"thumbnailUrl":"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg","datePublished":"2025-06-12T09:23:58+00:00","dateModified":"2025-06-12T10:37:41+00:00","description":"Optimization isn\u2019t just convex anymore. Non-convex & stochastic methods now drive AI, logistics, finance, energy & more\u2014adapting to noise, uncertainty & complexity in real-time. They're not niche\u2014they\u2019re foundational.","breadcrumb":{"@id":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#primaryimage","url":"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2025\/06\/iStock-1187037134-web.jpg","width":1000,"height":667},{"@type":"BreadcrumbList","@id":"https:\/\/nag.com\/insights\/non-convex-optimization-and-machine-learning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nag.com\/"},{"@type":"ListItem","position":2,"name":"Insights","item":"https:\/\/nag.com\/insights\/"},{"@type":"ListItem","position":3,"name":"AI\u2019s Hidden Workhorse: How Non-Convex Optimization Drives Machine Learning Forward"}]},{"@type":"WebSite","@id":"https:\/\/nag.com\/#website","url":"https:\/\/nag.com\/","name":"NAG","description":"Robust, trusted numerical software and computational expertise.","publisher":{"@id":"https:\/\/nag.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nag.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/nag.com\/#organization","name":"Numerical Algorithms Group","alternateName":"NAG","url":"https:\/\/nag.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","width":1244,"height":397,"caption":"Numerical Algorithms Group"},"image":{"@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/NAGTalk","https:\/\/www.linkedin.com\/company\/nag\/","https:\/\/www.youtube.com\/user\/NumericalAlgorithms","https:\/\/github.com\/numericalalgorithmsgroup"]}]}},"_links":{"self":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/64044","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights"}],"about":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/types\/insights"}],"author":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/users\/16"}],"version-history":[{"count":65,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/64044\/revisions"}],"predecessor-version":[{"id":64577,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/64044\/revisions\/64577"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media\/64045"}],"wp:attachment":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media?parent=64044"}],"wp:term":[{"taxonomy":"post-tag","embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/post-tag?post=64044"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}