Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.world is one of the many independent Mastodon servers you can use to participate in the fediverse.
Generic Mastodon server for anyone to use.

Administered by:

Server stats:

8.1K
active users

mastodon.world: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.0

#CSNA

3 posts2 participants0 posts today
FunctionalProgramming


#math.OC #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgDistributionally Robust Shape and Topology OptimizationThis article aims to introduce the paradigm of distributional robustness from the field of convex optimization to tackle optimal design problems under uncertainty. We consider realistic situations where the physical model, and thereby the cost function of the design to be minimized depend on uncertain parameters. The probability distribution of the latter is itself known imperfectly, through a nominal law, reconstructed from a few observed samples. The distributionally robust optimal design problem is an intricate bilevel program which consists in minimizing the worst value of a statistical quantity of the cost function (typically, its expectation) when the law of the uncertain parameters belongs to a certain ``ambiguity set''. We address three classes of such problems: firstly, this ambiguity set is made of the probability laws whose Wasserstein distance to the nominal law is less than a given threshold; secondly, the ambiguity set is based on the first- and second-order moments of the actual and nominal probability laws. Eventually, a statistical quantity of the cost other than its expectation is made robust with respect to the law of the parameters, namely its conditional value at risk. Using techniques from convex duality, we derive tractable, single-level reformulations of these problems, framed over augmented sets of variables. Our methods are essentially agnostic of the optimal design framework; they are described in a unifying abstract framework, before being applied to multiple situations in density-based topology optimization and in geometric shape optimization. Several numerical examples are discussed in two and three space dimensions to appraise the features of the proposed techniques.
#mathoc#csna#mathna
FunctionalProgramming


#math.NA #cs.NA

Origin | Interest | Match
arXiv logo
arXiv.orgEfficient and stable diffusion generated methods for ground state computation in Bose--Einstein condensatesThis paper investigates numerical methods for approximating the ground state of Bose--Einstein condensates (BECs) by introducing two relaxed formulations of the Gross--Pitaevskii energy functional. These formulations achieve first- and second-order accuracy with respect to the relaxation parameter \( τ\), and are shown to converge to the original energy functional as \( τ\to 0 \). A key feature of the relaxed functionals is their concavity, which ensures that local minima lie on the boundary of the concave hull. This property prevents energy increases during constraint normalization and enables the development of energy-dissipative algorithms. Numerical methods based on sequential linear programming are proposed, accompanied by rigorous analysis of their stability with respect to the relaxed energy. To enhance computational efficiency, an adaptive strategy is introduced, dynamically refining solutions obtained with larger relaxation parameters to achieve higher accuracy with smaller ones. Numerical experiments demonstrate the stability, convergence, and energy dissipation of the proposed methods, while showcasing the adaptive strategy's effectiveness in improving computational performance.
#mathna#csna
FunctionalProgramming


#math.NA #cs.NA #math.OC

Origin | Interest | Match
arXiv logo
arXiv.orgInterpolation-Based Gradient-Error Bounds for Use in Derivative-Free Optimization of Noisy FunctionsIn this paper, we analyze the accuracy of gradient estimates obtained by linear interpolation when the underlying function is subject to bounded measurement noise. The total gradient error is decomposed into a deterministic component arising from the interpolation (finite-difference) approximation, and a stochastic component due to noise. Various upper bounds for both error components are derived and compared through several illustrative examples. Our comparative study reveals that strict deterministic bounds, including those commonly used in derivative-free optimization (DFO), tend to be overly conservative. To address this, we propose approximate gradient error bounds that aim to upper bound the gradient error norm more realistically, without the excessive conservatism of classical bounds. Finally, drawing inspiration from dual real-time optimization strategies, we present a DFO scheme based on sequential programming, where the approximate gradient error bounds are enforced as constraints within the optimization problem.
#mathna#csna#mathoc
FunctionalProgramming


#math.NA #cs.AI #cs.NA #math.AP

Origin | Interest | Match
arXiv logo
arXiv.orgSolving nonconvex Hamilton--Jacobi--Isaacs equations with PINN-based policy iterationWe propose a mesh-free policy iteration framework that combines classical dynamic programming with physics-informed neural networks (PINNs) to solve high-dimensional, nonconvex Hamilton--Jacobi--Isaacs (HJI) equations arising in stochastic differential games and robust control. The method alternates between solving linear second-order PDEs under fixed feedback policies and updating the controls via pointwise minimax optimization using automatic differentiation. Under standard Lipschitz and uniform ellipticity assumptions, we prove that the value function iterates converge locally uniformly to the unique viscosity solution of the HJI equation. The analysis establishes equi-Lipschitz regularity of the iterates, enabling provable stability and convergence without requiring convexity of the Hamiltonian. Numerical experiments demonstrate the accuracy and scalability of the method. In a two-dimensional stochastic path-planning game with a moving obstacle, our method matches finite-difference benchmarks with relative $L^2$-errors below %10^{-2}%. In five- and ten-dimensional publisher-subscriber differential games with anisotropic noise, the proposed approach consistently outperforms direct PINN solvers, yielding smoother value functions and lower residuals. Our results suggest that integrating PINNs with policy iteration is a practical and theoretically grounded method for solving high-dimensional, nonconvex HJI equations, with potential applications in robotics, finance, and multi-agent reinforcement learning.
#mathna#csai#csna
FunctionalProgramming


#math.NA #cs.NA #math.OC

Origin | Interest | Match
arXiv logo
arXiv.orgPolynomial argmin for recovery and approximation of multivariate discontinuous functionsWe propose to approximate a (possibly discontinuous) multivariate function f (x) on a compact set by the partial minimizer arg miny p(x, y) of an appropriate polynomial p whose construction can be cast in a univariate sum of squares (SOS) framework, resulting in a highly structured convex semidefinite program. In a number of non-trivial cases (e.g. when f is a piecewise polynomial) we prove that the approximation is exact with a low-degree polynomial p. Our approach has three distinguishing features: (i) It is mesh-free and does not require the knowledge of the discontinuity locations. (ii) It is model-free in the sense that we only assume that the function to be approximated is available through samples (point evaluations). (iii) The size of the semidefinite program is independent of the ambient dimension and depends linearly on the number of samples. We also analyze the sample complexity of the approach, proving a generalization error bound in a probabilistic setting. This allows for a comparison with machine learning approaches.
#mathna#csna#mathoc
FunctionalProgramming


#math.NA #cs.AI #cs.NA #math.AP

Origin | Interest | Match
arXiv logo
arXiv.orgSolving nonconvex Hamilton--Jacobi--Isaacs equations with PINN-based policy iterationWe propose a mesh-free policy iteration framework that combines classical dynamic programming with physics-informed neural networks (PINNs) to solve high-dimensional, nonconvex Hamilton--Jacobi--Isaacs (HJI) equations arising in stochastic differential games and robust control. The method alternates between solving linear second-order PDEs under fixed feedback policies and updating the controls via pointwise minimax optimization using automatic differentiation. Under standard Lipschitz and uniform ellipticity assumptions, we prove that the value function iterates converge locally uniformly to the unique viscosity solution of the HJI equation. The analysis establishes equi-Lipschitz regularity of the iterates, enabling provable stability and convergence without requiring convexity of the Hamiltonian. Numerical experiments demonstrate the accuracy and scalability of the method. In a two-dimensional stochastic path-planning game with a moving obstacle, our method matches finite-difference benchmarks with relative $L^2$-errors below %10^{-2}%. In five- and ten-dimensional publisher-subscriber differential games with anisotropic noise, the proposed approach consistently outperforms direct PINN solvers, yielding smoother value functions and lower residuals. Our results suggest that integrating PINNs with policy iteration is a practical and theoretically grounded method for solving high-dimensional, nonconvex HJI equations, with potential applications in robotics, finance, and multi-agent reinforcement learning.
#mathna#csai#csna
FunctionalProgramming


#math.NA #cs.NA #math.AP

Origin | Interest | Match
arXiv logo
arXiv.orgInformation geometric regularization of the barotropic Euler equationShock waves in gas dynamics feature jump discontinuities that hinder numerical simulations. Viscous regularizations are prone to excessive dissipation of fine-scale structures. In this work, we propose the first inviscid regularization of the multidimensional Euler equation based on ideas from semidefinite programming, information geometry, geometric hydrodynamics, and nonlinear elasticity. The Lagrangian flow maps of Euler solutions are a dynamical system on the manifold of diffeomorphisms. We observe that shock formation arises from the manifold's geodesic incompleteness. Our regularization embeds it into an ambient space equipped with the information geometry of the logarithmic barrier function. Thus, the diffeomorphism manifold inherits a geodesically complete geometry. The resulting regularized conservation law replaces shocks with smooth profiles without affecting oscillatory structures. One and two-dimensional numerical experiments show its practical potential to enable higher-order methods without explicit shock capturing. While we focus on the barotropic Euler equations for concreteness and simplicity of exposition, our regularization easily extends to more general Euler and Navier-Stokes-type equations. Our approach regularizes the Wasserstein geometry of the mass density with its information geometry. The former captures the natural trajectories of physical particles and the latter that of statistical estimators. Information geometric regularization accounts for the mass density's dual nature as a statistical/computational tool summarizing the motion of physical particles. Thus, our work is a starting point for information geometric mechanics that views solutions of continuum mechanical PDEs as parameters of statistical models for unresolved scales and uses their information geometry to evolve them in time.
#mathna#csna#mathap
FunctionalProgramming


#math.NA #cs.LG #cs.NA #quant-ph

Origin | Interest | Match
arXiv logo
arXiv.orgCons-training Tensor Networks: Embedding and Optimization Over Discrete Linear ConstraintsIn this study, we introduce a novel family of tensor networks, termed constrained matrix product states (MPS), designed to incorporate exactly arbitrary discrete linear constraints, including inequalities, into sparse block structures. These tensor networks are particularly tailored for modeling distributions with support strictly over the feasible space, offering benefits such as reducing the search space in optimization problems, alleviating overfitting, improving training efficiency, and decreasing model size. Central to our approach is the concept of a quantum region, an extension of quantum numbers traditionally used in U(1) symmetric tensor networks, adapted to capture any linear constraint, including the unconstrained scenario. We further develop a novel canonical form for these new MPS, which allow for the merging and factorization of tensor blocks according to quantum region fusion rules and permit optimal truncation schemes. Utilizing this canonical form, we apply an unsupervised training strategy to optimize arbitrary objective functions subject to discrete linear constraints. Our method's efficacy is demonstrated by solving the quadratic knapsack problem, achieving superior performance compared to a leading nonlinear integer programming solver. Additionally, we analyze the complexity and scalability of our approach, demonstrating its potential in addressing complex constrained combinatorial optimization problems.
#mathna#cslg#csna
FunctionalProgramming


#math.NA #cs.LG #cs.NA #quant-ph

Origin | Interest | Match
arXiv logo
arXiv.orgCons-training Tensor Networks: Embedding and Optimization Over Discrete Linear ConstraintsIn this study, we introduce a novel family of tensor networks, termed constrained matrix product states (MPS), designed to incorporate exactly arbitrary discrete linear constraints, including inequalities, into sparse block structures. These tensor networks are particularly tailored for modeling distributions with support strictly over the feasible space, offering benefits such as reducing the search space in optimization problems, alleviating overfitting, improving training efficiency, and decreasing model size. Central to our approach is the concept of a quantum region, an extension of quantum numbers traditionally used in U(1) symmetric tensor networks, adapted to capture any linear constraint, including the unconstrained scenario. We further develop a novel canonical form for these new MPS, which allow for the merging and factorization of tensor blocks according to quantum region fusion rules and permit optimal truncation schemes. Utilizing this canonical form, we apply an unsupervised training strategy to optimize arbitrary objective functions subject to discrete linear constraints. Our method's efficacy is demonstrated by solving the quadratic knapsack problem, achieving superior performance compared to a leading nonlinear integer programming solver. Additionally, we analyze the complexity and scalability of our approach, demonstrating its potential in addressing complex constrained combinatorial optimization problems.
#mathna#cslg#csna
FunctionalProgramming


#math.NA #cs.AI #cs.DC #cs.MS #cs.NA #math.OC

Origin | Interest | Match
arXiv logo
arXiv.orgGPU-based complete search for nonlinear minimization subject to boundsThis paper introduces a GPU-based complete search method to enclose the global minimum of a nonlinear function subject to simple bounds on the variables. Using interval analysis, coupled with the computational power and architecture of GPU, the method iteratively rules out the regions in the search domain where the global minimum cannot exist and leaves a finite set of regions where the global minimum must exist. For effectiveness, because of the rigor of interval analysis, the method is guaranteed to enclose the global minimum of the nonlinear function even in the presence of rounding errors. For efficiency, the method employs a novel GPU-based single program, single data parallel programming style to circumvent major GPU performance bottlenecks, and a variable cycling technique is also integrated into the method to reduce computational cost when minimizing large-scale nonlinear functions. The method is validated by minimizing 10 multimodal benchmark test functions with scalable dimensions, including the well-known Ackley function, Griewank function, Levy function, and Rastrigin function. These benchmark test functions represent grand challenges of global optimization, and enclosing the guaranteed global minimum of these benchmark test functions with more than 80 dimensions has not been reported in the literature. Our method completely searches the feasible domain and successfully encloses the guaranteed global minimum of these 10 benchmark test functions with up to 10,000 dimensions using only one GPU in a reasonable computation time, far exceeding the reported results in the literature due to the unique method design and implementation based on GPU architecture.
#mathna#csai#csdc
FunctionalProgramming


#math.OC #cs.NA #cs.SY #eess.SY #math.AG #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgGeneralized EllipsoidsWe introduce a family of symmetric convex bodies called generalized ellipsoids of degree $d$ (GE-$d$s), with ellipsoids corresponding to the case of $d=0$. Generalized ellipsoids (GEs) retain many geometric, algebraic, and algorithmic properties of ellipsoids. We show that the conditions that the parameters of a GE must satisfy can be checked in strongly polynomial time, and that one can search for GEs of a given degree by solving a semidefinite program whose size grows only linearly with dimension. We give an example of a GE which does not have a second-order cone representation, but show that every GE has a semidefinite representation whose size depends linearly on both its dimension and degree. In terms of expressiveness, we prove that for any integer $m\geq 2$, every symmetric full-dimensional polytope with $2m$ facets and every intersection of $m$ co-centered ellipsoids can be represented exactly as a GE-$d$ with $d \leq 2m-3$. Using this result, we show that every symmetric convex body can be approximated arbitrarily well by a GE-$d$ and we quantify the quality of the approximation as a function of the degree $d$. Finally, we present applications of GEs to several areas, such as time-varying portfolio optimization, stability analysis of switched linear systems, robust-to-dynamics optimization, and robust polynomial regression.
#mathoc#csna#cssy
LLMs


#cs.AI #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgAI Assistants to Enhance and Exploit the PETSc Knowledge BaseGenerative AI, especially through large language models (LLMs), is transforming how technical knowledge can be accessed, reused, and extended. PETSc, a widely used numerical library for high-performance scientific computing, has accumulated a rich but fragmented knowledge base over its three decades of development, spanning source code, documentation, mailing lists, GitLab issues, Discord conversations, technical papers, and more. Much of this knowledge remains informal and inaccessible to users and new developers. To activate and utilize this knowledge base more effectively, the PETSc team has begun building an LLM-powered system that combines PETSc content with custom LLM tools -- including retrieval-augmented generation (RAG), reranking algorithms, and chatbots -- to assist users, support developers, and propose updates to formal documentation. This paper presents initial experiences designing and evaluating these tools, focusing on system architecture, using RAG and reranking for PETSc-specific information, evaluation methodologies for various LLMs and embedding models, and user interface design. Leveraging the Argonne Leadership Computing Facility resources, we analyze how LLM responses can enhance the development and use of numerical software, with an initial focus on scalable Krylov solvers. Our goal is to establish an extensible framework for knowledge-centered AI in scientific software, enabling scalable support, enriched documentation, and enhanced workflows for research and development. We conclude by outlining directions for expanding this system into a robust, evolving platform that advances software ecosystems to accelerate scientific discovery.
#csai#csna#mathna
FunctionalProgramming


#cs.LG #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgOn the algorithmic construction of deep ReLU networksIt is difficult to describe in mathematical terms what a neural network trained on data represents. On the other hand, there is a growing mathematical understanding of what neural networks are in principle capable of representing. Feedforward neural networks using the ReLU activation function represent continuous and piecewise linear functions and can approximate many others. The study of their expressivity addresses the question: which ones? Contributing to the available answers, we take the perspective of a neural network as an algorithm. In this analogy, a neural network is programmed constructively, rather than trained from data. An interesting example is a sorting algorithm: we explicitly construct a neural network that sorts its inputs exactly, not approximately, and that, in a sense, has optimal computational complexity if the input dimension is large. Such constructed networks may have several billion parameters. We construct and analyze several other examples, both existing and new. We find that, in these examples, neural networks as algorithms are typically recursive and parallel. Compared to conventional algorithms, ReLU networks are restricted by having to be continuous. Moreover, the depth of recursion is limited by the depth of the network, with deep networks having superior properties over shallow ones.
#cslg#csna#mathna
SearchEngine


#cs.LO #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgTime-Sensitive Importance SplittingState-of-the-art methods for rare event simulation of non-Markovian models face practical or theoretical limits if observing the event of interest requires prior knowledge or information on the timed behavior of the system. In this paper, we attack both limits by extending importance splitting with a time-sensitive importance function. To this end, we perform backwards reachability search from the target states, considering information about the lower and upper bounds of the active timers in order to steer the generation of paths towards the rare event. We have developed a prototype implementation of the approach for input/output stochastic automata within the Modest Toolset. Preliminary experiments show the potential of the approach in estimating rare event probabilities for an example from reliability engineering.
#cslo#csna#mathna
FunctionalProgramming


#cs.LG #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgOn the algorithmic construction of deep ReLU networksIt is difficult to describe in mathematical terms what a neural network trained on data represents. On the other hand, there is a growing mathematical understanding of what neural networks are in principle capable of representing. Feedforward neural networks using the ReLU activation function represent continuous and piecewise linear functions and can approximate many others. The study of their expressivity addresses the question: which ones? Contributing to the available answers, we take the perspective of a neural network as an algorithm. In this analogy, a neural network is programmed constructively, rather than trained from data. An interesting example is a sorting algorithm: we explicitly construct a neural network that sorts its inputs exactly, not approximately, and that, in a sense, has optimal computational complexity if the input dimension is large. Such constructed networks may have several billion parameters. We construct and analyze several other examples, both existing and new. We find that, in these examples, neural networks as algorithms are typically recursive and parallel. Compared to conventional algorithms, ReLU networks are restricted by having to be continuous. Moreover, the depth of recursion is limited by the depth of the network, with deep networks having superior properties over shallow ones.
#cslg#csna#mathna
deepseek


#cs.CL #cs.CE #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgTL;DR: Too Long, Do Re-weighting for Effcient LLM Reasoning CompressionLarge Language Models (LLMs) have recently achieved remarkable progress by leveraging Reinforcement Learning and extended Chain-of-Thought (CoT) techniques. However, the challenge of performing efficient language reasoning--especially during inference with extremely long outputs--has drawn increasing attention from the research community. In this work, we propose a dynamic ratio-based training pipeline that does not rely on sophisticated data annotations or interpolation between multiple models. We continuously balance the weights between the model's System-1 and System-2 data to eliminate redundant reasoning processes while preserving the model's reasoning capability. We validate our approach across models on DeepSeek-R1-Distill-7B and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying difficulty levels. Our method significantly reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. Our code and data will be available soon.
#cscl#csce#csna
LLMs


#cs.CL #cs.LG #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgTensorSLM: Energy-efficient Embedding Compression of Sub-billion Parameter Language Models on Low-end DevicesSmall Language Models (SLMs, or on-device LMs) have significantly fewer parameters than Large Language Models (LLMs). They are typically deployed on low-end devices, like mobile phones and single-board computers. Unlike LLMs, which rely on increasing model size for better generalisation, SLMs designed for edge applications are expected to have adaptivity to the deployment environments and energy efficiency given the device battery life constraints, which are not addressed in datacenter-deployed LLMs. This paper addresses these two requirements by proposing a training-free token embedding compression approach using Tensor-Train Decomposition (TTD). Each pre-trained token embedding vector is converted into a lower-dimensional Matrix Product State (MPS). We comprehensively evaluate the extracted low-rank structures across compression ratio, language task performance, latency, and energy consumption on a typical low-end device, i.e. Raspberry Pi. Taking the sub-billion parameter versions of GPT-2/Cerebres-GPT and OPT models as examples, our approach achieves a comparable language task performance to the original model with around $2.0\times$ embedding layer compression, while the energy consumption of a single query drops by half.
#cscl#cslg#csna
LLMs


#cs.LG #cs.NA #math.NA #math.ST #stat.TH

Origin | Interest | Match
arXiv logo
arXiv.orgTransformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization InsightsTransformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrated that real-world data and learning tasks exhibit low-dimensional structures, along with some noise or measurement error. The performance of transformers tends to depend on the intrinsic dimension of the data/tasks, though theoretical understandings remain largely unexplored for transformers. This work establishes a theoretical foundation by analyzing the performance of transformers for regression tasks involving noisy input data on a manifold. Specifically, the input data are in a tubular neighborhood of a manifold, while the ground truth function depends on the projection of the noisy data onto the manifold. We prove approximation and generalization errors which crucially depend on the intrinsic dimension of the manifold. Our results demonstrate that transformers can leverage low-complexity structures in learning task even when the input data are perturbed by high-dimensional noise. Our novel proof technique constructs representations of basic arithmetic operations by transformers, which may hold independent interest.
#cslg#csna#mathna
deepseek


#cs.CL #cs.CE #cs.NA #math.NA

Origin | Interest | Match
arXiv logo
arXiv.orgTL;DR: Too Long, Do Re-weighting for Effcient LLM Reasoning CompressionLarge Language Models (LLMs) have recently achieved remarkable progress by leveraging Reinforcement Learning and extended Chain-of-Thought (CoT) techniques. However, the challenge of performing efficient language reasoning--especially during inference with extremely long outputs--has drawn increasing attention from the research community. In this work, we propose a dynamic ratio-based training pipeline that does not rely on sophisticated data annotations or interpolation between multiple models. We continuously balance the weights between the model's System-1 and System-2 data to eliminate redundant reasoning processes while preserving the model's reasoning capability. We validate our approach across models on DeepSeek-R1-Distill-7B and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying difficulty levels. Our method significantly reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. Our code and data will be available soon.
#cscl#csce#csna
LLMs


https://arxiv.org/abs/2505.16932

#cs.LG #cs.AI #cs.CL #cs.NA #math.NA #math.OC

Result Details
arXiv.orgThe Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon AlgorithmComputing the polar decomposition and the related matrix sign function, has been a well-studied problem in numerical analysis for decades. More recently, it has emerged as an important subroutine in deep learning, particularly within the Muon optimization framework. However, the requirements in this setting differ significantly from those of traditional numerical analysis. In deep learning, methods must be highly efficient and GPU-compatible, but high accuracy is often unnecessary. As a result, classical algorithms like Newton-Schulz (which suffers from slow initial convergence) and methods based on rational functions (which rely on QR decompositions or matrix inverses) are poorly suited to this context. In this work, we introduce Polar Express, a GPU-friendly algorithm for computing the polar decomposition. Like classical polynomial methods such as Newton-Schulz, our approach uses only matrix-matrix multiplications, making it GPU-compatible. Motivated by earlier work of Chen & Chow and Nakatsukasa & Freund, Polar Express adapts the polynomial update rule at each iteration by solving a minimax optimization problem, and we prove that it enjoys a strong worst-case optimality guarantee. This property ensures both rapid early convergence and fast asymptotic convergence. We also address finite-precision issues, making it stable in bfloat16 in practice. We apply Polar Express within the Muon optimization framework and show consistent improvements in validation loss on large-scale models such as GPT-2, outperforming recent alternatives across a range of learning rates.
#cslg#csai#cscl
TrendingLive feeds
About