Defending against saddle point attack in Byzantine-robust distributed learning

D Yin, Y Chen, R Kannan… - … Conference on Machine …, 2019 - proceedings.mlr.press
… that w is a second-order stationary point of F(w) and output w. … Guarantees In this section,
we provide the theoretical result guaranteeing that Algorithm 1 converges to a second-order

Distributed gradient algorithm for constrained optimization with application to load sharing in power systems

P Yi, Y Hong, F Liu - Systems & Control Letters, 2015 - Elsevier
second-order distributed dynamics was proposed to solve an unconstrained optimization in
[11], while a similar algorithm … the connectivity of graph G , which guarantees that any agent’s …

A decentralized second-order method with exact linear convergence rate for consensus optimization

A Mokhtari, W Shi, Q Ling… - IEEE Transactions on …, 2016 - ieeexplore.ieee.org
… makes distributed computation of primal gradients impossible. … pendency of the convergence
constant with the algorithm’s … To guarantee that the second order approximation suggested …

Second-order optimization for non-convex machine learning: An empirical study

P Xu, F Roosta, MW Mahoney - Proceedings of the 2020 SIAM International …, 2020 - SIAM
… While some firstorder algorithms can guarantee … comparisons among first and secondorder
methods, we consider … which involve comparisons among various second-order methods, we …

[HTML][HTML] User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient

AS Dalalyan, A Karagulyan - Stochastic Processes and their Applications, 2019 - Elsevier
guarantees on the sampling error of these second-order LMCs. These guarantees reveal that
the second-order LMC algorithms … providing guarantees for sampling from a distribution in …

Nonnegative matrix factorization with constrained second-order optimization

R Zdunek, A Cichocki - Signal Processing, 2007 - Elsevier
… the second-order terms of a cost function to overcome the disadvantages of gradient (multiplicative)
algorithms. … The convergence of the CG is guaranteed in a finite number of iterations …

Optimal algorithms for non-smooth distributed optimization in networks

K Scaman, F Bach, S Bubeck… - Advances in Neural …, 2018 - proceedings.neurips.cc
… network only impacts a second-order term in O(1/t), where t is … yet efficient algorithm called
distributed randomized smoothing … Our performance guarantee for the MSPD algorithm is then …

Distributed learning with compressed gradient differences

K Mishchenko, E Gorbunov, M Takáč… - arXiv preprint arXiv …, 2019 - arxiv.org
… powerful local solver, such as a second order method), so that … differences, which we
call DIANA (Algorithm 1). Unlike the … , number of iterations sufficient to guarantee that E∥∇f(xk)∥2 …

Fast curvature matrix-vector products for second-order gradient descent

NN Schraudolph - Neural computation, 2002 - ieeexplore.ieee.org
… : assuming independently and identically distributed (iid) gaussian … and does not guarantee
positive definiteness. Practical … of common secondorder gradient algorithms, including the …

Second-order continuous-time algorithms for economic power dispatch in smart grids

X He, DWC Ho, T Huang, J Yu… - IEEE Transactions on …, 2017 - ieeexplore.ieee.org
… Under an appropriate initial condition, it is shown that the second-order distributed algorithms
guarantees to find an optimal solution and also has the advantage of keeping the power …