A collection of scipy multivariate single criteria optimization solver examples are located in this repository.

As for now, there presented following examples:

• Stochastic Newton-Gauss method is implemented A simple $\alpha \cdot cos(\beta\cdot x+\gamma)$ optimization problem considered in the example, but it is capable to optimize any given loss function.
• SGD from scratch example A simple $\alpha \cdot cos(\beta\cdot x+\gamma)$ optimization problem considered in the example, but it is capable to optimize any given loss function.
• Broyden-Fletcher-Goldfarb-Shano with loss function gradient provided.
  Need a fit of A*cos(x+phi)+b into a set of sensor measured data.
• Broyden-Fletcher-Goldfarb-Shano with no loss function gradient provided.
  Same problem considered:
  Need a fit of A*cos(x+phi)+b into a set of sensor measured data.
  The gradient will be estimated with finite differences.
• Nelder-Mead algorithm. No gradient needed.
  This algorithm requires only the definition of function been optimized.
• Trusted region constrained optimization
  The example calculates a control signal for a set of actuators, to
  obtain desired control vector.
• SLSQP constrained optimization
  The same example as in trusted-region: need to find a control
  vector for a set of actuators.
• Lagrange coefficients solution
  The same problem. Much more efficient and shorter analitical solution.
  Also we define an variable penalization matrix, instead of Identity matrix
  of the previous examples.

• A very simple implementation of k-means algorithm A buggy implementation, but it's quite good when to illustration simplicity.

• Moons classification with sklearn kernelized PCA
  A simple toy data classification using sklearn kernelized PCA.
• Kernelized PCA from scratch
  The same dataset as in above. RBF PCA written from scratch.

• Numerical Jacobian computation with finite differences
  A simple straightforward algorithm for Jacobian estimation.
• KDE: kernel density estimation
  How to aproximate an unknown PDF given the data points generated by it.