Play deep learning with CIFAR datasets

A learning rate range test implementation in PyTorch

Visualize Tensorflow's optimizers.

An easy neural network for Java!

Videos of deep learning optimizers moving on 3D problem-landscapes

Improving MMD-GAN training with repulsive loss function

Cyclic learning rate TensorFlow implementation.

FIR filter implementation with LMS in C++ & JAVA (JNI) & Python (swig)

Learning rate multiplier

One cycle policy learning rate scheduler in PyTorch

Automatic and Simultaneous Adjustment of Learning Rate and Momentum for Stochastic Gradient Descent

Stochastic Weight Averaging - TensorFlow implementation

How optimizer and learning rate choice affects training performance

Effective trash classification model using only a small number of annotated images

Improved Hypergradient optimizers, providing better generalization and faster convergence.

Implementation of learning rate finder in TensorFlow

Q-Learing algorithm solves simple mazes.

Pytorch implementation of arbitrary learning rate and momentum schedules, including the One Cycle Policy

TensorFlow/Keras implementation of the paper: 'Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates'

Residual Network Experiments with CIFAR Datasets.

Tensorflow-Keras callback implementing arXiv 1712.07628

A Jupyter notebook exploring sophisticated learning rate strategies for training deep neural networks

Detect happy dogs in real-time

Add-on functionality for the R implementation of Keras

As the learning rate is one of the most important hyper-parameters to tune for training convolutional neural networks. In this paper, a powerful technique to select a range of learning rates for a neural network that named cyclical learning rate was implemented with two different skewness degrees. It is an approach to adjust where the value is cycled between a lower bound and upper bound. CLR policies are computationally simpler and can avoid the computational expense of fine tuning with fixed learning rate. It is clearly shown that changing the learning rate during the training phase provides by far better results than fixed values with similar or even smaller number of epochs.

We built an optimization technique that, at each learning step, automatically learns which best learning rate to use for gradient descent.

Pytorch implementation of the paper: 'Cyclical Learning Rates for Training Neural Networks'

Benchmarking various Computer Vision models on TinyImageNet Dataset

Deep_Learning: Stochastic Gradient Noise heavy tail distribution Analysis