Online updating regularized kernel
However, similar bounds cannot be obtained for the FTL algorithm for other important families of models like online linear optimization.
To do so, one modifies FTL by adding regularisation.
This technique is suitable for real-world applications where the number of classes is often unknown and online learning from real-time data is required.
In online convex optimisation (OCO), the hypothesis set and the loss functions are forced to be convex to obtain stronger learning bounds.
Whenever a new class (non-native to the knowledge learnt thus far) is encountered, the classifier gets remodeled automatically and the parameters are calculated in such a way that it retains the knowledge learnt thus far.Mini-batch techniques are used with repeated passing over the training data to obtain optimized out-of-core versions of machine learning algorithms, for e.g. When combined with backpropagation, this is currently the de facto training method for training artificial neural networks.The simple example of linear least squares is used to explain a variety of ideas in online learning.The corresponding procedure will no longer be truly online and instead involve storing all the data points, but is still faster than the brute force method.This discussion is restricted to the case of the square loss, though it can be extended to any convex loss.