I make a distinction between latent variables and parameters. Latent variables change over time; parameters don’t.

(This is really a quantitative matter, not a qualitative one, as a “parameter” might change slowly (often called adaptation) and thus be promoted to a “variable.”)

This distinction carries over to computation: inferring a variable is just called “inference,” but inferring a parameter is typically called “learning.” Mathematically, these are really the same process, just about different quantities and on different timescales.

Analogously, model bias and inductive bias are both biases, but about variables versus parameters. Model bias is an inference bias in the mapping from input to output. Inductive bias is a learning bias, a bias in what is learned from data (i.e. in the mapping from a data set to parameters). This inductive bias in turn creates its own model bias.

An inductive bias is caused by both the model class (such as a neural network architecture) and the optimization procedure.

The following table compares the relevant concepts.

Table: Comparison of latent variables and parameters, and computations relevant to them. Here x is a vector of latent variables, Q is a parameter vector, d is observable input data, {d} is a data set, and M is a model class.