How is a hyperparameter different from other parameters in machine learning?

A hyperparameter is characterized by being set before the learning process begins and is external to the model. This definition distinguishes hyperparameters from other parameters that are typically learned or adjusted during the training process itself.

In machine learning, parameters usually refer to the internal configurations of the model that are optimized as the training data is processed (such as the weights in a neural network). These parameters are adjusted based on the input data and the corresponding output during the training phase. On the other hand, hyperparameters, such as learning rate, number of hidden layers, or batch size, are established prior to training and dictate how the training process will proceed.

Setting hyperparameters correctly is crucial as they can significantly influence the model's performance. Unlike parameters, which are updated automatically by learning algorithms, hyperparameters require manual tuning and validation based on various strategies, such as cross-validation, to find the combination that yields the best performance on unseen data. This aspect of hyperparameters underlines their role as external configurations rather than metrics derived from data during the learning phase.