Which method for hyperparameter optimization employs a population of parameter combinations that evolve over generations?

Genetic algorithms are a method for hyperparameter optimization that simulates the process of natural selection to evolve a population of potential solutions over generations. This approach begins with a diverse population of hyperparameter configurations. Through a series of simulated "breeding" operations—such as selection, crossover, and mutation—these configurations undergo iterative improvement.

In each generation, the best-performing hyperparameter combinations are selected based on a defined fitness function, such as model accuracy. The selected combinations are then modified and combined to create new configurations. This evolutionary process continues until a satisfactory solution is found or a specified number of generations is completed.

The other methods mentioned do not utilize an evolutionary process involving a population of parameter combinations. Bayesian optimization uses probabilistic models to make decisions about where to sample next, focused on balancing exploration and exploitation. Randomized search randomly samples hyperparameter values from a specified distribution, without consideration for evolution or selection. Grid search exhaustively evaluates a predetermined set of hyperparameter combinations, analyzing each individually rather than as part of a population evolving over time.