Martin Pelikan
David E. Goldberg
Fernando G. Lobo
Skip Abstract Section
Abstract
This paper summarizes the research on population-based probabilistic search algorithms based on modeling promising solutions by estimating their probability distribution and using the constructed model to guide the exploration of the search space. It settles the algorithms in the field of genetic and evolutionary computation where they have been originated, and classifies them into a few classes according to the complexity of models they use. Algorithms within each class are briefly described and their strengths and weaknesses are discussed.
- 1. S. Baluja, "Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning," Tech. Rep. No. CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, PA, 1994. Google Scholar
Digital Library
- 2. S. Baluja and S. Davies, "Using optimal dependency-trees for combinatorial optimization: Learning the structure of the search space," in Proceedings of the 14th International Conference on Machine Learning, 1997, pp. 30-38. Google ScholarDigital Library
- 3. P.A. Bosman, "Continuous iterated density estimation evolutionary algorithms within the IDEA framework," Personal communication, 2000.Google Scholar
- 4. P.A.N. Bosman and D. Thierens, "Linkage information processing in distribution estimation algorithms," in Proceedings of the Genetic and Evolutionary Computation Conference GECCO-99, Orlando, FL, W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela, and R.E. Smith (Eds.), 1999, vol. I, pp. 60-67.Google Scholar
- 5. J.S. De Bonet, C.L. Isbell, and P. Viola, "MIMIC: Finding optima by estimating probability densities," in Advances in Neural Information Processing Systems, M.C. Mozer, M.I. Jordan, and T. Petsche (Eds.), 1997, vol. 9, p. 424.Google Scholar
- 6. K. Deb and R.B. Agrawal, "Simulated binary crossover for continuous search space," Complex Systems, vol. 9, pp. 115-148, 1995.Google Scholar
- 7. J. Edmonds, "Optimum branching," J. Res. NBS, vol. 71B, pp. 233-240, 1967.Google Scholar
- 8. L.J. Eshelman and J.D. Schaffer, "Real-coded genetic algorithms and interval-schemata," in Foundations of Genetic Algorithms Workshop (FOGA-92), D. Whitley (Ed.), Vail; Colorado, 1992.Google Scholar
- 9. R. Etxeberria and P. Larrañaga, "Global optimization using Bayesian networks," in Second Symposium on Artificial Intelligence (CIMAF-99), Habana, Cuba, 1999, pp. 332-339.Google Scholar
- 10. M. Gallagher, M. Frean, and T. Downs, "Real-valued evolutionary optimization using a flexible probability density estimator," in Proceedings of the Genetic and Evolutionary Computation Conference, W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela, and R.E. Smith (Eds.), Orlando, Florida, USA, 1999, vol. 1, pp. 840-846.Google Scholar
- 11. D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley: Reading, MA, 1989. Google ScholarDigital Library
- 12. D.E. Goldberg, "Genetic and evolutionary algorithms in the real world," University of IIlinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IlliGAL Report No. 99013, 1999.Google Scholar
- 13. S. Handley, "On the use of a directed acyclic graph to represent a population of computer programs," in Proceedings of the First IEEE Conference on Evolutionary Computation, Piscataway, NJ, 1994, pp. 154-159.Google Scholar
- 14. G. Harik, "Linkage learning via probabilistic modeling in the ECGA," University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 99010, 1999.Google Scholar
- 15. G. Harik, E. Cantú-Paz, D.E. Goldberg, and B.L. Miller, "The gambler's ruin problem, genetic algorithms, and the sizing of populations," in Proceedings of the International Conference on Evolutionary Computation (ICEC'97), Piscataway, NJ, 1997, pp. 7-12.Google Scholar
- 16. G.R. Harik, F.G. Lobo, and D.E. Goldberg, "The compact genetic algorithm," in Proceedings of the International Conference on Evolutionary Computation (ICEC'98), Piscataway, NJ, 1998, pp. 523-528.Google Scholar
- 17. D. Heckerman, D. Geiger; and M. Chickering, "Learning Bayesian networks: The combination of knowledge and statistical data," Microsoft Research, Redmond, WA, Technical Report MSR-TR-94-09, 1994.Google Scholar
- 18. J.H. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press: Ann Arbor, MI, 1975. Google ScholarDigital Library
- 19. D. Knjazew and D.E. Goldberg, "OMEGA--Ordering messy GA: Solving permutation problems with the fast messy genetic algorithm and random keys," University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 2000004, 2000.Google Scholar
- 20. J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, The MIT Press: Cambridge, MA, 1992. Google ScholarDigital Library
- 21. S. Kullback and R.A. Leibler, "On information and sufficiency," Annals of Math. Stats., vol. 22, pp. 79-86, 1951.Google ScholarCross Ref
- 22. V. Kvasnicka, M. Pelikan, and J. Pospichal, "Hill climbing with learning (an abstraction of genetic algorithm)," Neural Network World, vol. 6, pp. 773-796, 1996.Google Scholar
- 23. L.A. Marascuilo and M. McSweeney, Nonparametric and Distribution-Free Methods for the Social Sciences, Brooks/Cole Publishing Company: CA, 1977.Google Scholar
- 24. T.M. Mitchell, Machine Learning, McGraw-Hill: New York, 1997. Google ScholarDigital Library
- 25. H. Mühlenbein, "The equation for response to selection and its use for prediction," Evolutionary Computation, vol. 5, no. 3, pp. 303-346, 1997. Google ScholarDigital Library
- 26. H. Mühlenbein and T. Mahnig, "Convergence theory and applications of the factorized distribution algorithm," Journal of Computing and Information Technology, vol. 7, no. 1, pp. 19-32, 1998.Google Scholar
- 27. H. Mühlenbein and T. Mahnig "FDA--A scalable evolutionary algorithm for the optimization of additively decomposed functions," Evolutionary Computation, vol. 7, no. 4, pp. 353-376, 1999. Google ScholarDigital Library
- 28. H. Mühlenbein, T. Mahnig, and A.O. Rodriguez, "Schemata, distributions and graphical models in evolutionary optimization," Journal of Heuristics, vol. 5, pp. 215-247, 1999. Google ScholarDigital Library
- 29. H. Mühlenbein and G. Paaß, "From recombination of genes to the estimation of distributions I. Binary parameters," in Parallel Problem Solving from Nature--PPSN IV, Berlin, A. Eiben, T. Bäck, M. Shoenauer, and H. Schwefel (Eds.), 1996, pp. 178-187. Google ScholarDigital Library
- 30. I. Ono and S. Kobayashi, "A real-coded genetic algorithm for function optimization using unimodal normal distribution crossovers," in Proceedings of the Seventh International Conference on Genetic Algorithms, San Francisco, T. Bäck (Ed.), 1997, pp. 246-253.Google Scholar
- 31. M. Pelikan, D.E. Goldberg, and E. Cantú-Paz, "Linkage problem, distribution estimation, and Bayesian networks," University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 98013, 1998.Google Scholar
- 32. M. Pelikan, D.E. Goldberg, and E. Cantú-Paz, "BOA: The Bayesian optimization algorithms," in Proceedings of the Genetic and Evolutionary Computation Conference GECCO-99, Orlando, FL, W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela, and R.E. Smith (Eds.), 1999, vol. I, pp. 525-532.Google Scholar
- 33. M. Pelikan, D.E. Goldberg, and E.Cantú-Paz, "Bayesian optimization algorithm, population sizing, and time to convergence," University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 2000001, 2000.Google Scholar
- 34. M. Pelikan, D.E. Goldberg, and E. Cantú-Paz, "Hierarchical problem solving by the Bayesian optimization algorithms," University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 2000002, 2000.Google Scholar
- 35. M. Pelikan, D.E. Goldberg, and E.Cantú-Paz, "Linkage problem, distribution estimation, and Bayesian networks," Evolutionary Computation, vol. 8, no. 3, pp. 311-341, 2000. Google ScholarDigital Library
- 36. M. Pelikan and H. Mühlenbein, "The bivariate marginal distribution algorithm," in Advances in Soft Computing--Engineering Design and Manufacturing, London, R. Roy, T. Furuhashi, and P.K. Chawdhry (Eds.), 1999, pp. 521-535.Google Scholar
- 37. I. Rechenberg, Evolutions strategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution, Frommann-Holzboog: Stuttgart, 1973.Google Scholar
- 38. F. Rothlauf, D.E. Goldberg, and A. Heinzl, "Bad codings and the utility of well-designed genetic algorithms," University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 200007, 2000.Google Scholar
- 39. S. Rudlof and M. Köppen, "Stochastic hill climbing with learning by vectors of normal distributions," in First On-line Workshop on Soft Computing, Nagoya, Japan, 1996.Google Scholar
- 40. R.P. Salustowicz and J. Schmidhuber, "Probabilistic incremental program evolution: Stochastic search through program space," in Machine Learning: ECML-97, M. van Someren and G. Widmer (Eds.), vol. 1224 of Lecture Notes in Artificial Intelligence, 1997, pp. 213-220. Google ScholarDigital Library
- 41. J. Schwarz and J. Ocenasek, "Experimental study: Hypergraph partitioning based on the simple and advanced algorithms BMDA and BOA," in Proceedings of the Fifth International Conference on Soft Computing, Brno, Czech Republic, 1999, pp. 124-130.Google Scholar
- 42. M. Sebag and A. Ducoulombier, "Extending population-based incremental learning to continuous search spaces," in Parallel Problem Solving from Nature--PPSN V, Berlin Heidelberg, 1998, pp. 418-427. Google ScholarDigital Library
- 43. I. Servet, L. Trave-Massuyes, and D. Stern, "Telephone network traffic overloading diagnosis and evolutionary computation techniques," in Proceedings of the Third European Conference on Artificial Evolution (AE'97), NY, G. Goos, J. Hartmanis, and J. Leeuwen (Eds.), 1997, pp. 137-144. Google ScholarDigital Library
- 44. R. Shachter and D. Heckerman, "Thinking backwards for knowledge acquisition," AI Magazine, vol. 7, pp. 55-61, 1987.Google Scholar
- 45. D. Thierens, "Analysis and design of genetic algorithm," Ph.D. thesis, Katholieke Universiteit Leuven, Leuven, Belgium, 1995.Google Scholar
- 46. H.-M. Voigt, H. Mühlenbein, and D. Cvetković, "Fuzzy recombination for the breeder genetic algorithm," in Proceedings of the Sixth International Conference on Genetic Algorithms, 1995, pp. 104-111. Google ScholarDigital Library
Index Terms
- A Survey of Optimization by Building and Using Probabilistic Models
Recommendations
Probabilistic model-building genetic algorithms
GECCO '07: Proceedings of the 9th annual conference companion on Genetic and evolutionary computationProbabilistic model-building algorithms (PMBGAs) replace traditional variation of genetic and evolutionary algorithms by (1) building a probabilistic model of promising solutions and (2) sampling the built model to generate new candidate solutions. ...
Optimization of multimodal continuous functions using a new crossover for the real-coded genetic algorithms
In this study, a new crossover approach to the real-coded genetic algorithm is proposed. The approach is simply based on efficiently tuned real-coded crossover operation using the probability distribution function of Gauss distribution to generate ...
Probabilistic model-building genetic algorithms
GECCO '11: Proceedings of the 13th annual conference companion on Genetic and evolutionary computationProbabilistic model-building algorithms (PMBGAs) replace traditional variation of genetic and evolutionary algorithms by (1) building a probabilistic model of promising solutions and (2) sampling the built model to generate new candidate solutions. ...
Comments