Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Evolvability and Speed of Evolutionary Algorithms in Light of Recent Developments in Biology

Evolvability and Speed of Evolutionary Algorithms in Light of Recent Developments in Biology Hindawi Publishing Corporation Journal of Artificial Evolution and Applications Volume 2010, Article ID 568375, 28 pages doi:10.1155/2010/568375 Review Article Evolvability and Speed of Evolutionary Algorithms in Light of Recent Developments in Biology 1, 2 1 Ting Hu and Wolfgang Banzhaf Department of Computer Science, Memorial University, St. John’s, NL, Canada A1B 3X5 Norris Cotton Cancer Center, Dartmouth-Hitchcock Medical Center, Lebanon, NH 03756, USA Correspondence should be addressed to Ting Hu, Ting.Hu@Dartmouth.edu Received 15 September 2009; Accepted 24 February 2010 Academic Editor: Franz Rothlauf Copyright © 2010 T. Hu and W. Banzhaf. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Biological and artificial evolutionary systems exhibit varying degrees of evolvability and different rates of evolution. Such quantities can be affected by various factors. Here, we review some evolutionary mechanisms and discuss new developments in biology that can potentially improve evolvability or accelerate evolution in artificial systems. Biological notions are discussed to the degree they correspond to notions in Evolutionary Computation. We hope that the findings put forward here can be used to design computational models of evolution that produce significant gains in evolvability and evolutionary speed. 1. Introduction natural evolution has improved profoundly in biology. This progress has, to a large degree, not been incorporated yet into The field of Evolutionary Computation (EC) has seen enor- computational models of evolution and therefore cannot be mous progress since it was founded in the Sixties and harvested for applications. We have argued that adopting Seventies of the 20th century [1–11], inspired by the new knowledge about natural evolution generated in areas evolutionary processes observed in the living world. such as molecular genetics, cell biology, developmental In EC, candidate solutions to optimization or learning biology, and evolutionary biology would substantially benefit problems are represented by structures similar to gene EC [16, 17]. sequences and their phenotypic expressions. The ensemble The question then arises what the most important and of such solutions is referred to as a population. Evolutionary revolutionary discoveries are in biology in recent times, and operators, such as mutation, recombination, and selection, how they can be sufficiently abstracted to provide material are applied to this population. Solutions gradually improve for computational models. As the number of scientists by repeating a variation-selection cycle through numerous working in the areas mentioned above is now higher than iterations of the evolutionary process. Essentially a search at any other time of the past and can be estimated to be well method, EC, often produces well-performing solutions to over a million, it becomes nontrivial to select those aspects complex optimization and learning problems arising from of evolution that will have the most impact in computational various areas, to the point where its problem solving models. A number of books have appeared in recent years capability mirrors or even exceeds that of humans [12]. Yet, that provide some guidance in this quest (see, e.g., [18–25]). EC is not without weaknesses, and new algorithmic variants Here we restrict ourselves to mainly review the con- are constantly being introduced, studied, and applied. cepts of evolvability and the speed of evolution. This is The fundamental idea of EC was gleaned from biology, motivated by the fact that EC approaches often suffer from and more specifically, from Darwin’s theory of evolution by progressive slow-down of evolutionary speed. While under natural selection [13] as embodied in the Neo-Darwinian some circumstances appreciated as convergence to a global synthesis [14, 15]. In the past decades, however, knowledge of optimum, for many real-world tasks convergence and the 2 Journal of Artificial Evolution and Applications corresponding slow-down of fitness improvements as well Altenberg [46] describes evolvability from the viewpoint as the reduction in the diversity of solutions is more a of EC as the ability of a genetic operator or representation predicament than an advantage. This is especially true for scheme to produce offspring fitter than their parents. In difficult problems where there is no hope to find an optimal biology, Kirschner and Gerhart [47] consider evolvability as solution, but where good solutions would already provide an organism’s capacity to generate heritable and selectable a benefit in the application. As a result, the development phenotypic variation. An explicit comparison between evolv- of systems that show continued evolutionary potential, ability in biological and computational systems has been open-ended evolution,asithas been termed,has gained performed by Wagner and Altenberg [45]. In their view, prominence. evolvability must be seen as the ability of random variants Open-ended evolution is a hallmark of Life. Thus, one to produce occasional improvements, which would depend alternative route to explore this topic in computation is critically on the plasticity of the genotype-phenotype map. via an Artificial Life approach [26]. And so a number of The authors emphasize “variability” determined by the artificial systems have been designed in the meantime with genotype-phenotype map as the propensity to vary, rather the aim to simulate organic life in silico,suchas Tierra than variation itself. Marrow [48] suggests that evolvability [27], Avida [28], and Evita [29]. In these systems, computer means the capability to evolve, and this characteristic should code is regarded as “digital organisms” with CPU time be relevant to both natural and artificial evolutionary the “energy” resource and memory the “material” resource. systems. He discusses a number of important contributions Digital organisms evolve through interactions with their on this topic in both biology and EC and raise some open neighbors and competition for resources. Fitness is not an questions for further research. explicit notion in these systems. However, this is still only Recently, a growing number of evolutionary biologists an initial step towards understanding and realizing open- and computer scientists have shown interest in this topic. ended evolution in artificial systems, as Bedau and Brown In an evolutionary system, many properties of a population [30] report that the long-term capability to adaptation are considered related to evolvability, including facilitation seems to be missing from these systems compared to real of extradimensional bypass and robustness against genotypic organisms (for an example of long-term capabilities in variation [49, 50], redundancy, flexibility during develop- simple organisms, see [31]); that is, this type of artificial mental processes [47], and mutation rate adaptation [51]. evolution lacks evolvability in the “long-term”. The notion of evolutionary capacitance has also been used in In EC, the situation is even exacerbated by the existence this context ([52], and references therein). of an explicitly defined fitness function, often in the form The detection and measurement of evolvability is an of a simple scalar. However, while it is the holy grail of intriguing and nontrivial problem. Phenotypic fitness is computational models of evolution to achieve continued directly observable and serves as a selection criterion. evolutionary potential, which has—to the best of our However, as a potential to generate better fitness and a knowledge—not been reached to date, progress has still capability for adaptive evolution, evolvability is a different been achieved by studying more limited concepts like that type of observable, which is more difficult to observe and of evolvability. In a nutshell, the hope is that by relating to quantify. Although a formal methodology on measuring properties of natural evolutionary systems to mechanisms evolvability has not yet been agreed upon in the literature, used by Nature to achieve them, we might learn enough to some empirical methods have been proposed nevertheless. design algorithmic mechanisms that exhibit similar features. Nehaniv [53] proposes the perspective of using evolu- So let us start by looking more closely at evolvability and the tionary system complexity to describe and measure evolv- rate of evolution. ability. He defines the exhibited evolvability as an observable outcome generated by evolvability and measures evolvability by the rate of increasing complexity of evolutionary entities 1.1. Evolvability. In the process of evolution, genotypic vari- in an evolutionary system. Wagner proposes to simply mea- ation explores new evolutionary material, the corresponding sure the number of nonneutral 1-step mutation variations in phenotypic variation provides adaptive characteristics, and a biological system of particular relevance to RNA evolution stabilization operators like selection preserve improvements in order to quantify evolvability [54]. As one can see, over previous generations. The cooperation of these activities Nehaniv’s definition entailing more complex entities will in is what allows evolution to work. Thus, the core mechanism general also lead to a larger number of nonneutral 1-step of evolution is to assemble the forces of these operations that variations and thus increase this measure of evolvability. yield adaptive improvements implying the evolvability of an Another perspective on evolvability is provided by Earl evolutionary system. A growing number of efforts have been and Deem [55] who suggest that evolvability can be selected dedicated to understanding [25, 32–38] and enhancing [39– for by variation in the environment. By observing genetic 45] evolvability. changes in protein evolution, they find that rapid or dramatic While the concept of evolvability is still very much environmental change generates strong selection pressure under discussion, we will adopt a definition that is equally for evolvability. Thus, high evolvability can be detected applicable to natural and artificial systems. and favored by such selection pressure. For an artificial Definition 1. Evolvability is the capability of a system to evolutionary system Reisinger et al. concur when they generate adaptive phenotypic variation and to transmit it via propose an indirect encoding representation to improve an evolutionary process. evolvability [56, 57]. A gradually changing fitness function Journal of Artificial Evolution and Applications 3 is designed to measure evolvability of representations and variation. If k = k , the fixation of these two types of changes a s to evolve a population that is adaptive under different is at the same rate, a special case indicating, for example, environments. Furthermore, as the pace of change of the pseudogenes. To summarize, measuring a large k /k ratio a s fitness function increases, stronger selection pressure for suggests that adaptation has been generated (and fixed) at a evolvability is imposed. high rate. This measurement has been widely applied in the analysis of adaptive molecular evolution and is accepted as a general method for measuring the rate of gene sequence 1.2. Rate of Evolution. Related to the theme of evolvability is evolution in biology. that of the rate of evolution. Evolvability defines how likely a Other than at the molecular level, Worden has defined system can generate adaptive phenotypic variations whereas the concept of genetic information in the phenotype (GIP) the rate of evolution describes how fast this evolutionary in his work on the speed limit for evolution [63]. GIP is process can proceed. The rate of evolution is a fascinating meant to be a measure of the amount of genetic information topic in evolutionary biology and has caused many debates expressed in observable phenotype, and he uses the rate of already since Darwin’s time. Darwin himself held the view increasing GIP to describe the rate of evolution. He proposes of phyletic gradualism, hypothesizing that most evolution that GIP measurement can be applied in both biology and occurs uniformly, gradually moulded by selective conditions. EC. Others were of a different opinion, and Eldredge and Gould As we can see from these examples, both phenotypic proposed the theory of punctuated equilibria [58]. According effects and genotypic effects have to be taken into account to this idea, evolution occurs through bursts of innovation when measuring the rate of evolution. followed by long periods of stasis, a major challenge to In artificial systems used for EC, the goal of evolution Darwin’s orthodoxy. is much more specific than in nature: to find the solutions to a given problem. The rate of evolution in EC, therefore, Definition 2. Rate of Evolution is a quantitative measure usually refers to the speed of solving a specific problem, of the changesobservableinanevolutionarysystemover for example, to the speed of fitness improvements or the generational (or otherwise appropriately defined) time- speed of approaching a fixed objective. The ability to define scales. explicit phenotypic fitness is one of the most distinguishing In biology, the rate of evolution has different defini- features that differentiate EC from natural evolution. For the measurement of the rate of evolution, however, it offers a tions and measures depending on the underlying objects examined, for instance, gene sequences, proteins, organisms, trap: to go entirely phenotypic, since in order to investigate and so forth. In molecular biology, the rate of evolution the performance of a computational model, the rate of usually describes the rate of mutants being preserved as evolution is mostly measured by the speed of fitness function advantageous, that is, those that can generate phenotypic improvements. Other ad hoc methods are also utilized in EC, improvements. This is observed by looking at the fixation of like the efficiency of algorithms and CPU time. alleles in genes. Biologists use the k /k ratio to measure the Another method, however at a deeper level than sim- a s rate of gene sequences evolution [59–62]. It is known that ple fitness function improvement, deserves mentioning: Bedau and Packard [64], for instance, propose a method some changes to a gene sequence may lead to differences in the amino acid sequence of an encoded protein while others for visualizing evolutionary adaptation. This method is will not, due to the degenerate code employed for trans- useful to identify and measure the capability of creating lation. Therefore, such a measure can be used to compare adaptation during evolutionary processes. It is based on two homologous protein-coding gene sequences of related calculating evolutionary activity statistics of components in species. The k /k ratio resulting from a measurement of an evolutionary system, such as the numbers of particular a s the number of nonsynonymous (amino acid) substitutions genes (or alleles) in each generation and the persistence per nonsynonymous site (k ) to the number of synonymous of these genes (or alleles) during evolution. During a substitutions per synonymous site (k ) characterizes the rate decade of extensive development, the notion of evolution- of evolution between these two sequences. Since k measures ary activity has been applied to various scales of genetic neutral evolution (without considering functional improve- components, including alleles, allele tokens, phenotypic equivalence classes of alleles, and whole genotypes, in ments under selection pressure), the k /k ratio reflects a s the amount of adaptive evolution against the background both artificial evolutionary systems and in the biosphere. amount of variation. Note that this is an approximation since In their more recent work, these authors emphasize two there are nonsynonymous changes in amino acid sequences aspects for evolutionary adaptation: the extent and the that do not change the function of the protein in which they intensity of evolutionary activity [65, 66]. The extent of appear. evolutionary activity refers to how much of an adaptive In case k /k > 1, fixation of nonsynonymous substitu- structure is present in an evolutionary system, while the a s tions is faster than that of synonymous substitutions, which intensity concerns the capability of generating new adaptive structures. The measures of cumulative evolutionary activity means that positive selection fixes amino acid changes faster than silent changes. Mostly, however, one finds k /k < 1, the and mean cumulative evolutionary activity characterize the a s case where deleterious substitutions are eliminated by puri- extent of a system’s evolutionary adaptation. On the other fying selection (negative selection), and the rate of fixation hand, new activity is a measure of the intensity of a of amino acid changes is smaller than the background rate of system’s evolutionary adaptation. Evolutionary activity can 4 Journal of Artificial Evolution and Applications be quantified and visualized during evolutionary adaptation. organismal evolution, since they provide abundant material Its derivative is the concentration of a component’s current for mutation and selection to generate new gene functions presence, and its second derivative can be argued to reflect in a modular way. By studying the recent nucleotide the rate of evolution at a particular time. Evolutionary substitutions in human evolution, Hawks et al. [73]find activity is also claimed to be a straightforward method for that, as a population becomes more adapted to its current studying evolvability [65]. The argument is that, since a environment, the rate of adaptive evolution slows down. system with high evolvability can create highly adaptive vari- However, a growing population size can provide the potential ation, the quantification of evolvability can be achieved by for rapid adaptive innovation. Thus, enlarging the popula- measuring the levels of extent and intensity of evolutionary tion size and changing environmental conditions can both activity. promote the rate of adaptive evolution. Kashtan et al. [74] confirm in a recent report that a varying environment can 1.3. Observations. It has been observed both in the pale- speed up evolution in an artificial evolutionary system. Other ontological record [67] and, more recently, through studies properties and techniques on the acceleration of evolution of molecular evolutionary systems [68] that the rate of have been also investigated in biology and computing. evolution in biological systems is greatly varying. At times This review discusses evolvability and methods for selective sweeps pass through a population that all but accelerating artificial evolution by drawing ideas from com- plex natural systems. Notions from biology are introduced wipe out certain less advantageous alleles, while at other times seemingly nothing happens in terms of evolutionary and their potential in designing new algorithms in EC is changes. Thus we can legitimately speak of an acceleration discussed. The review is organized along the work-flow of evolutionary algorithms. Section 2 starts with the character- of evolution under certain conditions, and of a slow-down under others. istics of populations; variation operations are investigated In EC, on the other hand, the state of the art can be in Section 3, separated into genotypic variation, phenotypic summarized by the observation that under most conditions, variation, and the transformations between them. Selection algorithms tend to show exponential decay in progress is discussed in Section 4, together with notions of fitness. The review concludes with a summary in Section 5. toward an optimum with often a painfully slow convergence for a large part of runs, or, alternatively, a premature con- vergence of the algorithm to the detriment of the produced 2. Population solution, resulting in a stagnation of the search algorithm before it has reached an acceptable outcome. This has been The general idea of EC is to adopt mechanisms of evolution realized to be related to the record dynamics shown in many from nature. In Darwin’s theory of evolution, both the natural and human systems [69, 70]. Record dynamics refers notion of variation and of natural selection are based on to the slow-down of records, for instance, in competition natural populations. However, populations simulated in a sports events, where after some time records become more computer are usually simplified from their natural counter- and more difficult to break, due to the unchanging human parts. A notable difference between natural and simulated physiology and the limitations this physiology imposes on population systems is that no identical individuals exist achieving certain targets. in a natural population, whereas this is allowed and most Contrast that with the world of natural evolution, where often the case with simulated populations. Tiny variances are there is always a way to beat previous opponents, and to considered an essential aspect of natural populations as they evolve in another direction that allows to increase fitness lead to the large diversity in natural evolution via amplifying in some unforeseen ways. Surely, the implicit definition of effects produced by selection. Hence, more details should be fitness plays an important role here, as it allows enormous taken into account also in computational populations that flexibility in achieving function. Further, the fact that the will ultimately allow a better differentiation of individuals. environment is permanently changing can be expected to be Because the representation chosen for individuals and the a key contributor to the evolvability in natural environments. size parameter of a population can affect the performance Finally, the ability of living tissue (an intentionally vague of a computational model, it is an essential step in EC to term) to assemble in a hierarchical fashion, starting from determine these features of the simulated population. atoms and molecules upward into ecosystems, provides building blocks and interactions of great richness that allows evolution to progress at different speeds, and notably to 2.1. Representation. The first step for setting up evolution accelerate under favorable conditions. with a population is to decide on the representation of A number of detailed observations on the factors that evolutionary individuals. Each individual should be encoded can accelerate evolution in the living world have been as a candidate solution to a given problem, which sub- made in the past. Simon [71] raises the “nearly completely sequently determines the search space of the algorithm. decomposable” property in multicellular organisms and Therefore, choosing a representation is important because proposes it to be an important property that can lead to it predicates the input to the search process that should faster fitness increases. In research on yeast genes, Gu et al. produce a satisfactory output. Here, we highlight a two [72] report rapid evolution of gene expression and regulatory biological mechanisms, a protection mechanism for robust divergence after gene duplication. Gene (and segmental) information preservation, and a communication mechanism duplication events contribute substantially to genomic and for information interaction between different molecules. Journal of Artificial Evolution and Applications 5 2.1.1. Robustness and Redundancy. Living systems may seem Table 1: Mechanisms responsible for creating redundancy and antiredundancy at the cellular level. (Adapted from Krakauer and wasteful and luxurious to computer scientists. The most Plotkin [86].) distinguishing aspects of biology compared to other natural sciences are complexity and diversity, which are indeed of Redundancy Antiredundancy central concern to biologists. In the face of cruel com- Overlapping reading petitive circumstances, organisms show great redundancy Gene duplication frames and resilience. Redundancy exists at different levels in Nonconservative codon natural organisms, including the genomic, transcriptomic, Neutral codon usage bias and phenotypic levels, all for the benefit of the robustness of the organism. — Gene silencing We adopt Wagner’s definition for robustness here. Polyploidy Haploidy Single regulatory element Multiple regulatory Definition 3 (robustness). The robustness of a biological or for n genes elements for n genes engineering system is its capability to continue functioning Chaperone and heat shock in the face of genetic or environmental perturbations [25]. proteins Checkpoint genes inducing Checkpoint genes In biology, the genome of an organism is defined as apoptosis promoting repair the information encoded in DNA sequences and inherited Telomerase induction Loss of telomerase from generation to generation. The double helix structure of DNA sequences itself is a form of protective redundancy Dominance Incomplete dominance of genetic information. Genomes carry genes and other Autophagy — noncoding DNA sequences. A gene is a string of base pairs mRNA surveillance — grouped by a function that is embodied in a protein or Bulk transmission Bottlenecks in transmission polypeptide (protein fragment). Noncoding DNA sequences, Molecular quality control — formerly called “junk DNA”, are not expressed as proteins, tRNA suppressor molecules — although they might be transcribed into RNA and involved Modularity — in manufacturing proteins or controlling that process. All in all, genes are only quite small a fraction of the entire Multiple organelle copies Single organelle copies genome [75], with more than 98% of the human genome, Serial metabolic pathways Parallel metabolic pathways for instance, being noncoding DNA sequences [76]. Fur- Uncorrelated gene Correlated gene expression thermore, even a gene sequence itself is divided into exons expression and introns, where exons directly determine the protein DNA error repair Loss of error repair amino acid sequence but introns do not. Nevertheless, these noncoding DNA sequences are not useless. Recent biological discoveries show that they play an important role in the ncRNA. About 98% of all transcribed sequences in humans regulation of gene transcription [77]. Regulation mecha- are of this type [84]. Although many of the functions of these nisms will be discussed later in Section 3.3.1.Wrenetal. noncoding sequences are unclear, the high complexity of the [78] find that tandem-repeat polymorphisms in genes are transcriptome hints at its importance in the mechanisms of quite common, and that such polymorphisms can enhance organizing gene expression in a robust way [85]. the ability of some genes to respond rapidly to fluctuating Krakauer and Plotkin [86] go further and propose the selection pressure. The mechanism of gene duplication will new concept of antiredundancy. In their opinion antiredun- be discussed in detail in Section 3.1.1. Moreover, diploid dancy emerges as does redundancy in cells, and natural organisms have two copies of each chromosome, one copy organisms would be able to modify the redundancy proper- inherited from each parent. Recent research has also found ties of genotypes during evolution. Table 1 shows a summary that a large number of DNA segments appear in more of observed mechanisms responsible for both redundancy than two copies. Copy Number Variations (CNVs) in human and antiredundancy at the cellular level. Mechanisms for and other mammalian genomes discovered lately account redundancy mask the phenotypic effect of mutations and for a substantial amount of genetic variation other than allow mutants to stay in populations, while mechanisms for single nucleotide polymorphisms (SNPs) [79–82]. CNVs and antiredundancy enhance the efficiency of local selection to SNPs are considered to substantially contribute to genotypic remove damaged components. variation, a phenomenon that will be discussed in detail later Going even further, we finally arrive at the phenotype: in Section 3.1.1. redundancy at the phenotypic level lies in an organism’s Further down the line toward the phenotype is the robustness against intrinsic or environmental changes. With transcriptome which describes the set of all transcribed low robustness, a species will gradually decline and finally go RNAs in cells. In the human transcriptome, the proportion of extinct due to lethal mutations because random mutations transcribed nonprotein-coding sequences is large and shows in the genome usually cause deleterious changes with a great complexity [83]. Substantially more DNA is transcribed potential to destroy the offspring. than is translated, and only a small proportion of mRNAs are It seems that robustness and evolvability have a con- translated into proteins. The rest is called noncoding RNA or tradictory relationship to each other. When a system has 6 Journal of Artificial Evolution and Applications high robustness in its genome, it can be tolerant to intrinsic redundancy into our algorithms to make them resilient or environmental changes, but that should leave it less against changes while improving adaptivity. Such capabilities evolvable, as variation would be masked, and vice versa. In certainly complicate the algorithms but may be worthwhile recent contributions, Wagner [50, 54] resolves this apparent if the resulting robustness can generate higher evolvabil- contradiction. He distinguishes robustness and evolvability ity when applying intense pressure to produce adaptive as quantities at both the genotypic and the phenotypic responses. Evolution might even be accelerated because levels. If one considers genotype, the more robust a genetic the system has a quick and robust reply to evolutionary sequence is, the less innovation this sequence will produce. pressures. With the growth of computational power available However, robustness and evolvability are characteristics of today ideas like these can be more easily explored than an entire system and if investigated at phenotypic level before. show a strong correlation. A system with high phenotypic robustness harbors a great number of “neutral” variations 2.1.2. Molecular Interaction. Natural living systems are that have no functional effects. These neutral variations remarkably diverse starting from so simple organisms as do not change phenotypic function during relatively static bacteria to highly complex creatures such as primates. evolutionary periods but may be able to generate adaptation This diversity is not the result of vastly different chemical later under certain genetic or environmental changes. Thus, constituents of organisms. In fact, many species carry out a system with high phenotypic robustness simply masks similar metabolic, cell division and replication processes changes but provides great potential for phenotypic inno- under similar assembly principles [98]. The differences that vation in the future, for example, if conditions change and distinguish species are caused by the arrangement and previously neutral changes suddenly have an effect. This is distribution of basic building blocks [99] and molecular the core of the argument that high robustness and high interactions contribute significantly to these organizational evolvability are in fact correlated in nature [54], and this mechanisms. has been supported in many subsequent research [87–91]. Molecular interactions in a cell happen between the Specifically, Draghi et al. [92]gofurtherand forfirsttime same type of molecules, such as protein-protein interactions, quantify the effects of robustness/neutrality on adaptation in or between different types of molecules, such as protein- an evolving population. They suggest a complex relationship DNA or RNA-protein interactions. Signals can also be between robustness and evolvability, which depends on the sent between and responded to by cells in multicellular topology of the genotype network. Their results indicate that organisms. Molecular interactions can be triggered by energy if the genotype space has no epistatic effects, a more robust supply, for example, in metabolic pathways, chains of interac- population will have less evolvability. With epistasis, on the tions catalyzed by enzymes, or triggered by external stimuli, other hand, they find a nonmonotonic relationship between for example, signaling pathways that enable communication robustness and evolvability, that is, evolvability is the highest through the cell membrane [100]. Proteins are not only at an intermediate level of robustness. a product enabling various organismal structures but also Redundancy is wide-spread in natural organisms as an work as control factors in various processes from the efficient protection mechanism against internal or environ- synthesis of a cell, metabolism, gene regulation, to sexual mental changes, whereas in EC models components that reproduction. do not seem to be immediately relevant are often consid- Metabolism is a key process to maintain the growth and ered superfluous. In recent years, however, representation reproduction of cells. The metabolic network of a cell is an redundancy has arisen as a by-product of computational elaborated set of numerous chemical reactions catalyzed by evolution and has attracted increasing interest from EC enzymes [101]. Different types and amounts of enzymes are researchers. produced according to different energy supplies, and these enzymes will determine different metabolic pathways by Definition 4 (representations redundancy). In genetic and their catalysis. In the process of gene expression, the function evolutionary algorithms, representations are redundant if the achieved can be controlled by molecular interactions. For number of genotypes exceeds the number of phenotypes instance, the process of how a parsimonious bacterium [93]. responds to food supplies during metabolism shows a simple genetic switch mediated by molecular interactions. Since Rothlauf and Goldberg [93] examine the effects of the metabolic pathways of bacteria are much simpler than redundant representations on the performance of an EC those of multicellular organisms, the regulation of gene system both theoretically and empirically and propose that expression is more easily understandable in bacteria. The redundant representations can increase the reliability and phenomenon of enzyme induction [22] describes the adapta- efficiency of EC models. Specifically in genetic programming, tion of a bacterium to material supplies by producing varying representation redundancy is usually identified as introns (or amounts of enzyme. What triggers this production and noneffective, neutral code) [1] in programs. Researchers have how does this mechanism work? The Jacob-Monod model investigated both the positive and negative effects of introns (shown in Figure 1) first described the regulation mechanism [94–97], and a positive relation between neutral code and of inhibiting or repressing genes by inhibitory proteins, evolvability in genetic programming has been suggested. The called repressors in bacteria. The binding of lactose to a important role of redundancy in evolvability has now been repressor enables the production of RNAs by removing the realized. We might, therefore, consider designing protective repressor from its binding sites on the gene sequence where Journal of Artificial Evolution and Applications 7 Now No lactose My landing Lactose ican’t present present site is blocked! bind! RNA polymerase Lactose Repressor RNA Repressor polymerase No RNA made Makes RNA Lac gene Lac gene (a) (b) Figure 1: The genetic switch in the Jacob-Monod model. A specific repressor protein acts as a switch. When it binds to a DNA site near the gene encoding beta-galactosidase, the RNA polymerase protein cannot bind nor can it synthesize RNA from the gene. The gene is turned off. When lactose is present, it binds to the repressor and keeps it from the DNA site. The gene turns on. (Adapted from Kirschner and Gerhart [22].) RNA polymerase can bind. However, this is not a simple makes eukaryotic cells well conserved but enormously adap- on-off switch model. The continuity lies in the binding tive to generate new phenotypes in changing environments duration which determines the rate of protein synthesis. [103]. Therefore, if more sugar is absorbed during metabolism, Computational models have already been used to analyze more protein is synthesized by RNA translation. This simple and understand complex multi-input/output and higher- sugar metabolism model captures the mechanism of how order signaling systems have been examined in bioinfor- a repressor affects gene function. The enzyme here works matics [104]. In contrast, current EC models are mostly as a trigger for the protein synthesis process under various limited to representing evolutionary material based on the molecular interactions. In addition, most enzyme effects infrastructure of natural organisms, while disregarding the are sensitive to ambient temperature [102], which is an vast potential of interaction mechanisms for regulation and important parameter to control metabolic interactions. signaling at both the molecular and cellular levels. The Signaling and cellular responses to signals are complex. absence of such mechanisms in EC, however, points to These responses are controlled by a plethora of positive and significant research opportunities in this area. negative feedback loops. The presence of feedback compli- cates the simple picture of a linear pathway but is an essential 2.2. Population Size. After the encoding of an individual part of the signaling process [98]. This makes signaling is determined, a population is set up. Several features of pathways involving molecular or cellular communication a a computational population are tightly connected to its network-like structure, with complex regulatory processes evolutionary capabilities, the most relevant of which is at work. The cellular infrastructure of eukaryotic organisms population size itself. is only a few times larger than that of bacteria, but the In nature, different species have different population complexity of their signaling network control differs greatly, sizes, a characteristic that plays an important role in evolu- by orders of magnitude. The linkage between various parts tion. In the living world it is common that smaller groups of the gene expression apparatus in eukaryotic organisms constituting species evolve faster, though smaller groups have is weakened by a far less precisely defined control than a higher probability of becoming extinct, while species with that found in prokaryotic cells [47]. For instance, geometric larger populations evolve slower and can stay unchanged for requirements for binding sites are significantly relaxed in relatively long periods. However, neither a small nor a large eukaryotic gene regulation. A repressor does not have to population size is unconditionally beneficial in evolution. bind at the exact position of a target but needs only to The relation between them should be understood in different bind in the neighborhood. By lowering constraints for scenarios. cooperation, such a weak linkage also enables potential The study of population genetics was formulated by interactions between different gene sequences. Signaling Fisher [105], Haldane [106], and Wright [107]. It focuses between cells is possible only after a sufficiently large number on gene frequency changes in populations under the effects of repressors participate simultaneously. A single signal may of natural selection, mutation, genetic drift, and population incur a very complex response [49]. Allosteric proteins, size fluctuation. In this field, scientists have examined which have multiple sites for interaction, also make gene the role of population size in molecular evolution using expression more flexible because they have different sites mathematical analysis. The rate of molecular evolution for different functions. Regulatory decisions on which genes is usually measured by the nonsynonymous to synony- are transcribed when, where, and under what circumstances mous substitution ratio k /k , discussed in Section 1.2. a s 8 Journal of Artificial Evolution and Applications Decades ago Kimura [108] proposed a strong dependency populations will also favor evolving robustness by increasing of the rate of molecular evolution on population size. genetic drift pressure and a buffering mechanism of hiding More recently, Gillespie [109, 110] has conjectured that mutations from being reduced by selection. This hypothesis there is only a very weak dependency on population size. is supported by Elena et al. [117]. Among the different Somewhat in the middle between these opinions, Ohta authors, there is agreement that the effect of population size, [111] finds population size to be related to the rate of either large or small, varies in different models. evolution under particular assumptions regarding mutation In general, the size of populations in EC is orders of types. The nearly neutral theory of molecular evolution magnitude lower than the size of populations of many proposed by Kimura and Ohta [112, 113] predicts that naturally occurring species, especially those of simpler there is a substantial number of nearly neutral mutations organisms like bacteria. The commonly adopted population (including slightly deleterious and slightly advantageous size in EC varies from tens to thousands, with a few ones) in molecular evolution, and that these contribute exceptions. Genetic Programming generally uses relatively to evolution by providing potential for future phenotypic large populations, due to the more complex and nonlinear innovation. Ohta [111] predicts that population size affects fitness landscape than can be found in other branches of the rate of evolution under various mutation scenarios. If the EC family. However, the size of GP populations run is most mutations are deleterious, a smaller population can limited by resource constraints in the range of hundreds evolve faster, because the chance of a slightly deleterious of thousands. Some of these algorithms have to run on mutant being favored by selection is greater within a smaller parallel machines or on GPUs, since the evaluation of a large population and these nearly neutral mutations bring genetic population of individuals requires enormous computational variation and may further trigger phenotypic innovation. In power (see, e.g., [118–120]). An order of magnitude like contrast, if mutations are mostly advantageous, the rate of that of humankind, a billion individuals, is unheard of in evolution in a larger population is greater. If most mutations EC approaches, which already points to a vast potential for are neutral, the evolution rate is nearly independent of doing research on EC in the future. A whole landscape of EC population size. Since in general random mutations are more methods might emerge with populations that are large. deleterious than advantageous in natural systems, species As a result of the use of small population sizes in the EC with a small population size usually evolve faster. community, efforts have been dedicated to the optimization A number of studies focus on testing the relation between of population size [121], since a high correlation between population size and evolution rate by using comparisons. population size and the performance of an EC algorithm Island endemic species usually have small population sizes is presumed. The challenge is that adapting population because they are restricted to a limited geographical region. size is problem-specific and to date it is still unclear how Woolfit and Bromham [114] study species on islands in to estimate the relation among various EC parameters. In support of the effect of population size on the rate of general, current work on this topic concentrates on two tasks: molecular evolution. They compare island endemic species (i) initializing a proper population size prior to a run, and to closely related species on a nearby mainland and find (ii) adjusting population size during a run. Most theoretical that island endemic species have a significantly higher work on population size initialization is based on Goldberg’s nonsynonymous to synonymous nucleotide substitution component decomposition approach and the notion of ratio than their counterparts on the mainland. This result Building Blocks [122, 123]. With many other publications, indicates that a decrease in the population size will lead these contributions propose to choose the population size to an increase in the rate of evolution. Wright et al. [115] according to the “hardness” of a specific problem. They study tropical species which are generally regarded to have state a general principle in setting population size: the more a rapid molecular evolution rate due to several factors such difficult a problem is, the more diversity is required and the as latitude and climate. It is believed that tropical organisms larger the population should be. possess great species richness and dynamics with small but In the meantime, it has been found that even for a highly diverse populations [116]. However, there are also specific problem the requirements for population size may exceptions in that increasing population size can accelerate differ during different stages of evolution. As a result, evolution as well. By studying the recent rapid molecular empirical methods for adjusting population size during a evolution in human genomes, Hawks et al. [73] suggest that run have been proposed, such as the Genetic Algorithm with if a population is highly adapted to a current environment, Variable Population Size (GAVaPS) proposed by Arabas et evolution will become stagnant. Under these circumstances a al. [124], the parameter-less GA by Harik and Lobo [125], growing population size can provide the potential for rapid the Adaptive Population size Genetic Algorithm (APGA) by adaptive innovation. Thus, enlarging the population size Back et al. [126], and the Population Resizing on Fitness under chaotic environments can promote the rate of adaptive Improvement GA (PRoFIGA) by Eiben et al. [127]. However, evolution. mechanisms for dynamically adjusting population size in Population size is also involved in research on genome EC are much simpler than those found in nature, in that a robustness. Visser et al. [85] postulate that the population fluctuating population size still has little to do with mutation size should be sufficiently large for selection to be effective and selection patterns in different evolutionary stages. This to evolve the robustness of a system. Small populations relation requires further exploration as it seems to be a have difficulty to achieve this robustness. In a different promising indicator for population size adjustment during study Krakauer and Plotkin [86] find, however, that small arun. Journal of Artificial Evolution and Applications 9 3. Variation What triggers mutation and what is the relation between mutation and selection? Does selection pressure indeed Mutation and recombination operators are a main aspect generate new mutations or simply allow existing mutants of evolvability, since they generate the necessary variation to be fixed faster than before? Research on mutation under among individuals that later can be acted on by cumulative selection has received wide interest since Darwin’s time, but selection processes. Due to the complex mapping process controversies have arisen regarding the effect of selection from genotypic to morphological level in biology, genotypic pressure on mutation, and different models have been and phenotypic variation will be discussed separately. proposed in the meantime [129]. It is now believed that it is impossible to separate any form of mutation from the effect of selection. In order to investigate “directed” 3.1. Genotypic Variation. Genotypic variation generally mutation pathways Roth and Andersson [130]define adap- means changes to DNA sequences in both protein-coding tive mutations as fitter mutations that arise under selective and noncoding regions in the form of point mutation and conditions. In subsequent work [131–133], they propose a gene rearrangement. Gene sequences are highly conserved gene duplication-amplification model to study the mutage- against lethal changes that would likely lead to destructive nesis stimulated by enhancement of selection. In addition, consequences otherwise because a tiny mutant at the genetic a recent study by Weinreich et al. [134] on the effects levelcan causeagreatchangeinfunction[22]. In contrast, of Darwinian selection on random mutation argues that changes to the regulatory or noncoding part of sequences are environmentalselection canmakesomemultistep mutation considered more able to increase adaptability and plasticity pathways unaccessible. By studying “five point mutations” of a system. In this section, we will discuss the general form in a lactamase allele that can increase bacterial resistance of mutation first and then gene duplication as the most to an antibiotic, several mutation pathways are in principle important form of rearrangement, followed by a comparison possible for these mutations. After calculating the different between point mutation and gene rearrangement. probabilities of these pathways, their experimental results show that under intramolecular interactions that increase the fitness of proteins, only a small number of pathways are 3.1.1. Mutation. Although there can be many definitions of really accessible. This is quite an interesting result because mutation, we here adopt one that emphasizes the primary mutations might be channeled by some unknown fitness- difference to recombination, namely, that works with mate- increasing principle(s) and the resulting proteins might rial from just one individual organism. be reproducible and even predictable. These feedback and interaction mechanisms may reduce the harm that mutations Definition 5 (mutation). Mutation is the process that creates could bring to an organism. This point of view also conforms new genetic material from the addition or multiplication to Kirschner and Gerhart’s definition of evolvability [47], of stochasticity in various forms to some original genetic which they define as “the ability to reduce the potential material of an organism. deleterious mutations and the ability to reduce the number of mutations needed to produce phenotypically novel traits”. If mutations can be channeled, fewer changes might be needed Point Mutation. Searching for the essential driving force of evolution has been a central topic in evolutionary biology. to generate a required adaptation and, therefore, evolvability Since Darwin declared that natural selection is the main force would be improved by this reduction in cost of mutations. In EC, mutation is regarded as an important exploratory of evolution, controversies have arisen on different aspects of this explanation. In modern biology, the two main schools operator. Artificial evolutionary search should be good at of thought are selectionism and neutralism [128]. Some both exploring suitable genetic novelty and maintaining suc- scientists argue that genotypic variation is maintained by cessive improvements. Holland [7] discusses this principle selection, which is the central perspective of neo-Darwinians. as the tension between “exploration” and “exploitation”. The mutation rate is important to keep this balance, and it has Other evolutionists insist that high genotypic variation can be explained as a result of neutral mutations. In either case, already been studied as an evolvable parameter contributing mutation is accepted as a major mechanism to generate to evolvability. Bedau et al. [51, 135]divideevolutionary adaptation conceptually into two stages: the novelty stage, genotypic variation. Mutation canhappenanywhereonaDNAsequence, where an evolving system enhances its adaptability against that is, in either coding or noncoding regions, and may a changing environment, and the memory stage, where the consequently cause functional, regulatory, or structural evolving system is building up this adaptability through changes, or no changes at all. The neutralist hypothesis is incremental improvements. By providing a simple two- that the majority of observed sequence variation stored in the dimensional model, Bedau et al. postulate that the mutation population is neutral. This is due to the compensating mech- rate should increase during the novelty stage and decrease anisms of biological systems [128]. Most new mutations are during the memory stage. This fluctuation of mutation rates is able to keep the balance between evolutionary novelty deleterious, a few are advantageous, and many are neutral. However, most of the extant polymorphism observed in and memory and thus increases the evolvability of adaptive populations is the neutral variants. Deleterious mutations systems. have been purged and advantageous mutations have swept However, compared to natural evolutionary systems, through the populations. genotypic variation in computational systems is somewhat 10 Journal of Artificial Evolution and Applications different loci, with the consequence that duplicate genes appear in one chromosome while the other turns out to Crossover contain pseudogenes. Retroposition happens when an mRNA is retrotranscribed into a complementary DNA (cDNA) and then inserted into the original genome. Besides such gene duplication, duplication at other scales in cells has been discovered recently [138, 140], including segmental duplication and whole-genome duplication. Here, we (a) only consider gene duplication. The main products of gene duplication are called paralogous genes, a type of homologous genes. Homologous genes have two main categories, paralogs Transcription and RNA splicing mRNA and orthologs. Paralogs are results of gene duplication and code for proteins with different functions. Orthologs are the Reverse transcription products of speciation events and the proteins they code for serve similar functions. cDNA Once a gene duplication has occurred, a complex fixation Randomly insert process on the duplicate genes takes place. Purifying selection into genome and gene conversion are the main pressures affecting the sur- vival of duplicate genes [141]. Most duplicate genes become Parental gene pseudogenes after one or more mutations disable them in different chromosome and no promoting function is yielded. However, multiple (b) copies of identical genes can, after duplication, promote functional redundancy against fatal changes. The process Figure 2: Two common modes of gene duplication. (a) Unequal of pseudogenization is reported to occur in the early stages crossover, which results in a recombination event in which the two of a rapid evolution [142] process, with evidence of many recombining sites lie at nonidentical locations in the two parental pseudogenes found in the human genome. Other duplicate DNA molecules. (b) Retroposition, which occurs when a message genes are changed by selection pressure and functional RNA(mRNA)isretrotranscribedtocomplementary DNA(cDNA) divergence. Subfunctionalization and neofunctionalization and then inserted into the genome. Squares represent exons and bold lines represent introns. (Adapted from Zhang [139].) are the two main mechanisms of functional divergence [139]. In subfunctionalization of two gene duplicates, shown in Figure 3,eachcopyadoptsadifferent aspect of the function arbitrary and not as adaptive. First, the fixation process of the original gene. Both copies will be stably maintained of mutations is not simulated appropriately in most EC because both aspects of the function are indispensable. algorithms, because all changes to individual sequences are Subfunctionalization leads to functional specialization by mostly translated into phenotypic properties. Recovery or dividing multifunctional genes once the newly emerged repair mechanisms are usually not applied to individuals genes perform better. Alternatively, some relatively new suffering deleterious mutations, which make those individ- function can also evolve after gene duplication [143], and this uals unfavored during the selection process. Second, the process is called neofunctionlization.Thishas been termed selection-driven mutation pathways found in natural systems the Dykhuizen-Hartl Effect [144] earlier, where a random are an interesting direction to explore for computational mutation is preserved in the duplicated gene by reducing models and should be considered in future research in EC. selection pressure due to functional redundancy that results from gene duplication. Such mutations may accumulate and Gene Duplication. Gene duplication is an important mech- induce a genetic function change depending on conditions anism creating new genes and new genetic subsystems. of the (dynamic) environment. New adaptive functions may This mechanism has been recognized to generate abundant thus be generated and later preserved during evolution. By genetic material and contributes substantially to biological possibly creating novel functions and allowing evolution under fewer constraints, neofunctionlization is an important evolution [136]. A large number of duplicate genes have been discovered to exist in vertebrate genomes [137], and consequence of gene duplication. a repeated number of whole genome duplications have In brief, the mechanism of gene duplication contributes been established as key events in evolutionary history [138]. substantially to genomic and organismal evolution. It pro- In modern biology, gene duplication and its subsequent vides abundant material for mutation and selection and function-specialized divergence are widely believed to be a allows to specialize function or generate completely new major reason for functional novelty. functions. The acceleration of protein sequence evolution Gene duplication is usually generated by unequal after gene duplication has recently been confirmed in research on yeast genes by Gu et al. [72]. The authors crossover or retroposition [139] (see Figure 2). Unequal crossover is similar to but different from normal crossover use an additive expression distance between duplicate genes that occurs when two chromosomes exchange a propor- to measure the rate of expression divergence, and rapid tion of DNA at the same locus in base pair sequences. evolution of gene expression as well as regulatory divergence Unequal crossover happens if this exchange occurs in after gene duplication is observed. Journal of Artificial Evolution and Applications 11 A1 A2 evolvability, possibly even more than simple point mutations Expressed in T1 and T2 [153]. Recent development of technology has now facilitated the shift in focus from a locus-based analysis to a genome- wide assessment of genotypic variation [79, 154]. Gene duplication Genetic rearrangements rather than point mutations can maintain the connective information carried by gene sequences. Because genes form networks of functional control, rearrangement is better able to preserve internal Complementary structures. Genetic changes are highly constrained by gene degenerate mutations sequences and gene rearrangements occur far more fre- quently than point mutations. The ubiquity of Copy Number Variations (CNVs) has been realized recently in mammalian genomes by different Expressed in T1 Expressed in T2 groups of biologists, such as Sebat et al. [81], Iafrate et al. [80], and Tuzun et al. [82]. CNV is regarded as a Figure 3: Division of expression after gene duplication. Squares predominant type of genotypic variation leading to vast represent genes, closed ovals represent cis-acting elements that phenotypic diversity in mammalia. CNVs show that large regulate gene transcription, and open ovals represent deactivated segments of DNA, with sizes from thousand to millions of cis-elements. Consider a gene that is expressed in tissues T1 and T2, base pairs, can vary in copy number of genes. This variation with a cis-acting regulatory element A1 controlling the expression in T1 and A2 controlling the expression in T2. Following gene canleadtoprotein dosage differentiation in the expression duplication, one daughter gene might lose the A1 element whereas of genes, and CNV is therefore regarded as being responsible the other gene might lose A2, so that each is expressed in only one for a significant proportion of phenotypic variation [79]. of the two tissues. (Adapted from Zhang [139].) The mechanisms that create CNV have not yet been clearly understood, but some hypotheses have been proposed in the literature. Fredman et al. [155]and Shaw andLupski[156] propose that CNV might be the result of large segmental gene One key idea how gene duplication can speed up duplications or nonhomologous recombination events. evolution is Altenberg’s constructional selection [145, 146]. Recent bioinformatics research uses statistical and com- The idea is that gene duplication enriches the genome with putational tools to analyze chromosomal evolution by a genes that are good at increasing fitness when duplicated. comparison of genome-rearrangements between sequences This is a second-order effect that can be considered to of related species [157]. Although the biochemical mech- contribute to evolvability. For a more general review, see anisms of gene rearrangement are still far from being [147]. fully understood, we believe it is time to start using such In summary, the mechanism of gene duplication can rearrangement operations in computational models in EC. considerably increase evolvability of a system by reducing the Particularly, the recent discovery of CNVs requires attention cost of mutations. In EC, the idea of using gene duplication by computer scientists, in order to achieve similar benefits in and deletion operators was proposed some time ago. Those EC. operators are in general based on the method of variable- length genotypes and are executed with predefined dupli- 3.1.2. Recombination. Recombination has been considered cation or deletion probability [46, 148–151]. Unfortunately, both as an exploratory and as a stabilizing operation in so far only application-oriented work has appeared with biology and in EC. Here we emphasize the origin of the different representations [152], and a common framework genetic material being used for new combinations. Due to for this concept is missing. More details of gene duplication the size of search spaces, both effects are possible. in biology should be taken into account to benefit compu- tational evolution. In particular, the question of how gene Definition 6 (recombination). Recombination is a process duplication reduces the limitations of mutation and selection that generates combinations of existing genetic material from and in the process promotes evolvability needs to be studied. a multitude of organismic sources. Is there a way to implement functional specialization and innovation through gene duplication in EC? Recombination is regarded as an important force shaping genomes and phenotypes. Since some highly efficient and Point Mutation versus Gene Rearrangement. A point muta- accurate computational methods can be used in biology, tion occurs when a base on a DNA sequence is changed into analysis of gene recombination has made much progress by another base at the same locus. Gene rearrangement is a way of comparing aligned genome sequences. These com- change in the order of a DNA sequence on a chromosome. parisons facilitate a better understanding of several aspects This change can be an inversion, translocation, addition, or of genetic and evolutionary biology, notably genotypic and deletion of genes. Earlier research focused mostly on Single phenotypic variation and genome structures [158]. Nucleotide Polymorphisms (SNPs) in genomes due to the Recombination exchanges genetic material between two enormous complexity of genetic sequence analysis, but gene DNA sequences swapping strands between one or multiple rearrangements have always been believed to contribute to crossover points. Recombination can occur on homologous 12 Journal of Artificial Evolution and Applications or nonhomologous sequences. The former is more promi- value. Different from natural recombination mechanisms, nent in research because it is more common and efficient most adaptive recombination rate proposals simply react to in generating adaptation in nature. Generally, research on the current status of the search, in order to escape from recombination focuses on prevalent eukaryotic organisms local optima. However, rate adaptation in biology is much rather than prokaryotes, which do not have the sex property. more complex and suggests other models for computation. Unequal crossover is fairly rare and may lead to duplication For instance, the rate may vary among different individ- or loss of some genes (discussed in Section 3.1.1) and other uals or in different modules serving subfunctions in the results [159]. Combination events can take place between genome. Such function-specific recombination rates could different gene sequences, as in intergenic recombination,or also consider the method of “compartmentalization” for between alleles on the same gene sequence, as in intragenic modularity (Section 3.2.2). The notion of epistatic clustering recombination [158]. Despite various forms of recombina- in contributing to evolution of evolvability has recently tion, their outcome is crossover at one or multiple points and been studied [170]. Genetic linkage patterns between dif- a swapping of fragments of genetic sequences. ferent loci are claimed to affect recombination rates, and Kondrashov [160]proposesinhis deterministic mutation the simultaneous optimization of different recombination hypothesis that sexual recombination can remove deleterious rates on different traits would be realized by a method genes. It is generally believed that most nonneutral muta- called epistatic clustering. Evolvability would be improved tions are slightly deleterious. Kondrashov suggests that sexual through coevolution of trait clustering and recombination recombination can distinguish individuals with cumulative, mechanisms. slightly deleterious mutations, and the ensuing selection pressure can eventually remove those disadvantaged muta- 3.2. Phenotypic Variation. As mentioned in Section 2.1.2, tions. Further, Hadany and Beker [161] strengthen this despite their vast phenotypic differences, metabolic processes perspective in their research on the evolution of obligatory and cell structures in bacteria and humans are quite similar sex. Their model supports that sexual recombination offers [22]. What, then, makes humans so different morpho- both short-term and long-term advantages to sexually logically from other organisms? It is the regulation and reproducing individuals and has a positive effect on the reuse of these structural elements in different combina- physiological fitness of an organism. tions that generates different complex phenotypic outcomes. The rate of recombination can significantly affect the rate Unfortunately, the relation between genotypic variation and of adaptation. It is usually higher than the rate of mutation, phenotypic variation is still not fully clarified in current which implies that recombination introduces much less biological opinion. Since selection acts on phenotypes rather lethality to an evolutionary population than mutation. than on genotypes, phenotypic variation should be used to Instead, it advances evolution remarkably by stabilizing explain the immense diversity among organisms. Here, we adaptive traits from parents to offspring. This contributes discuss several aspects of phenotypic variation. We leave the to evolvability in the same way as other purifying selection discussion of the mapping process between genotype and does because the bounds on epistatic interactions between phenotype that controls the direction of phenotypic changes loci get progressively strengthened through selection over resulting from genotypic variation to Section 3.3.1. generations. By drawing a recombination map of the human genome, Kong et al. [162] discovered that recombination 3.2.1. Conservation and Relaxation. According to Kirschner rates vary in different regions of the genome. This variation and Gerhart, evolution possesses two important features: is duetosuchfunctionalfeaturesasgenedensity,other conservation at the molecular level and relaxation at the gene properties, and frequency of sequence repetitions. anatomical and physiological level [22]. By conservation it Recombination rates are also different in autosomes between is meant that the genetic components of organisms tend to different sexes. Recombination contributes to producing maintain relatively stable structures; relaxation refers to the both genotypic and phenotypic variation and is able to less constrained phenotypic diversification of organisms. The repair DNA double strand breaks. Sexual reproduction is an authors state that conservation on the genotypic level reduces important outcome of recombination. the constraints on the phenotypic level. In EC, recombination operations are considered an In Darwin’s evolutionary theory, all organisms have essential search strategy. Chromosome coding is much more evolved from the same ancestor. After primal initialization flexible in computation than in nature, and thus, various and evolution, genetic structures of organisms are highly recombination techniques have been proposed and studied, conserved during the course of billions of years [101]. including double-parent and multiparent crossover [163], This can well explain why the number of human genes is fixed-length chromosome and variable-length chromosome only a few times that of bacterial genomes but significant crossover [164, 165], and homologous and nonhomologous anatomical and behavioral differences exist between them. crossover [166–168]. High recombination rates are usually The surprisingly small number of genes in humans and other also adopted in computation because of its perceived effi- complex organisms demonstrates that the great diversity ciency in generating beneficial genetic and phenotypic varia- and complexity at the anatomical and physiological levels tion. Elsewhere, adaptive recombination rates are proposed have to rely on and organize/reuse limited genetic material. to strike a balance between exploration and exploitation When certain organisms need to improve their adaptivity [169]. In most of these adaptive recombination rate schemes, in order to survive in a new environment, the regulation modification of recombination rates is based on fitness system only has to recombine existing mechanisms for the Journal of Artificial Evolution and Applications 13 generation of adaptive functions, which requires little or Definition 7 (modularity). In a complex system, modularity no new genetic material [47]. Not only are gene sequences refers to the property that a loose horizontal coupling exits highly conserved, but also the core processes of coordination between the entities at the same level of this system [172]. of the genetic material are well conserved since the time they Simon [71, 173] further defines that “a system is nearly initially emerged [22]. These conserved core processes are decomposable if it consists of a hierarchy of components, such used repeatedly for different purposes and functions under that, at any level of the hierarchy, the rate of interaction different circumstances, at different times, with different within components at that level is much higher than the rates genetic material. The Baldwin Effect [171] explains that of interactions between different components”. Although this phenotypic variation is not generated out of the blue but “Near Decomposability (ND)” is attributed to a vertical through regulation of existing components in organisms: separation while modularity describes the separable property mutation simply stabilizes and extends what has already of components horizontally at the same level, they seem existed to improve somatic adaptability towards external closely related in that they both describe how a complex stimulations. system is decomposed into subsystems. This conservation mechanism can efficiently prevent Themodularitypropertyofgenotype-phenotypemap- lethal changes in genotypes and is an economic method pings has been extensively studied in gene expression. It to increase the adaptability of organisms. New material is reduces harmful pleiotropic effects of gene expression and not needed to adapt to changing environments, but few can lead to adaptive phenotypic variation. Pleiotropy is a modifications will suffice. general property of genotypic variation, expressing the fact Functional innovation is heavily constrained due to that one change at the genetic level can cause a multitude molecular interactions among various genetic components of functional changes at the phenotypic level. Pleiotropy that are involved to produce a specific trait. If the partic- can generate both advantageous and disadvantageous results. ipation of more genetic components is needed, it becomes Pleiotropy can sometimes generate unexpectedly improved harder for functions to change. In fact, relatively little genetic functions but can also be harmful or even fatal to evolution- material is required to generate all proteins of organisms. ary systems [174]. Since a gene can affect multiple functions, Under selection pressure from a changing environment, optimizing one particular function at the phenotypic level organisms have to yield adaptive phenotypic traits to survive, inevitably incurs side-effects on other functions. Bonner however, and the highly conserved core processes mentioned [175] proposes the notion of “gene nets” by grouping above are used repeatedly to generate new cooperation gene actions and their products into discrete units during among the conserved genetic material, bringing about fitter evolution. In general, for a given organism, the mapping function and behavior. Relying more on the combinatorics of from genotype to phenotype can be divided into modules components is equivalent to relaxing phenotypic variation. such that the sets of genes in one module only affect the The relaxation on phenotypic variation has been high- functions in that same module. The mapping is therefore lighted as the notion of “deconstraint” in Kirschner and decomposed into groups of independent “submappings”. Gerhart’s [47] research on evolvability which studies the Bonner finds that the phenomenon of gene nets becomes mapping from genotype to phenotype. Enhancing pheno- increasingly prevalent as organisms become more com- typic variability under changing environmental conditions plex. Wagner and Altenberg [45]investigate modularity allows nature more evolvability. Not only can deleterious in genotype-phenotype mappings from both perspectives, changes be avoided, but also nonlethal genetic and pheno- biology and EC. They interpret modularity as a means for typic variation is indeed the material from which innovation dividing phenotypic traits into different “compartments” to can be generated. reduce interference among different optimization modules. Turning again to EC: what is the role of conservation With such modularity, optimization of a function in one and relaxation in EC? First, an economic use of genomes module has no effect on functions in other modules. As a or building blocks can help to conserve genetic information. result, pleiotropy can be confined to a known set of functions Second, it can be assumed that by reducing the constraints during evolution. Figure 4 shows a simple example of this on changes to a phenotype the exploratory capability of idea of modular separation. a computational system to find better solutions can be Wagner and Altenberg [45] further propose that mod- enhanced. How such a process can be implemented in actual ularity results from evolutionary modifications in natural systems is presently unknown, but a worthwhile line of organisms. In their view, the evolution of modularity follows inquiry. two mechanisms, dissociation and integration. Dissociation is the suppression of pleiotropic effects by disconnecting 3.2.2. Modularity. Modularity is a widespread structural interactions between different modules, while integration is property of complex systems. It has attracted considerable realized by strengthening of pleiotropic connections among interest from studies of both natural and artificial evolution- traits in the same modules. Both mechanisms are driven by ary systems and is regarded as strongly related to evolvability selection pressure. [45] and the acceleration of evolution [71]. Modularity exists at various levels, for example, at the Thus, modularity can be conceptualized as an evolu- level of gene expression or embryonic development. Here we tionary mechanism to promote evolvability. It reduces the adopt the definition of modularity proposed by Simon [172] interdependence of disjoint components and consequently in his research on hierarchies in complex systems. reduces the chance of pleiotropic damage by mutation [47]. 14 Journal of Artificial Evolution and Applications F1 F2 and reusing relatively simple genotypic material will be a major force in shaping complex phenotypes also in EC. C1 C2 3.2.3. Facilitated Variation. Kirschner and Gerhart [22] BC emphasize that variation is much less random at the phenotypic level of organisms than at the genotypic level, where genetic mutations show considerable randomness. Since phenotypic variation should be favored by selection via modifying existing evolutionary components, they call this variation facilitated. Kirschner and Gerhart summarize three principles of facilitated variation. It serves (i) to reduce lethal pleiotropic effects, (ii) to increase phenotypic variation in light of G1 G2 G3 G4 G5 G6 a given number of genetic changes, and (iii) to improve genetic diversity in evolutionary populations (by reducing lethality). Evolution is not so much affected by the content of Figure 4: Example of a modular representation. Complexes C1 = genetic and protein structures but by regulation capabilities {A, B, C, D} and C2 = {E, F, G} serve to functions F1 and F2. to organize and reuse these functional parts and to decide Each character complex has a primary function, F1 for C1 and F2 the targets of such regulation. The core processes instead for C2. Only weak influences exist of C1 on F2 and vice versa. The are conserved being built in a special way, only to be linked genetic representation is modular because the pleiotropic effects of together under new circumstances like time, place, and the genes M1 = {G1, G2, G3} have primarily pleiotropic effects on the number of genetic components that may participate in the characters in C1 and M2 = {G4, G5, G6} on the characters in generating new phenotypic variation. It is clear that only complex C2. There are more pleiotropic effects on the characters adaptive phenotypic variation can be maintained during within each complex than between them. (Adapted from Wagner evolution, and the relevant product proteins mostly will have and Altenberg [45].) multiple functions for various adaptive requirements under selection. Variation in EC systems seems to be more random than It allows genotypic variation and selection to affect separate that in natural evolution. Despite the limitations in recogniz- features in a complex system and to evolve various functions ing these phenomena in biology, we should explore methods without interference [176]. Subsystems as part of an entire to reduce randomness in computational models in order to system can evolve faster to optimize their local subfunc- make evolving processes more “intelligent” and to facilitate tions individually, by decreasing crosstalk between genetic the discovery of good solutions. Some steps have been taken changes. In a study of encoding schemes in EC by Kazadi in EC literature in this light. Researchers have designed et al. [177], a compartment is defined similar to a module more sophisticated techniques to improve the adaptation in the genotype-phenotype mapping, and such compart- of algorithms. One idea was developed for Evolutionary mentalization at different levels is claimed to contribute to Strategies first and later applied to other branches in EC the acceleration of evolution. In RNA research, Manrubia [179–182]. Further, the evolution of “smarter” operators for and Briones [178] propose that the increase of molecule EC in a higher-level evolutionary process has been examined length and subsequent increase in functional complexity in metaevolution [183–186]. Amorerecentcontribution could be mediated by modular evolution. They find that looking at the effect of changing environments on variation short replicating RNA sequences with a small population in a computational framework has been the GA of Parter size can be assembled in a modular way and can create et al. [187]. complex multifunctional molecules faster than conventional evolution of complex individuals toward multiple optima. Modularity in general has been widely used in computer 3.3. Transformation from Genotype to Phenotype. It is at science and engineering by subdividing complex entities into the intersection between genotypes and phenotypes where smaller components to yield higher computational efficiency most of the mechanisms reside that allow for facilitated and has similarly played a key role in EC from the outset. variation. A subject of much study both in natural and In fact, it can be argued that the building block hypothesis artificial systems has been the genotype-phenotype map. is at its core an argument about modularity. However, In recent years, epigenetic effects, long suspected to have complex genotype-phenotype maps and other mechanisms enormous influence on the final expression of the phenotype, (like growth and development) to generate modularity in have assumed center stage in biology. Epigenetics [188]is EC are relative newcomers and it is expected that studying a rapidly developing and prominent research topic, both these mechanisms in biology will result in more sophisticated in relation to the development of healthy phenotypes as means to produce modularity in EC. Since modularity is the well as those who show deficiencies. This will constitute most universal property of phenotypes in natural systems, the second part of this section. Finally, epigenetics and a there is ample ground to expect that the economic and consequence of the amplified power of expression regulation sophisticated mechanisms used by Nature through regulating through epigenetics are the mechanisms of development Journal of Artificial Evolution and Applications 15 of multicellular organisms. These are discussed in the last Duplication Transcription Translation DNA RNA Protein subsection here, concluding the transformation of genotypes into phenotypes. Figure 5: Central Dogma. The Central Dogma of biology by Crick holds that genetic information normally flows from DNA to RNA 3.3.1. Genotype-Phenotype Mapping. In EC, mapping from to protein, which involves the mechanisms of gene replication, genotype to phenotype is often an encoding process, espe- transcription and translate. cially in evolutionary algorithms and evolutionary strategies, where the mapping mechanism is used in most cases to directly calculate a fitness function of an individual. How- ever, in nature, the mapping process is much more complex, living systems have evolved for billions of years, regulatory typically from highly conserved genotypic information to core processes in various organisms have remained mostly greatly divergent polymorphism in phenotypes. The funda- unchanged despite species divergence. By comparison of mental process in biological genotype-phenotype mapping related species from the same ancestors, such as humans is gene expression, and the most important mechanism and chimpanzees, at both the molecular and organismal in this process is regulation of gene expression, which levels King and Wilson [153] had already found in 1975 will be discussed next. Since research on transcriptional that genetic structures in these two species are almost regulation has discovered increasing evidence that RNA plays the same while at the organismal level, the anatomy, an important role in gene expression, the transcriptome, physiology, behavior and ecology of these two species are that is, the set of all transcribed RNAs, will be reviewed significantly different. This suggested to them that the then. complex adaptive evolution is produced by a combination and multiple utilization of similar, highly conserved Regulation of Gene Expression. In biology, the core processes genetic components under the control of regulatory (Section 3.2.1) of organisms are responsible for generating systems. anatomy and behavior using genetic and cell materials. These A key step in the regulation of gene expression is core processes include metabolism, gene expression, and transcription. Studies there are concentrated on two primary interaction among molecules and cells [22], which are well components: promotors and transcription factors.Promotors, conserved but still under exploration. Regulation of gene also known as cis-regulatory sequences, are responsible expression is the most important mechanism among the for regulatory transcription. Cis-regulatory sequences are core processes to facilitate organismal novelties in evolution. noncoding DNA sequences which determine when and Kirschner and Gerhart highlight the characteristics of “con- where “their” genes are transcribed by regulating access of servation” and “economy” in regulatory core processes in polymerase to transcription start sites. Transcription factors [22]. are proteins interacting with these cis-regulatory sequences Scientists have been trying to understand the process of by binding to certain sites on DNA sequences. Readers gene expression for decades. In 1956, Crick proposed the interested in more details are referred to Wray et al. [77]. Central Dogma of molecular biology, as shown in Figure 5, Transcription factors act either as activators or as repressors which describes the transmission of genetic information of gene expression. For example, if a transcription factor from DNA to protein. The circular arrow around DNA A binds to a site on a DNA sequence that is responsible symbolizes that a DNA is a template for self-replication. for generating protein B, then this factor A is regarded as The arrow from DNA to RNA indicates that an RNA is a repressor to protein B. In addition, as a protein itself, transcribed on a DNA template, and the arrow from RNA factor A also has its template gene sequence. If another to protein signifies that a protein is translated on an RNA transcription factor C can bind to this site and represses the template. generation of protein A, C acts as a repressor to A but in Subsequent biological research revealed that the process turn as an activator to the expression of protein B. These of gene expression is much more complex than such a activators and repressors can work together as a network linear flow and involves a considerable number of complex of logic control. Promotors usually contain a number of regulation operations. The Central Dogma was challenged binding sites for transcription factors, where each site can by discoveries of proteins playing an important role in only be occupied by one factor at a time. These binding regulation of gene expression and, most recently, the non- sites occupy, however, only a small fraction of sequences coding RNA control of chromosome architecture proposed and are distributed unevenly. Some binding sites of different by Mattick [189]. In this section, we concentrate on gene functions can overlap. Furthermore, binding affinities of expression regulation by proteins and will discuss RNA different materials are important for regulation as well. On effects in next section. the other hand, most transcription factors have numerous Recall the discussion of genome redundancy in target genes and use priorities in binding with any of Section 2.1.1. Coding regions on genetic sequences that them [77]. This sophisticated network endows the regulation can be expressed into proteins only occupy a small portion system with high robustness and plasticity necessary for of the entire genome in eukaryotic cells. This discovery evolution of capabilities of organisms. indicated that a huge number of regulatory elements exist Kauffman [190, 191] holds a long standing opinion in genomes that participate in generating adaptation in that gene regulation networks are dynamical systems and evolution according to changes in environments. Although that many phenotypic traits are encoded in the dynamical 16 Journal of Artificial Evolution and Applications attractors of these systems. Dynamical attractors refer to Duplication Transcription Translation DNA RNA Protein cyclic trajectories of the transformations of states of these networks and their study provides clues to the behavior and properties of gene regulatory networks. Kauffman’s point of view—namely, that the topology of a gene regulatory net- Figure 6: Eukaryotic genetic system. Expression of genes is not with an irreversible linear flow in eukaryotes, but involves frequent work largely decides cell types, cell fates, and functional states feedback and interactions among different molecules including of the cell—has been supported by a number of more recent DNA, RNA, and protein, as the dotted lines shown here. studies [192–195]. Meanwhile, simplifying computational models has been proposed to study dynamical attractors. Aldana et al. [196] model gene activities using random Boolean networks (RBN) with varying topologies. They work on evolvability and dynamics in artificial regulatory report that a network with scale-free output topology and networks is necessary. operating close to the critical regime (neither ordered nor chaotic) possesses the greatest robustness and evolvability compared to networks with other topologies and acting in The Transcriptome. The transcriptome, or collection of tran- different dynamical regimes. Further support comes from scripts, refers to all RNAs produced in a single or a group of [197–201], which again confirms Wagner’s argument that cells, working as an intermediate component of gene expres- high robustness and high evolvability can coexist in natural sion. In high-level eukaryotes such as humans, most regions systems (see Section 2.1.1). of the transcriptome are not translated into protein. What Evolution of cis-regulatory sequences as noncoding necessitates the existence of such a large number of RNAs sequences is considerably different from that of protein- in the transcriptome of high-level eukaryotes? Regulatory coding sequences and is less understood. King and Wilson function is one answer to this question. Although regulation [153] suggest that protein-coding sequences are highly of gene expression starts with the transcription step, these conserved during evolution since they were synthesized. It transcribed but nontranslated sequences or noncoding RNA is mutations on promoters that causes most morphological sequences act as regulators for translation in gene expression variation. Research on the evolution of transcriptional and currently attract increasing interest in biological research regulation has become mainstream in molecular biology [83, 212]. in recent years [77]. In particular, Roderiguez-Trelles et al. An RNA is not just a temporary medium between genes [75] find that significant substitution rate differences exist and proteins as described in the Central Dogma. In high- among different promotors, and even some neighboring level eukaryotes, the information transmission from DNA cis-regulatory promotors involved in the same regulatory to protein is not a one-way process but involves many func- network can have different evolution rates. Moreover, Stone tionalities of the transcriptome. The new perspective of gene and Wray [202] propose that local point mutations on expression proposed by Mattick [189, 212] can be described binding sites can lead to rapid evolution in gene expression, in Figure 6. Compared to a prokaryotic genetic system, an which indicates their potential of accelerating evolution. eukaryotic system has a parallel control mechanism with Wagner [203] points out that other simple changes such multiple outputs and information transfers. Rather than a as gene duplication and deletion of promotors can also simple medium of gene expression, RNA metabolism and result in rapid evolution in gene regulatory networks. interaction have been discovered playing an important role By comparing genomes, Fondon and Garner [204] dis- in gene expression regulation. cover that gene-associated tandem repeat expansions and Mattick [84] proposes that noncoding RNAs participate contractions exist and give rise to rapid morphological extensively in gene expression regulation, being present in evolution. In their experimental research, a tandem repeat about 98% of all transcriptional outputs in eukaryotes. In mutation shows both elevated purity and intensive length research on the human transcriptome, Frith et al. [83]find polymorphism among different dog breeds. Mutations on that noncoding RNAs play an important role in generating noncoding sequences can modify regulation of the target phenotypic variation. Noncoding RNAs can be classified into genes, the length of coding loci to transcribe, and the two categories: introns and other noncoding RNAs. occurrence conditions. Furthermore, they also result in Regulation of the transcriptome shows contributions morphological variation and accelerated phenotypic evolu- to evolvability and rapid evolution. Introns, an important tion. category of noncoding RNAs, are found more susceptible Since the mechanisms of regulation of gene expression to mutations than their neighboring protein-coding exons. can well explain many phenomena in evolvability and rapid Rather than having no function, as thought previously, it evolution in living systems, research on artificial regulatory was found that introns do have influence on regulation networks has now started in computer science. Several (see, e.g., [213]). The fewer constraints imposed on introns models of artificial evolution regulatory networks have been by selection offer flexibility to generate new functions and proposed such as Banzhaf et al. [205–208], Chavoya and rapid protein sequence evolution during the process of Duthen [209], Mattiussi and Floreano [210], and Nehaniv regulation, especially in connection with alternative splicing. [211]. These artificial models intend to generate regulatory The evolution of RNA communication networks may also behavior akin to that of natural systems. However, these accelerate the evolution of gene expression, as observed research efforts are still in their early stages, and more by Mattick [84]. These RNA communication networks, Journal of Artificial Evolution and Applications 17 Euchromatin Heterochromatin The main mechanisms of epigenetic control are DNA methylation and histone modification [215]. Modifications Me Ac Ac Me P Me P Me Me Me to chromatin, either on the DNA sequence itself (DNA Ac methylation) or on its surrounding proteins (histone modi- fication), affect gene expression and can be inherited from cell generation to cell generation during cell division. DNA methylation is a chemical addition to DNA sequences. Genes (a) (b) with methyl marks are repressed in expression, despite their unchanged DNA content [219]. In histone modification, the Figure 7: Euchromatin and Heterochromatin. Histone tails have tails of histone proteins are modified by different molecular three types of modification including acetylation (Ac), phosphory- attachments, for examole, acetyl, phosphoryl, and methyl lation (p), and methylation (Me). Euchromatin (a) is the loosely groups (see Figure 7). If acetyl groups are attached to the packed state that most histone tails are attached by acetyl groups. histone tails of a chromatin, it will be loosely packed, a Heterochromatin (b) is the tightly packed state that most histone state called euchromatin. In euchromatin, DNA is readable tails are attached by methyl groups. (Adapted from Jenuwein and and can be transcribed into RNA and later translated Allis [218].) into proteins. In contrast, if methyl groups are attached to histone tails, chromatin is tightly compressed, a state called heterochromatin. In the heterochromatin state, genes are inaccessible to the transcriptional machinery such as which describe interaction among different layers of RNA RNA polymerase or to transcription factors, and genes are signaling, provide a sophisticated regulatory architecture, prevented from being transcribed [220]. Other mechanisms enabling DNA-DNA, DNA-RNA, or RNA-RNA communi- recognized to be responsible for epigenetic regulation of cation, DNA methylation, chromatin generation, and RNA gene expression include chromatin remodeling, histone translation. variant composition, and noncoding RNA regulation. A Compared to natural systems, the genotype-phenotype discussion of these mechanisms can be found by Allis et al. mapping in EC is rather primitive still and a transcrip- [188]. tome is mostly missing in algorithms. The complex RNA The key feature of epigenetic mechanisms is their ability parallel information transfer framework inspires various to coordinate internal and environmental signals which applications. Based on what computational models have can collaborate to modify protein production [215]. The already achieved with artificial regulatory networks, more underlying interactions involve various molecules, such as mechanisms should be implemented, especially the newly DNA, RNA, and proteins, but the extensive feedback between discovered powerful mechanisms of transcriptome regula- these molecules is still beyond our current understanding. tion (see a step in this direction here [214]). We believe that epigenetics opens up a new field in evolvability studies for both biology and EC. Sophisticated 3.3.2. Epigenetic Mechanism. Epigenetics has become a new epigenetic feedback networks suggest a new structure for EC compared to the linear flow of computation usually research direction in evolutionary biology [21]. Literally, “epi”-genetic control lies in the regulation of gene expression employed in the literature. For instance, in dynamic opti- without changing the DNA sequence itself; so it is “beyond mization problems, not all genes responsible for different subfunctions need to be expressed all the time. We anticipate the conventional genetic” control. Epigenetic regulation arises during the processes of organism development and cell that a “controller switch” can be integrated into the genotype proliferation, triggered by intrinsic signals or environmental allowing short-term changes, where fragments of the genome stimulations [215]. Epigenetic changes are heritable in the can be turned on and off in response to external feedback. short term from cell generation to cell generation, and Such a mechanism for repression of expression has barely been used in computation. Similar multilayer adaptive these stable alterations do not involve mutations on DNA sequences. Epigenetic regulation of DNA expression lies at encoding schemes have been proposed, for example, the the heart of many complex and long-term human diseases messy Genetic Algorithm (mGA) [164] that combines short building blocks to form variable-length chromosomes to [216]. Previous research in genetics mostly focused on the increasingly cover all features of a problem, or diploid Genetic sequential information carried by DNA. However, DNA Algorithm,for example, [221] using a two-chromosome sequences are coiled up in cells in intimate complexes with representation to adapt phenotypic variation in dynamic the help of so-called histone proteins. A DNA sequence environments. However, existing work has not embedded the wrapped with histones comprises a nucleosome. Chromatin organizational epigenetic control in algorithms that would is the complex of nucleosomes in the nucleus of cells which allow significant flexibility in changing environments. We participates in the control process of gene expression. The anticipate that epigenetic mechanisms will play a crucial role in increasing the evolvability of EC algorithms. chromatin composition varies according to cell type and response to internal and external signals. The different composition of chromatin may affect expression and thus 3.3.3. Development. Evolutionary developmental biology change the produced proteins even in the absences of DNA with the subject of the relation between evolution and devel- sequence modification [217]. opment, nick-named evo-devo, has arisen as a productive 18 Journal of Artificial Evolution and Applications research direction which tries to unify concepts that have secondly at benefiting a population’s ability to diversify and been separated for a long number of years. The develop- persist. mental viewpoint provides crucial clues to many puzzles and The importance of the developmental point of view controversies that have arisen in genetics and evolutionary seems to be partially realized by the EC community. A biology in the past [222]. Vice versa, evolution is key to new area named generative and developmental systems has understanding the developmental mechanisms that have emerged and attracted studies. Artificial or computational shaped multicellular life [223]. embryogeny was first introduced to simulate the development process in silico (see, e.g., [232–234]). More recently, inspired Definition 8 (development). Development is the process by the complex mapping from genotype to phenotype, by which a multicellular organism unfolds its phenotype, computer scientists have started to allow more freedom and starting from a fertilized single-cell stadium (the zygote), to scalability when representing individuals, a topic known a mature multicellular stadium through a defined sequence as indirect encoding. With an indirect encoding scheme, a of stages that are under the control of its genome and heavily genotype does not map directly to units of structure in its influenced by its environment. phenotype, but a growth or developmental process is allowed in this mapping [235]. Various encoding methods have West-Eberhard [224] discusses the relation of develop- been proposed using, for example, hierarchical grammars ment and evolution and suggests that it is important to [236, 237], or simulating cell chemical processes [238, 239]. reexamine major themes of evolutionary biology in the Indirect encoding schemes have shown advantages over light of development. Molecular biology has extensively traditional one-to-one direct encodings [240, 241]. Indirect investigated evolution on the genotypic level, studying the encoding is a first step to simulate biological development mechanisms of gene expression and protein formation, in computational systems by allowing more freedom and the effect of mutations on genes, and other questions. It complexity in the genotype-phenotype mapping, but it is is, however, development which produces the multicellular by no means the full story of development. As evolutionary phenotypes and their variation that ultimately is screened by developmental biology continues to produce new insights, selection. So in order to examine the effectofamutationon it will be imperative for the EC community to increase its the evolution of multicellular organisms, one has to look at efforts to design new algorithms that are inspired by evo- the effects of this mutation in development. Development devo. emphasizes the time-dimension of an organism and the continuity of phenotypic changes in its interaction with the environment. 4. Selection A major focus of the field of evo-devo is to study the Although Darwin’s theory of evolution being directed pri- role of phenotypic plasticity,or developmental plasticity in marily by natural selection has been the subject of much evolution [225–228]. Phenotypic plasticity is the phenotypic argument, selection is an extremely important operation responsiveness of an organism to environmental input, and it to stabilize the functional traits already generated by some is the most universal property of the phenotype of organisms. exploratory operations [128]. Selection mechanisms are Organisms can alter their form, status, behavior, movement, divided into two types by their effects on different stages or other features in response to environmental stimuli. These of evolution. First, positive selection enhances the fixation changes mostly will not involve any modifications of their of advantageous alleles thus improving the diversity in early genome. This flexibility is a result of the development pro- stages of evolution [139, 143]. Second, negative selection, cess, with a complex mapping from genotype to phenotype. also known as stabilizing selection or purifying selection, The effect of phenotypic plasticity on the rate of evo- occurs at later stages of evolution when genetic diversity lution is a subject of debate [229]. It either can accelerate decreases when such selection eliminates deleterious alleles evolution since new and adaptive alternative phenotypes and only stabilizes specific traits [141]. The balance between are generated to match the current environment or can selection and diversity of an evolutionary population has also be considered to delay the rate of genetic changes been a critical problem, and the dynamic pressure and some since this flexibility is able to provide adaptation to an consequences of selection are still under active investigation. organism without the need to modify its genotype. The In general, selection pressure is produced by two factors, the role of phenotypic plasticity in evolution depends on which environment and mating competition, both of which will be level of evolution is studied, and under what conditions discussed next. [227]. It is, however, clear that both major properties of the phe- notype, its plasticity and its modularity (see Section 3.2.2), 4.1. Environmental Selection. Environmental selection orig- are the result of the hierarchical organization of the inates in the external surroundings and enforces the adap- development process producing the phenotype from the tivity of organisms to survive. Since Darwin environmental genotype. These characteristics both contribute a great deal selection has received extensive attention in evolutionary to the evolvability of living systems [230]. Kirschner and biology. Natural selection is an extremely important driving Gerhart [47, 231] hold the opinion that plasticity and mod- force for adaptive evolution in natural populations [242]. ularity contribute to evolvability due to their advantages, The first response of a living organisms to a changing first at providing adaptation at the individual level, and environment is somatic adaptation. A simple example of Journal of Artificial Evolution and Applications 19 somatic adaptation is human temperature compensation pressure is increasingly strong when the environment [22]. When the external temperature increases above normal, becomes uncertain. Dramatic environmental changes lead to humans will sweat to adapt to this new environment. selection for better evolvability. They consider evolvability Shivering will occur if temperature falls below a normal as a selectable trait, and facilitating environmental changes value. Somatic adaptation happens directly as an organismal can be a method to accelerate evolution. A recent simulation reaction to a changing environment and is not fixed in by Kashtan et al. [74] in a biologically realistic setting also morphological structures as an evolutionary change unless suggests that varying environments may accelerate natural some deeper adaptations caused by somatic changes can evolution. In their work, different scenarios of temporarily increase the survivability of an organism. Organisms have changing optima were used. Kashtan and Alon [246]report plenty of latent traits within their somatic adaptability; that a goal that varies in a modular way can speed up so they have a fairly high tolerance to changes in their evolution. Other work [247] takes into account the effect of environment [22]. Somatic adaptation mechanisms can only the rate of environmental change. By observing the dynamics adjust existing functions to external changes. However, if of adaptive walks under scenarios of varying speeds, they evolution acts for a long time under environmental selection, find that environments with varying rates of change have changes may be stabilized by mutations in the germ-line after noticeably different effects on the fixation of beneficial somatic adaptation has tested them through promotion of mutations, the substitution time required, and the final survivability of the organism. phenotypic variation. Selection can act at different levels depending on its In EC, selection strategies are considered affecting search targets [47]. These might be individual selection, individual- capability significantly during an evolutionary process. and-clade selection, or clade selection. At the individual Different selection strategies have been proposed and the level, the selection process has the fewest constraints since dynamics of selection pressure has been studied extensively it directly affects phenotypic function fitness, and fewer [248, 249]. Since the effects of environmental selection on mutational changes are required for a new adaptive trait. An the evolution of evolvability have been recognized, further individual can also interact with others in a clade, such as research on the dynamics of selection is required. Moreover, through recombination, and survive under selective pressure somatic adaptation might be considered when applying as a member in this clade. At the highest level, selection selection. Group-based selection methods should also be can happen on the level of an entire clade given large studied for varying selection pressure, so that a balance environmental impact, and the entire clade can, as a whole, between the development of a minority and of the entire escape from extinction. Some small groups of the lineage population can be dynamically achieved. might go extinct, but the entire line will be able to survive even if it might have to go through population bottlenecks. This idea has drawn more attentions in some subsequent 4.2. Sexual Selection. Sexual selection was proposed by works [243, 244]. Darwin as the pressure away from the possibility of mating Interactions between different species may also cause failure. Two forms of sexual selection pressure are met by environmental changes. Phillips and Shine [245]reportan mature high-level animals: the battle between male individ- interesting phenomenon on species invasion. Toxic cane uals who fight, and the competition through mating choice toads induced morphological changes among a species of made by females. Fisher [105]proposeda runaway process, snakes in Australia. Generally, native natural ecosystems can where a male trait and female preference for it can both be devastated by the invasion of new species. At upon the evolve dramatically over time until finally checked by severe arrival of an invasive species, the number of native organisms counter-selection. In modern biology, scientists pay much may decrease. However, as these native organisms adapt attention to these sex-based competitions that can generate towards the invaders, the impact of the invasion declines and evolve several kinds of traits in high-level organisms. and a new balance is achieved. Morphological changes For instance, Kirkpatrick and Ravigne [250] find that some are fixed subsequently. Complex natural ecosystems possess secondary sexual characteristics among individuals of the communities with highly frequent and dense interactions same sex can trigger rapid speciation. between species as well as between species-specific functional Sexual selection happens at the interspecies level and traits within a species. affects reproductive fitness of individuals. Reproductive Environmental selection is now widely accepted as con- fitness is the probability of successfully generating offspring. tributing significantly to natural evolution, and has entered Sexual selection has two main forms: intrasexual selection the mainstream of studies in evolvability. As a potential to and intersexual selection. Intrasexual selection is known generate adaptation, evolvability is difficult to observe and as the combat between competitive male individuals, and to select for. However, there is increasing research arguing usually occurs in the form of a fight. Intersexual selection that evolvability is selectable and environmental selection is based on the choice made by the opposite sex. Male sec- can improve the evolution of evolvability. In the real world, ondary sexual characteristics and female mating preferences the environment is changing constantly and fixes beneficial can affect each other and evolve cooperatively [251]. This mutations, and there is a growing acceptance that a changing joint selection pressure, combined with natural selection, is environment is a key ingredient to studying evolvability. a powerful force for rapid evolution. Selection pressure is a critical operator to control an evolu- Recent research in biology has connected sexual selection tionary process. Earl and Deem [55] suggest that selection to the acceleration of evolution. Colegrave [252] finds that 20 Journal of Artificial Evolution and Applications the rate of adaptation can be increased by sex mechanisms among evolutionary components, internal or external to because sexual selection allows a rapid adaptive response these organisms during a long, continuing evolutionary under changing conditions by fixing beneficial mutations. process [170]. In reality, the fitness of individuals in a Swanson and Vacquier [253] observe that rapid evolution system can vary a great deal. Moreover, a large-scale quality emerges in reproductive proteins. This rapid evolution is differentiation exists in almost every natural evolutionary forced by three main selective factors: sperm competition, system, and these vastly diverse evolution systems exhibit sexual selection, and sexual conflict. Sperm competition is substantial evolvability. Since selection and evaluation act quite fierce in that each sperm will compete with billions of directly on observable phenotypic functions but evolv- others to fuse with the only egg, and this competition exists ability only provides the potential for better functions, in multiple steps for the sperm. Sexual selection happens selection and evaluation for evolvability are not observable when different eggs have varying affinities for a special allele directly. of a sperm-surface protein, and only the egg with the highest Since EC has been widely applied in many areas of affinity is most likely to bind to this sperm. Sexual conflict industry and academia, fitness evaluation arises as a difficult means that only one egg can be fused with the sperm to avoid problem because it is usually very CPU-intensive. In the polyspermy such that only one embryo is fertilized. These current literature, two main methods of fitness evaluation types of mechanisms add considerable selection pressure to are employed, absolute fitness and relative fitness. Absolute reproductive proteins and thus trigger rapid evolution in fitness of each individual usually refers to its value of a certain regions of these proteins. specified fitness function. Relative fitness compares different The concept of mating choice was already applied in individuals and gives a rank to each individual to produce EC decades ago by Miller [254, 255]. Some coevolutionary a record of winners. This latter method is good at sup- algorithms have been proposed to simulate mechanisms pressing exceptionally good individuals, thus, helping an from sexual selection by constructing subgroups which can evolutionary system to escape from premature convergence. affect each other cooperatively to evolve in parallel. As more In fact, evaluating the fitness of each individual is usually and more knowledge has been accumulated by biologists difficult for many optimization problems in the real world on the complex process of sexual selection, especially on because explicit fitness can be hard to define and expensive the advantages that sex mechanisms contribute to the to calculate. As a result, fitness approximation has been acceleration of speciation and evolution, this knowledge proposed with differing levels of approximation, including should be better incorporated in EC. “problem approximation”, “functional approximation”, and “evolutionary approximation”. Jin [260] has surveyed these approaches. They are sensitive to training data and to varying 4.3. Fitness Evaluation. Fitness evaluation measures behavior constraints of different models; so a common framework or function of individuals or species. In nature, fitness of would be required. Moreover, Reisinger and Miikkulainen an individual or species is implicit and subject to natural [56] propose an evolvable representation and an evaluation selection, whereas in EC, fitness is mostly based on numerical strategy to exert indirect selection pressure on evolvability. values of an individual as solution to a given problem, and In their work, a systematically changing fitness function this fitness is explicit. is adopted according to a special evolvable representation that can reflect efficiently how genetic changes restructure Definition 9. Fitness is the measure to quantify an evolution- phenotypic variation. Thus, evolvability can be evaluated ary individual/component with regard to its ability to survive through the way such a systematic structure can expand in and reproduce in a certain environment. phenotypes. These approaches might provide a good starting In nature, adaptable species survive by passing different point to simulate the implicit adaptive fitness evaluation challenges, and less fit species may become extinct during from nature, a method that has good prospects for detecting evolution. Adaptability lies not only in the currently existing evolvability in EC. adaptivity to the environment but also in the capability to generate more adapted offspring. In essence, fitness of 5. Conclusion natural organisms is implicit and is subject to natural selection. Empirically, biologists use mathematical methods Since Darwin proposed his theory of natural evolution based to quantify fitness. Individual fitness usually refers to the on heritable variation and natural selection, an enormous viability of an individual, that is, its probability to survive research effort has been dedicated to revealing the intricacies [256]. Moreover, individuals having more offspring can be of the processes involved. In modern biology, a host of details considered as fitter ones since their genetic information is about mechanisms of evolution and factors that can affect more likely to be preserved. Other than at the individual evolution have been revealed. Besides understanding the level, in classic population genetics literature [257], the geno- history of evolution, biologists are currently paying attention type fitness quantifies the frequency changes of a genotype to the capability of organisms to evolve and to the evolution in a population during transformation from one generation of such capability in an open-ended natural evolutionary to the next. Various measures have been proposed in the process. Varying evolution rates among different species or biological literature (see [258, 259] for detailed reviews). different regions of genetic material in an organism attract The above implicit fitness in natural organisms empha- researchers’ interest under the aspect of the acceleration sizes evolvability under intricate pressures from interactions of evolution. Meanwhile, in artificial evolutionary systems, Journal of Artificial Evolution and Applications 21 one is also working on improving the power of systems by [12] “Annual “HUMIES” Awards for Human Competitive Results,”ProducedbyGenetic andEvolutionaryComputa- studying more intelligent and adaptive mechanisms. tion held by GECCO: Genetic and Evolutionary Compu- Evolvability, as the capability to generate adaptation by tation Conference, ACM, New York, NY, USA, since 2004, producing fitter offspring via evolutionary operations, has http://www.humancompetitive.org. received considerable interest in recent research in both [13] C. Darwin, On the Origin of Species by Means of Natural biology and EC. Substantial work has been published on this Selection, John Murray, London, UK, 1859. topic in both areas, and we have tried to cover many of the [14] S. J. Gould, The Structure of Evolutionary Theory,Harvard factors that contribute to evolvability in this review. After University Press, Cambridge, Mass, USA, 2002. some phenomena of rapid evolution were found in nature, [15] E. Mayr and W. B. Provine, Eds., The Evolutionary Synthesis, acceleration of evolution also became an hot research topic, Harvard University Press, Cambridge, Mass, USA, 2nd not the least because an increase in the speed of artificial edition, 1998. evolution would greatly benefit applications. [16] W. Banzhaf, G. Beslon, S. Christensen, et al., “Guidelines: More is to come. As a result of the unrelenting progress in from artificial evolution to computational evolution: a biology, a lively discussion has now ensued as to what a gene research agenda,” Nature Reviews Genetics,vol. 7, no.9,pp. really is [261–264]. There were times when the notion of a 729–735, 2006. gene was simple. But with the advent of alternative splicing, [17] J. Timmis, M. Amos, W. Banzhaf, and A. Tyrrell, “Going back intron activity during regulation, iRNA, and other wonderful to our roots: second generation biocomputing,” International intricacies [265], the simple life of geneticists seems to be Journal of Unconventional Computing, vol. 2, no. 4, pp. 349– over. In this review, our perspective was much more limited 378, 2006. by discussing only two very narrow aspects of evolution, its [18] S. Carroll, J. Grenier, and S. Weatherbee, From DNA to speed and the issue of evolvability. We hope that the ideas Diversity, Blackwell, Oxford, UK, 2005. discussed here can inspire new methods and applications of [19] G. Cochran and H. Harpending, The 10,000 Year Explosion, EC. Basic Books, New York, NY, USA, 2009. [20] G. Forgacs and S. A. Newman, Biological Physics of the Developing Embryo, Cambridge University, Cambridge, UK, Acknowledgments [21] E. Jablonka and M. J. Lamb, Evolution in Four Dimensions: Funding by NSERC under the Discovery Grant Program Genetic, Epigenetic, Behavioral, and Symbolic Variation in the RGPIN 283304-07 is gratefully acknowledged. The authors History of Life, MIT Press, Cambridge, Mass, USA, 2005. thank Y. P. Chen and T. Weise for valuable discussions. [22] M. W. Kirschner and J. C. Gerhart, The Plausibility of Life, Yale University Press, New Haven, Conn, USA, 2005. References [23] M. Novak, Evolutionary Dynamics, Harvard University Press, Cambridge, Mass, USA, 2006. [1] W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, [24] S. R. Palumbi, The Evolution Explosion, Norton, New York, Genetic Programming—An Introduction: On the Automatic NY, USA, 2002. Evolution of Computer Programs and Its Applications,Morgan [25] A. Wagner, Robustness and Evolvability in Living Systems, Kaufmann, San Francisco, CA, USA, 1998. Princeton University Press, Princeton, NJ, USA, 2005. [2] K. A. De Jong, Evolutionary Computation: A Unified [26] M. A. Bedau, J. S. McCaskill, N. H. Packard, et al., “Open Approach, MIT Press, Cambridge, Mass, USA, 2006. problems in artificial life,” Artificial Life, vol. 6, no. 4, pp. 363– [3] A.Eiben andJ.Smith, Introduction to Evolutionary Comput- 376, 2000. ing, Springer, Berlin, Germany, 2003. [27] T. S. Ray, “An approach to the synthesis of life,” in Artificial [4] D.B.Fogel, Evolutionary Computation: Toward a New Life II, pp. 371–408, Addison-Wesley, Reading, Mass, USA, Philosophy of Machine Intelligence, IEEE Press, Orlando, Fla, USA, 1995. [28] C. Adami and C. T. Brown, “Evolutionary learning in the 2D [5] L.J.Fogel,A.J.Owens,and M. J. Walsh, Artificial Intelligence artificial life system “avida”,” in Artificial Life IV, pp. 377–381, through Simulated Evolution, John Wiley & Sons, New York, MIT press, Cambridge, Mass, USA, 1994. NY, USA, 1966. [29] M. A. Bedau, E. Snyder, and C. T. Brown, “A comparison of [6] D.E.Goldberg, Genetic Algorithms in Search, Optimization evolutionary activity in artificial evolving systems and in the and Machine Learning, Addison-Wesley, Reading, Mass, USA, biosphere,” in Proceedings of the 4th European Conference on Artificial Life, pp. 125–134, Springer, 1997. [7] J. H. Holland, Adaptation in Natural and Artificial Systems, [30] M. A. Bedau and C. T. Brown, “Visualizing evolutionary University of Michigan Press, Ann Arbor, Mich, USA, 1975. activity of genotypes,” Artificial Life, vol. 5, no. 1, pp. 17–35, [8] J.R.Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press, Cambridge, Mass, USA, 1992. [31] J. E. Barrick,D.S.Yu, S. H. Yoon,etal., “Genomeevolution and adaptation in a long-term experiment with Escherichia [9] M. Mitchell, An Introduction to Genetic Algorithms,MIT coli,” Nature, vol. 461, no. 7268, pp. 1243–1247, 2009. Press, Cambridge, Mass, USA, 1998. [32] P. Alberch, “From genes to phenotype: dynamical systems [10] I. Rechenberg, Evolution Strategy, Frommann-Holzboog, and evolvability,” Genetica, vol. 84, no. 1, pp. 5–11, 1991. Stuttgart, Germany, 1973. [33] J. F. Y. Brookfield, “Evolution: the evolvability enigma,” [11] H.-P. Schwefel, Evolution and Optimum Seeking: The Sixth Generation, John Wiley & Sons, New York, NY, USA, 1995. Current Biology, vol. 11, no. 3, pp. R106–R108, 2001. 22 Journal of Artificial Evolution and Applications [34] A. J. R. Carter, J. Hermisson, and T. F. Hansen, “The role of [56] J. Reisinger and R. Miikkulainen, “Selecting for evolvable epistatic gene interactions in the response to selection and representations,” in Proceedings of the Genetic and Evolution- the evolution of evolvability,” Theoretical Population Biology, ary Computation Conference (GECCO ’06), vol. 2, pp. 1297– vol. 68, no. 3, pp. 179–196, 2005. 1304, ACM, 2006. [35] J. L. Hendrikse, T. E. Parsons, and B. Hall-grimsson, “Evolv- [57] J. Reisinger, K. O. Stanley, and R. Miikkulainen, “Towards ability as the proper focus of evolutionary developmental an empirical measure of evolvability,” in Proceedings of the biology,” Evolution and Development, vol. 9, no. 4, pp. 393– Genetic and Evolutionary Computation Conference Workshop 401, 2007. Program (GECCO ’05), pp. 257–264, ACM, 2005. [36] D. Houle, “Comparing evolvability and variability of quanti- [58] N. Eldredge and S. J. Gould, “Punctuated equilibria: an tative traits,” Genetics, vol. 130, no. 1, pp. 195–204, 1992. alternative to phyletic gradualism,” in Models in Paleobiology, pp. 82–115, Freeman Cooper, 1972. [37] K. F. Liem, “Key evolutionary innovations, differential diver- sity, and symecomorphosis,” in Evolutionary Innovations,pp. [59] W. H. Li, C. I. Wu, and C. C. Luo, “A new method 147–170, University of Chicago Press, Chicago, Ill, USA, for estimating synonymous and nonsynonymous rates of nucleotide substitution considering the relative likelihood [38] P. D. Sniegowski and H. A. Murphy, “Evolvability,” Current of nucleotide and codon changes,” Molecular Biology and Biology, vol. 16, no. 19, pp. R831–R834, 2006. Evolution, vol. 2, no. 2, pp. 150–174, 1985. [39] G. Bell, “Evolutionary genetics: the evolution of evolution,” [60] T. Miyata and T. Yasunaga, “Molecular evolution of mRNA: Heredity, vol. 94, no. 1, pp. 1–2, 2005. a method for estimating evolutionary rates of synonymous and amino acid substitutions from homologous nucleotide [40] R. Dawkins, “The evolution of evolvability,” in Artificial Life: The Quest for a New Creation, pp. 201–220, Addison-Wesley, sequences and its application,” Journal of Molecular Evolution, vol. 16, no. 1, pp. 23–36, 1980. Reading, Mass, USA, 1989. [41] A. G. Jones, S. J. Arnold, and R. Burger, “The mutation [61] M. Nei and T. Gojobori, “Simple methods for estimating matrix and the evolution of evolvability,” Evolution, vol. 61, the numbers of synonymous and nonsynonymous nucleotide substitutions,” Molecular Biology and Evolution, vol. 3, no. 5, no. 4, pp. 727–745, 2007. pp. 418–426, 1986. [42] L. Partridge and N. H. Barton, “Evolving evolvability,” Nature, vol. 407, no. 6803, pp. 457–458, 2000. [62] Z. Yang and J. R. Bielawski, “Statistical methods for detecting [43] M. Pigliucci, “Perspectives: is evolvability evolvable?” Nature molecular adaptation,” Trends in Ecology and Evolution, vol. 15, no. 12, pp. 496–503, 2000. Reviews Genetics, vol. 9, pp. 75–82, 2008. [63] R. P. Worden, “A speed limit for evolution,” Journal of [44] M. Radman, I. Matic, and F. Taddei, “Evolution of evolvabil- ity,” Annals of the New York Academy of Sciences, vol. 870, pp. Theoretical Biology, vol. 176, no. 1, pp. 137–152, 1995. 146–155, 1999. [64] M. A. Bedau and N. H. Packard, “Measurement of evolution- [45] G. P. Wagner and L. Altenberg, “Complex adaptations and ary activity, teleology, and life,” in Artificial Life II, pp. 431– the evolution of evolvability,” Evolution,vol. 50, no.3,pp. 461, Addison-Wesley, Reading, Mass, USA, 1992. 967–976, 1996. [65] M. A. Bedau, “Quantifying the extent and intensity of adap- [46] L. Altenberg, “The evolution of evolvability in genetic tive evolution,” in Proceedings of the Genetic and Evolution programming,” in Advances in Genetic Programming, pp. 47– Computation Conference Workshop Program (GECCO ’99), 74, MIT Press, Cambridge, Mass, USA, 1994. pp. 34–37, Morgan Kaufmann, San Francisco, CA, USA, [47] M. Kirschner and J. Gerhart, “Evolvability,” Proceedings of the 1999. National Academy of Sciences (USA), vol. 95, no. 15, pp. 8420– [66] M. J. Raven and M. A. Bedau, “General framework for 8427, 1998. evolutionary activity,” in Proceedings of the 7th European [48] P. Marrow, “Evolvability: evolution, computation, and biol- Conference on Artificial Life, pp. 676–685, Springer, 2003. ogy,” in Proceedings of the Genetic and Evolutionary Compu- [67] G. G. Simpson, Tempo and Mode in Evolution, Columbia tation Conference Workshop on Evolvability (GECCO ’99),pp. University Press, New York, NY, USA, 1944. 30–33, Morgan Kaufmann, San Francisco, CA, USA, 1999. [68] Z. D. Blount, C. Z. Borland, and R. E. Lenski, “Historical [49] M. Conrad, “The geometry of evolution,” BioSystems, vol. 24, contingency and the evolution of a key innovation in an no. 1, pp. 61–81, 1990. experimental population of Escherichia coli,” Proceedings of [50] A. Wagner, “Robustness, evolvability, and neutrality,” FEBS the National Academy of Sciences (USA), vol. 105, no. 23, pp. Letters, vol. 579, no. 8, pp. 1772–1778, 2005. 7899–7906, 2008. [51] M. A. Bedau and N. H. Packard, “Evolution of evolvability [69] L. P. Oliveira, H. J. Jensen, M. Nicodemi, and P. Sibani, via adaptation of mutation rates,” BioSystems,vol. 69, no.2- “Record dynamics and the observed temperature plateau in 3, pp. 143–162, 2003. the magnetic creep-rate of type-II superconductors,” Physical [52] J. Masel, “Evolutionary capacitance may be favored by Review B, vol. 71, Article ID 104526, 2005. natural selection,” Genetics, vol. 170, no. 3, pp. 1359–1371, [70] P. Sibani, M. R. Schmidt, and P. Alstrøm, “Fitness opti- 2005. mization and decay of extinction rate through biological [53] C. L. Nehaniv, “Measuring evolvability as the rate of evolution,” Physical Review Letters, vol. 75, no. 10, pp. 2055– 2058, 1995. complexity increase,” in Proceedings of the 7th Artificial Life Workshop, pp. 55–57, MIT Press, 2000. [71] H. A. Simon, “Near decomposability and the speed of [54] A. Wagner, “Robustness and evolvability: a paradox evolution,” Industrial and Corporate Change,vol. 11, no.3, pp. 587–599, 2002. resolved,” Proceedings of the Royal Society B, vol. 275, no. 1630, pp. 91–100, 2008. [72] X. Gu, Z. Zhang, and W. Huang, “Rapid evolution of expres- [55] D. J. Earl and M. W. Deem, “Evolvability is a selectable trait,” sion and regulatory divergences after yeast gene duplication,” Proceedings of the National Academy of Sciences of USA, vol. Proceedings of the National Academy of Sciences (USA), vol. 101, no. 32, pp. 11531–11536, 2004. 102, no. 3, pp. 707–712, 2005. Journal of Artificial Evolution and Applications 23 [73] J. Hawks, E. T. Wang, G. M. Cochran, H. C. Harpending, [91] R. Wilds, S. A. Kauffman, and L. Glass, “Evolution of complex and R. K. Moyzis, “Recent acceleration of human adaptive dynamics,” Chaos, vol. 18, no. 3, Article ID 033109, 2008. evolution,” Proceedings of the National Academy of Sciences [92] J. A. Draghi, T. L. Parsons, G. P. Wagner, and J. B. Plotkin, (USA), vol. 104, no. 52, pp. 20753–20758, 2007. “Mutational robustness can facilitate adaptation,” Nature, vol. 463, no. 7279, pp. 353–355, 2010. [74] N. Kashtan, E. Noor, and U. Alon, “Varying environments can speed up evolution,” Proceedings of the National Academy [93] F. Rothlauf and D. E. Goldberg, “Redundant representations of Sciences (USA), vol. 104, no. 34, pp. 13711–13716, 2007. in evolutionary computation,” Evolutionary Computation, [75] F. Rodriguez-Trelles, R. Tarrio, and F. J. Ayala, “Evolution vol. 11, no. 4, pp. 381–415, 2003. of cis-regulatory regions versus codifying regions,” Interna- [94] W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, “The evolu- tional Journal of Developmental Biology,vol. 47, no.7-8,pp. tion of size and shape,” in Advances in Genetic Programming, 665–673, 2003. vol. 3, pp. 163–190, MIT Press, Cambridge, Mass, USA, 1999. [76] S. Asthana, W. S. Noble, G. Kryukov, C. E. Grant, S. Sunyaev, [95] S. Luke, “Code growth is not caused by introns,” in and J. A. Stamatoyannopoulos, “Widely distributed noncod- Proceedings of the Genetic and Evolutionary Computation ing purifying selection in the human genome,” Proceedings of Conference (GECCO ’00), pp. 228–235, Morgan Kaufmann, the National Academy of Sciences (USA), vol. 104, no. 30, pp. San Francisco, CA, USA, 2000. 12410–12415, 2007. [96] P. Nordin, F. Francone, and W. Banzhaf, “Explicitly defined [77] G. A. Wray, M. W. Hahn, E. Abouheif, et al., “The evolution introns and destructive crossover in genetic programming,” of transcriptional regulation in eukaryotes,” Molecular Biol- in Advances in Genetic Programming, vol. 2, pp. 111–134, MIT ogy and Evolution, vol. 20, no. 9, pp. 1377–1419, 2003. Press, Cambridge, Mass, USA, 1996. [97] T. Yu and J. Miller, “Neutrality and the evolvability of boolean [78] J. D. Wren,E.Forgacs,J.W.FondonIII,etal., “Repeat polymorphisms within gene regions: phenotypic and evolu- function landscape,” in Proceedings of the 4th European tionary implications,” American Journal of Human Genetics, Conference on Genetic Programming (EuroGP ’01), vol. 2038 vol. 67, no. 2, pp. 345–356, 2000. of Lecture Notes in Computer Science, pp. 204–217, Springer, [79] J. L. Freeman, G. H. Perry, L. Feuk, et al., “Copy number vari- ation: new insights in genome diversity,” Genome Research, [98] M. Beckerman, Molecular and Cellular Signaling, Springer, Berlin, Germany, 2005. vol. 16, no. 8, pp. 949–961, 2006. [80] A. J. Iafrate, L. Feuk, M. N. Rivera, et al., “Detection of large- [99] F. Jacob, “Evolution and tinkering,” Science, vol. 196, no. 4295, pp. 1161–1166, 1977. scale variation in the human genome,” Nature Genetics, vol. 36, no. 9, pp. 949–951, 2004. [100] J. Cohen, “Bioinformatics—an introduction for computer scientists,” ACM Computing Surveys, vol. 36, no. 2, pp. 122– [81] J. Sebat, B. Lakshmi, J. Troge, et al., “Large-scale copy number 158, 2004. polymorphisminthe humangenome,” Science, vol. 305, no. 5683, pp. 525–528, 2004. [101] J. D. Watson, N. H. Hopkins, J. W. Roberts, J. A. Steitz, and A. M. Weiner, Molecular Biology of the Gene, Benjamin- [82] E. Tuzun, A. J. Sharp, J. A. Bailey, et al., “Fine-scale structural Cummings, 5th edition, 2004. variation of the human genome,” Nature Genetics, vol. 37, no. 7, pp. 727–732, 2005. [102] B. Alberts, D. Bray, J. Lewis, M. Raff,K.Roberts,and J. D. Watson, Molecular Biology of the Cell, Garland, New York, NY, [83] M. C. Frith, M. Pheasant, and J. S. Mattick, “The amazing USA, 4th edition, 2002. complexity of the human transcriptome,” European Journal of Human Genetics, vol. 13, no. 8, pp. 894–897, 2005. [103] M. Z. Ludwig, N. H. Patel, and M. Kreitman, “Functional analysis of eve stripe 2 enhancer evolution in Drosophila, [84] J. S. Mattick, “Non-coding RNAs: the architects of eukaryotic rules governing conservation and change,” Development, vol. complexity,” EMBO Reports, vol. 2, no. 11, pp. 986–991, 2001. 125, no. 5, pp. 949–958, 1998. [85] J. A. de Visser, J. Hermisson, G. P. Wagner, et al., “Perspective: [104] S. R. Neves and R. Iyengar, “Modeling of signaling networks,” evolution and detection of genetic robustness,” Evolution, vol. BioEssays, vol. 24, no. 12, pp. 1110–1117, 2002. 57, no. 9, pp. 1959–1972, 2003. [105] R. A. Fisher, Genetical Theory of Natural Selection, Clarendon, [86] D. C. Krakauer and J. B. Plotkin, “Redundancy, antiredun- Oxford, UK, 1930. dancy, and the robustness of genomes,” Proceedings of the [106] J. B. S. Haldane, The Causes of Evolution,Harperand Row, National Academy of Sciences (USA), vol. 99, no. 3, pp. 1405– New York, NY, USA, 1932. 1409, 2002. [107] S. Wright, “Evolution in Mendelian populations,” Genetics, [87] J. D. Bloom, S. T. Labthavikul, C. R. Otey, and F. H. Arnold, vol. 16, no. 2, pp. 97–159, 1931. “Protein stability promotes evolvability,” Proceedings of the [108] M. Kimura, “Evolutionary rate at the molecular level,” National Academy of Sciences (USA), vol. 103, no. 15, pp. 5869–5874, 2006. Nature, vol. 217, no. 5129, pp. 624–626, 1968. [109] J. H. Gillespie, “The role of population size in molecular [88] B. C. Daniels,Y.-J. Chen,J.P.Sethna,R.N.Gutenkunst, and C. R. Myers, “Sloppiness, robustness, and evolvability in evolution,” Theoretical Population Biology, vol. 55, no. 2, pp. 145–156, 1999. systems biology,” Current Opinion in Biotechnology, vol. 19, no. 4, pp. 389–395, 2008. [110] J. H. Gillespie, “Is the population size of a species relevant to its evolution?” Evolution, vol. 55, no. 11, pp. 2161–2169, [89] S. F. Elena and R. Sanjuan, “The effect of genetic robustness on evolvability in digital organisms,” BMC Evolutionary 2001. Biology, vol. 8, no. 1, article 284, 2008. [111] T. Ohta, “Population size and rate of evolution,” Journal of Molecular Evolution, vol. 1, no. 4, pp. 305–314, 1972. [90] R. C. McBride, C. B. Ogbunugafor, and P. E. Turner, “Robustness promotes evolvability of thermotolerance in an [112] T. Ohta, “The nearly neutral theory of molecular evolution,” RNA virus,” BMC Evolutionary Biology, vol. 8, no. 1, article Annual Review of Ecology and Systematics, vol. 23, no. 1, pp. 231, 2008. 263–286, 1992. 24 Journal of Artificial Evolution and Applications [113] T. Ohta and M. Kimura, “On the constancy of the evolution- adaptive mutation controversy,” Annual Review of Microbi- ary rate of cistrons,” Journal of Molecular Evolution, vol. 1, no. ology, vol. 60, pp. 477–501, 2006. 1, pp. 18–25, 1971. [130] J. R. Roth and D. I. Andersson, “Adaptive mutation: how [114] M. Woolfit and L. Bromham, “Population size and molecular growth under selection stimulates Lac+ reversion by increas- evolution on islands,” Proceedings of the Royal Society B, vol. ing target copy number,” Journal of Bacteriology, vol. 186, no. 272, no. 1578, pp. 2277–2282, 2005. 15, pp. 4855–4860, 2004. [115] S. Wright, J. Keeling, and L. Gillman, “The road from Santa [131] H. Hendrickson, E. S. Slechta, U. Bergthorsson, D. I. Rosalia: a faster tempo of evolution in tropical climates,” Andersson, and J. R. Roth, “Amplification-mutagenesis: evi- Proceedings of the National Academy of Sciences (USA), vol. dence that “directed” adaptive mutation and general hyper- 103, no. 20, pp. 7718–7722, 2006. mutability result from growth with a selected gene amplifica- [116] G. C. Stevens, “The latitudinal gradient in geographical tion,” Proceedings of the National Academy of Sciences (USA), range: how so many species coexist in the tropics,” The vol. 99, no. 4, pp. 2164–2169, 2002. American Naturalists, vol. 133, no. 2, pp. 240–256, 1989. [132] E. Kugelberg, E. Kofoid, A. B. Reams, D. I. Andersson, and J. [117] S. F. Elena, C. O. Wilke, C. Ofria, and R. E. Lenski, “Effects R. Roth, “Multiple pathways of selected gene amplification of population size and mutation rate on the evolution of during adaptive mutation,” Proceedings of the National mutational robustness,” Evolution, vol. 61, no. 3, pp. 666–674, Academy of Sciences (USA), vol. 103, no. 46, pp. 17319–17324, [118] W. Banzhaf, S. Harding, W. Langdon, and G. Wilson, “Accel- [133] E. S. Slechta, K. L. Bunny, E. Kugelberg, E. Kofoid, D. erating genetic programming through graphics processing I. Andersson, and J. R. Roth, “Adaptive mutation: general units,” in Genetic Programming Theory and Practice VI,pp. mutagenesis is not a programmed response to stress but 229–248, Springer, Berlin, Germany, 2009. results from rare coamplification of dinB with lac,” Proceed- [119] H. Juille ´ and J. B. Pollack, “Massively parallel genetic ings of the National Academy of Sciences (USA), vol. 100, no. programming,” in Advances in Genetic Programming, vol. 2, 22, pp. 12847–12852, 2003. pp. 339–357, MIT Press, Cambridge, Mass, USA, 1996. [134] D. M. Weinreich, N. F. Delaney, M. A. DePristo, and D. [120] W. B. Langdon, “A many threaded CUDA interpreter for L. Hartl, “Darwinian evolution can follow only very few genetic programming,” in Proceedings of the 13th European mutational paths to fitter proteins,” Science, vol. 312, no. Conference on Genetic Programming (EuroGP ’10), vol. 6021 5770, pp. 111–114, 2006. of Lecture Notes in Computer Science, pp. 146–158, Springer, [135] M. A. Bedau and R. Seymour, “Adaptation of mutation rates in a simple model of evolution,” in Complex Systems: Mecha- [121] F. G. Lobo and C. F. Lima, “A review of adaptive population nism of Adaptation, pp. 179–183, IOS Press, Amsterdam, The sizing schemes in genetic algorithms,” in Proceedings of Netherlands, 1994. the Workshops on Genetic and Evolutionary Computation [136] S. Ohno, Evolution by Gene Duplication, Springer, Berlin, (GECCO ’05), pp. 228–234, ACM, 2005. Germany, 1970. [122] D. E. Goldberg, “Sizing populations for serial and parallel [137] M. Nei, “Gene duplication and nucleotide substitution in genetic algorithms,” in Proceedings of the 3rd International evolution,” Nature, vol. 221, no. 5175, pp. 40–42, 1969. Conference on Genetic Algorithms, pp. 70–79, Morgan Kauf- [138] J. Spring, “Genome duplication strikes back,” Nature Genet- mann, San Francisco, CA, USA, 1989. ics, vol. 31, no. 2, pp. 128–129, 2002. [123] D. E. Goldberg, K. Deb, and J. H. Clark, “Genetic algorithms, [139] J. Zhang, “Evolution by gene duplication: an update,” Trends noise, and the sizing of populations,” Complex Systems, vol. in Ecology and Evolution, vol. 18, no. 6, pp. 292–298, 2003. 6, no. 4, pp. 333–362, 1992. [140] J. A. Bailey, Z. Gu, R. A. Clark, et al., “Recent segmental [124] J. Arabas, Z. Michalewicz, and J. Mulawka, “GAVaPS—a duplications in the human genome,” Science, vol. 297, no. genetic algorithm with varying population size,” in Proceed- 5583, pp. 1003–1007, 2002. ings of the 1st IEEE Conference on Evolutionary Computation [141] M. Nei and A. P. Rooney, “Concerted and birth-and-death (CEC ’94), vol. 1, pp. 73–78, IEEE, 1994. evolution of multigene families,” Annual Review of Genetics, [125] G. R. Harik and F. G. Lobo, “A parameter-less genetic vol. 39, no. 15, pp. 121–152, 2005. algorithm,” in Proceedings of the Genetic and Evolutionary [142] P. M. Harrison, H. Hegyi, S. Balasubramanian, et al., Computation Conference (GECCO ’99), pp. 258–267, Morgan “Molecular fossils in the human genome: identification and Kaufmann, San Francisco, CA, USA, 1999. analysis of the pseudogenes in chromosomes 21 and 22,” [126] T. Back, A. E. Eiben, and N. A. L. van der Vaart, “An empirical Genome Research, vol. 12, no. 2, pp. 272–280, 2002. study on GAs “with-out parameters”,” in Proceedings of the [143] J. Zhang, H. F. Rosenberg, and M. Nei, “Positive Darwinian 6th International Conference on Parallel Problem Solving from selection after gene duplication in primate ribonuclease Nature (PPSN ’00), vol. 1917 of Lecture Notes in Computer genes,” Proceedings of the National Academy of Sciences (USA), Science, pp. 315–324, Springer, 2000. vol. 95, no. 7, pp. 3708–3713, 1998. [127] A. E. Eiben, E. Marchiori, and V. A. Valko, ´ “Evolutionary [144] D. Dykhuizen and D. L. Hartl, “Selective neutrality of 6PGD algorithms with on-the-fly population size adjustment,” in allozymes in E. coli and the effects of genetic background,” Proceedings of the 8th International Conference on Parallel Genetics, vol. 96, no. 4, pp. 801–817, 1980. Problem Solving from Nature (PPSN ’04), vol. 3242 of Lecture Notes in Computer Science, pp. 41–50, Springer, 2004. [145] L. Altenberg, “Knowledge representation in the genome: new [128] M. Nei, “Selectionism and neutralism in molecular evolu- genes, exons, and pleiotropy,” Genetics, vol. 110, p. s41, 1985. tion,” Molecular Biology and Evolution, vol. 22, no. 12, pp. [146] L. Altenberg, “Genome growth and the evolution of the 2318–2342, 2005. genotype-phenotype map,” in Proceedings of the Evolution [129] J. R. Roth, E. Kugelberg, A. B. Reams, E. Kofoid, and D. and Biocomputation Workshop, vol. 899 of Lecture Notes in I. Andersson, “Origin of mutations under selection: the Computer Science, pp. 205–259, Springer, 1995. Journal of Artificial Evolution and Applications 25 [147] D. Sankoff, “Gene and genome duplication,” Current Opinion Problem Solving from Nature (PPSN ’92), pp. 269–278, in Genetics and Development, vol. 11, no. 6, pp. 681–684, North-Holland, 1992. 2001. [166] W. B. Langdon, “Size fair and homologous tree genetic pro- [148] F. Gruau, “Genetic micro programming of neural networks,” gramming crossovers,” Genetic Programming and Evolvable in Advances in Genetic Programming, pp. 495–518, MIT Press, Machines, vol. 1, no. 1, pp. 95–119, 2000. Cambridge, Mass, USA, 1994. [167] P. Nordin, W. Banzhaf, and F. Francone, “Efficient evolution [149] J. R. Koza, “Gene duplication to enable genetic programming of machine code for cisc architectures using blocks and to concurrently evolve both the architecture and work- homologous crossover,” in Advances in Genetic Programming, performing steps of a computer program,” in Proceedings of vol. 3, pp. 275–299, MIT Press, Cambridge, Mass, USA, 1999. the 14th International Joint Conference on Artificial Intelli- [168] R. Poli and N. F. Mcphee, “Exact schema theory for gp gence, pp. 695–717, Morgan Kaufmann, San Francisco, CA, and variable-length gas with homologous crossover,” in USA, 1995. Proceedings of the Genetic and Evolutionary Computation [150] K. Schmitt, “Using gene duplication and gene deletion Conference (GECCO ’01), pp. 104–111, Morgan Kaufmann, in evolution strategies,” Tech. Rep., Collaborative Research San Francisco, CA, USA, 2001. Center 531, University of Dortmund, Dortmund, Germany, [169] M. Srinivas and L. M. Patnaik, “Adaptive probabilities 2004. of crossover and mutation in genetic algorithms,” IEEE [151] M. Schutz, “Other operations: gene duplication and dele- Transactions on Systems, Man and Cybernetics, vol. 24, no. 4, tion,” in Handbook of Evolutionary Computation, pp. 8–15, pp. 656–667, 1994. Oxford University Press, Oxford, UK, 1997. [170] J. W. Pepper, “The evolution of evolvability in genetic linkage [152] R. Calabretta, S. Nolfi, D. Parisi, and G. P. Wagner, “Dupli- patterns,” BioSystems, vol. 69, no. 2-3, pp. 115–126, 2003. cation of modules facilitates the evolution of functional [171] G. G. Simpson, “The Baldwin effect,” Evolution, vol. 7, no. 2, specialization,” Artificial Life, vol. 6, no. 1, pp. 69–84, 2000. pp. 110–117, 1953. [153] M. C. King and A. C. Wilson, “Evolution at two levels in [172] H. A. Simon, “The organization of complex systems,” in humans and chimpanzees,” Science, vol. 188, no. 4184, pp. Hierarchy Theory: The Challenge of Complex Systems,pp. 1– 107–116, 1975. 27, George Braziller, 1973. [154] J. H. Moore, F. W. Asselbergs, and S. M. Williams, “Bioin- [173] H. A. Simon, “The architecture of complexity,” Proceedings formatics challenges for genome-wide association studies,” of the American Philosophical Society, vol. 106, pp. 467–482, Bioinformatics, vol. 26, no. 4, pp. 445–455, 2010. 1962. [155] D. Fredman, S. J. White, S. Potter, E. E. Eichler, et al., [174] L. Altenberg, “The schema theorem and Price’s theorem,” “Complex SNP-related sequence variation in segmental in Fundations of Genetic Algorithms, vol. 3, pp. 23–49, MIT genome duplications,” Nature Genetics,vol. 36, no.8,pp. Press, Cambridge, Mass, USA, 1995. 861–866, 2004. [175] J. T. Bonner, The Evolution of Complexity by Means of Natural [156] C. J. Shaw and J. R. Lupski, “Implications of human genome Selection, Princeton University, Princeton, NJ, USA, 1988. architecture for rearrangement-based disorders: the genomic [176] J. E. G. Leigh, “Levels of selection, poten-tial conflicts, and basis of disease,” Human Molecular Genetics, vol. 13, no. 1, their resolution: the role of the “common good”,” in Levels of pp. R57–R64, 2004. SelectioninEvolution, pp. 15–30, Princeton University Press, [157] D. Sankoff, “Rearrangements and chromosomal evolution,” Princeton, NJ, USA, 1999. Current Opinion in Genetics and Development,vol. 13, no.6, [177] S. T. Kazadi, D. Lee, R. Modi, J. Sy, and W. Lue, “Levels of pp. 583–587, 2003. compartmentalization in artificial evolution,” in Proceedings [158] D. Posada, K. A. Crandall, and E. C. Holmes, “Recombina- of the Genetic and Evolutionary Computation Conference tion in evolutionary genomics,” Annual Review of Genetics, (GECCO ’00), pp. 841–848, Morgan Kaufmann, San Fran- vol. 36, pp. 75–97, 2002. cisco, CA, USA, 2000. [159] G. P. Smith, “Evolution of repeated DNA sequences by [178] S. C. Manrubia and C. Briones, “Modular evolution unequal crossover,” Science, vol. 191, no. 4227, pp. 528–535, and increase of functional complexity in replicating RNA 1976. molecules,” RNA, vol. 13, no. 1, pp. 97–107, 2007. [160] A. S. Kondrashov, “Deleterious mutations and the evolution [179] T. Back, “Self-adaptation in genetic algorithms,” in Proceed- of sexual reproduction,” Nature, vol. 336, no. 6198, pp. 435– ings of the 1st European Conference on Artificial Life: Toward 440, 1988. a Practice of Autonomous Systems, pp. 263–271, MIT Press, [161] L. Hadany and T. Beker, “Sexual selection and the evolution 1992. of obligatory sex,” BMC Evolutionary Biology, vol. 7, no. 1, [180] N. Hansen and A. Ostermeier, “Completely derandomized article 245, 2007. self-adaptation in evolution strategies,” Evolutionary Compu- [162] A. Kong, D. F. Gudbjartsson, J. Sainz, et al., “A high- tation, vol. 9, no. 2, pp. 159–195, 2001. resolution recombination map of the human genome,” [181] R. Hinterding, Z. Michalewicz, and A. Eiben, “Adaptation Nature Genetics, vol. 31, no. 3, pp. 241–247, 2002. in evolutionary computation: a survey,” in Proceedings of the [163] A. E. Eiben, “Multiparent recombination in evolutionary IEEE Conference on Evolutionary Computation (CEC ’97),pp. computing,” in Advances in Evolutionary Computing: Theory 65–69, IEEE, 1997. and Applications, pp. 175–192, Springer, Berlin, Germany, [182] H. Kita, “A comparison study of self-adaptation in evolution 2003. strategies and real-coded genetic algorithms,” Evolutionary [164] D. E. Goldberg, B. Korb, and K. Deb, “Messy genetic Computation, vol. 9, no. 2, pp. 223–241, 2001. algorithms: motivation, analysis, and first results,” Complex [183] B. Edmonds, “Meta-genetic programming: co-evolving the Systems, vol. 3, no. 5, pp. 493–530, 1989. operators of variation,” Turkish Journal of Electrical Engineer- [165] I. Harvey and I. Harvey, “The SAGA cross: the mechanics of ing and Computer Sciences, vol. 9, pp. 13–29, 2001. recombination for species with variable length genotypes,” [184] W. Kantschik, P. Dittrich, M. Brameier, and W. Banzhaf, in Proceedings of the 2nd International Conference on Parallel “Empirical analysis of different levels of meta-evolution,” 26 Journal of Artificial Evolution and Applications in Proceedings of the Congress on Evolutionary Computation [201] I. Shmulevich, S. A. Kauffman, and M. Aldana, “Eukaryotic (CEC ’99), vol. 3, pp. 2086–2093, IEEE Press, 1999. cells are dynamically ordered or critical but not chaotic,” Proceedings of the National Academy of Sciences (USA), vol. [185] L. Spector and A. Robinson, “Genetic programming and 102, no. 38, pp. 13439–13444, 2005. autoconstructive evolution with the push programming language,” Genetic Programming and Evolvable Machines, vol. [202] J. R. Stone and G. A. Wray, “Rapid evolution of cis-regulatory 3, no. 1, pp. 7–40, 2002. sequences via local point mutations,” Molecular Biology and [186] J. Tavares, P. Machado, A. Cardoso, F. B. Pereira, and E. Costa, Evolution, vol. 18, no. 9, pp. 1764–1770, 2001. “On the evolution of evolutionary algorithms,” in Proceedings [203] A. Wagner, “Evolution of gene networks by gene duplica- of the 7th European Conference on Genetic Programming tions: a mathematical model and its implications on genome (EuroGP ’04), vol. 3003 of Lecture Notes in Computer Science, organization,” Proceedings of the National Academy of Sciences pp. 389–398, Springer, 2004. (USA), vol. 91, no. 10, pp. 4387–4391, 1994. [187] M. Parter, N. Kashtan, and U. Alon, “Facilitated variation: [204] J. W. Fondon III and H. R. Garner, “Molecular origins of how evolution learns from past environments to generalize rapid and continuous morphological evolution,” Proceedings to new environments,” PLoS Computational Biology, vol. 4, of the National Academy of Sciences (USA), vol. 101, no. 52, no. 11, article e1000206, 2008. pp. 18058–18063, 2004. [188] C. D. Allis, T. Jenuwein, and D. Reinberg, Eds., Epigenetics, [205] W. Banzhaf, “Artificial regulatory networks and genetic Cold Spring Harbor Laboratory, Woodbury, NY, USA, 2007. programming,” in Genetic Programming Theory and Practice, [189] J. S. Mattick, “Challenging the dogma: the hidden layer of pp. 43–62, Kluwer Academic Publishers, Dordrecht, The non-protein-coding RNAs in complex organisms,” BioEssays, Netherlands, 2003. vol. 25, no. 10, pp. 930–939, 2003. [206] W. Banzhaf, “On the dynamics of an artificial regulatory [190] S. A. Kauffman, “Metabolic stability and epigenesis in networks,” in Proceedings of the 7th European Conference on randomly constructed genetic nets,” Journal of Theoretical Artificial Life: Advances in Artificial Life, vol. 2801 of Lecture Biology, vol. 22, no. 3, pp. 437–467, 1969. Notes in Artificial Intelligence, pp. 217–227, Springer, 2003. [191] S. A. Kauffman, Origins of Order: Self-Organization and [207] W. Banzhaf and P. D. Kuo, “Network motifs in natural Selection in Evolution, Oxford University Press, Oxford, UK, and artificial transcriptional regulatory networks,” Journal of Biological Physics and Chemistry, vol. 4, no. 2, pp. 85–92, [192] R. Albert and H. G. Othmer, “The topology of the regulatory interactions predicts the expression pattern of the segment [208] P. Kuo, W. Banzhaf, and A. Leier, “Network topology and polarity genes in Drosophila melanogaster,” Journal of Theo- the evolution of dynamics in an artificial genetic regulatory retical Biology, vol. 223, no. 1, pp. 1–18, 2003. network model created by whole genome duplication and [193] C. Espinosa-Soto, P. Padilla-Longoria, and E. R. Alvarez- divergence,” BioSystems, vol. 85, no. 3, pp. 177–200, 2006. Buylla, “A gene regulatory network model for cell-fate deter- [209] A. Chavoya and Y. Duthen, “Evolving an artificial regulatory mination during Arabidopsis thaliana flower development network for 2D cell patterning,” in Proceedings of the IEEE that is robust and recovers experimental gene expression Symposium on Artificial Life (ALIFE ’07), pp. 47–53, IEEE, profiles,” Plant Cell, vol. 16, no. 11, pp. 2923–2939, 2004. Honolulu, Hawaii, USA, 2007. [194] S. Huang, G. Eichler, Y. Bar-Yam, and D. E. Ingber, “Cell [210] C. Mattiussi and D. Floreano, “Analog genetic encoding for fates as high-dimensional attractor states of a complex gene the evolution of circuits and networks,” IEEE Transactions on regulatory network,” Physical Review Letters, vol. 94, no. 12, Evolutionary Computation, vol. 11, no. 5, pp. 596–607, 2007. Article ID 128701, 2005. [211] C. L. Nehaniv, “Self-replication, evolvability and asyn- [195] S. Huang and D. E. Ingber, “Shape-dependent control of chronicity in stochastic worlds,” in Stochastic Algorithms: cell growth, differentiation, and apoptosis: switching between Foundations and Applications, pp. 126–169, Springer, Berlin, attractors in cell regulatory networks,” Experimental Cell Germany, 2005. Research, vol. 261, no. 1, pp. 91–103, 2000. [212] J. S. Mattick, “RNA regulation: a new genetics?” Nature [196] M. Aldana, E. Balleza, S. Kauffman, and O. Resendiz, Reviews Genetics, vol. 5, no. 4, pp. 316–323, 2004. “Robustness and evolvability in genetic regulatory networks,” [213] C. P. Mack and G. K. Owens, “Regulation of smooth muscle Journal of Theoretical Biology, vol. 245, no. 3, pp. 433–448, α-actin expression in vivo is dependent on CArG elements within the 5 and first intron promoter regions,” Circulation [197] E. Balleza, E. R. Alvarez-Buylla, A. Chaos, S. Kauffman, I. Research, vol. 84, no. 7, pp. 852–861, 1999. Shmulevich, and M. Aldana, “Critical dynamics in genetic [214] A. Gonc ¸ alves and E. Costa, “A computational model of regulatory networks: examples from four kingdoms,” PLoS gene regulatory networks and its topological properties,” One, vol. 3, no. 6, article e2456, 2008. in Proceedings of the 11th International Conference on the [198] M. Nykter, N. D. Price, M. Aldana, et al., “Gene expression Simulation and Synthesis of Living Systems (ALIFE ’08),pp. dynamics in the macrophage exhibit criticality,” Proceedings 204–211, 2008. of the National Academy of Sciences (USA), vol. 105, no. 6, pp. [215] R. Jaenisch and A. Bird, “Epigenetic regulation of gene 1897–1900, 2008. expression: how the genome integrates intrinsic and environ- [199] R. Serra, M. Villani, A. Graudenzi, and S. A. Kauffman, “Why mental signals,” Nature Genetics, vol. 33, pp. 245–254, 2003. a simple model of genetic regulatory networks describes the [216] S. B. Baylin and J. G. Herman, “DNA hypermethylation in distribution of avalanches in gene expression data,” Journal of Theoretical Biology, vol. 246, no. 3, pp. 449–460, 2007. tumorigenesis: epigenetics joins genetics,” Trends in Genetics, vol. 16, no. 4, pp. 168–174, 2000. [200] R. Serra, M. Villani, and A. Semeria, “Genetic network models and statistical properties of gene expression data in [217] C. D. Allis, T. Jenuwein, and D. Reinberg, “Overview and knock-out experiments,” Journal of Theoretical Biology, vol. concepts,” in Epigenetics, pp. 23–61, Cold Spring Harbor 227, no. 1, pp. 149–157, 2004. Laboratory, Woodbury, NY, USA, 2007. Journal of Artificial Evolution and Applications 27 [218] T. Jenuwein and C. D. Allis, “Translating the histone code,” [237] S. K. Sharma and G. W. Irwin, “Fuzzy coding of genetic Science, vol. 293, no. 5532, pp. 1074–1080, 2001. algorithms,” IEEE Transactions on Evolutionary Computation, [219] A. P. Wolffe and M. A. Matzke, “Epigenetics: regulation vol. 7, no. 4, pp. 344–355, 2003. through repression,” Science, vol. 286, no. 5439, pp. 481–486, [238] W. Banzhaf and J. Miller, “The challenge of complexity,” in 1999. Frontiers in Evolutionary Computation, pp. 243–260, Kluwer [220] S. I. S. Grewal and D. Moazed, “Heterochromatin and Academic Publishers, Dordrecht, The Netherlands, 2004. epigenetic control of gene expression,” Science, vol. 301, no. [239] D. Roggen and D. Federici, “Multi-cellular development: is 5634, pp. 798–802, 2003. there scalability and robustness to gain?” in Proceedings of the [221] A. S. Uyar and A. E. Harmanci, “A new population based 8th International Conference on Parallel Problem Solving from adaptive domination change mechanism for diploid genetic Nature (PPSN ’04), vol. 3242 of Lecture Notes in Computer algorithms in dynamic environments,” Soft Computing, vol. Science, pp. 391–400, 2004. 9, no. 11, pp. 803–814, 2005. [240] S. Harding and W. Banzhaf, “Artificial development,” in [222] W. Arthur, “The effect of development on the direction Organic Computing, pp. 201–220, Springer, Berlin, Germany, of evolution: toward a twenty-first century consensus,” 2008. Evolution and Development, vol. 6, no. 4, pp. 282–288, 2004. [241] S. Harding, J. F. Miller, and W. Banzhaf, “Self modifying cartesian genetic programming: fibonacci, squares, regres- [223] R. A. Raff, The Shape of Life: Genes, Development, and the Evolution of Animal Form, University of Chicago Press, sion and summing,” in Proceedings of the 12th European Chicago, Ill, USA, 1996. Conference on Genetic Programming (EuroGP ’09), vol. 5481 [224] M. West-Eberhard, Developmental Plasticity and Evolution, of Lecture Notes in Computer Science, pp. 133–144, Springer, Oxford University Press, Oxford, UK, 2003. Tubingen, Germany, 2009. [225] A. D. Bradshaw, “Unravelling phenotypic plasticity—why [242] A. P. Hendry, “Evolutionary biology: the power of natural should we bother?” New Phytologist, vol. 170, no. 4, pp. 644– selection,” Nature, vol. 433, no. 7027, pp. 694–695, 2005. 648, 2006. [243] L. Ancel-Meyers, F. D. Ancel, and M. Lachmann, “Evolution [226] T. J. Garland and S. A. Kelly, “Phenotypic plasticity and of genetic potential,” PLoS Computational Biology, vol. 1, no. experimental evolution,” The Journal of Experimental Biology, 3, pp. 236–243, 2005. vol. 209, no. 12, pp. 2344–2361, 2006. [244] A. Crombach and P. Hogeweg, “Evolution of evolvability in [227] T. D. Price, A. Qvarnstr, and D. E. Irwin, “The role of pheno- gene regulatory networks,” PLoS Computational Biology, vol. typic plasticity in driving genetic evolution,” Proceedings of 4, no. 7, article e1000112, 2008. the Royal Society B, vol. 270, no. 1523, pp. 1433–1440, 2003. [245] B. L. Phillips and R. Shine, “Adapting to an invasive species: [228] F. Valladares, D. Sanchez-Gomez, and M. A. Zavala, “Quan- toxic cane toads induce morphological change in Australian snakes,” Proceedings of the National Academy of Sciences titative estimation of phenotypic plasticity: bridging the gap between the evolutionary concept and its ecological (USA), vol. 101, no. 49, pp. 17150–17155, 2004. applications,” Journal of Ecology, vol. 94, no. 6, pp. 1103– [246] N. Kashtan and U. Alon, “Spontaneous evolution of mod- 1116, 2006. ularity and network motifs,” Proceedings of the National [229] W. T. Wcislo, “Behavioral environments and evolutionary Academy of Sciences (USA), vol. 102, no. 39, pp. 13773–13778, change,” Annual Review of Ecology and Systematics, vol. 20, 2005. pp. 137–169, 1989. [247] S. Collins, J. de Meaux, and C. Acquisti, “Adaptive walks [230] J. Draghi and G. P. Wagner, “Evolution of evolvability in a toward a moving optimum,” Genetics, vol. 176, no. 2, pp. developmental model,” Evolution, vol. 62, no. 2, pp. 301–315, 1089–1099, 2007. 2008. [248] T. Blickle and L. Thiele, “A comparison of selection schemes [231] J. C. Gerhart and M. W. Kirschner, Cells, Embryos, and used in evolutionary algorithms,” Evolutionary Computation, Evolution, Blackwell Science, Oxford, UK, 1997. vol. 4, no. 4, pp. 361–394, 1996. [232] P. Bentley and S. Kumar, “Three ways to grow designs: [249] D. E. Goldberg and K. Deb, “A comparative analysis of a comparison of embryogenies for an evolutionary design selection schemes used in genetic algorithms,” in Foundations problem,” in Proceedings of the Genetic and Evolutionary of Genetic Algorithms 1, pp. 69–93, Morgan Kaufmann, San Mateo, CA, USA, 1991. Computation Conference Workshop on Evolvability (GECCO ’99), pp. 35–43, Morgan Kaufmann, San Francisco, CA, USA, [250] M. Kirkpatrick and V. Ravigne, “Speciation by natural and 1999. sexual selection: models and experiments,” The American [233] F. Gruau, D. Whitley, and L. Pyeatt, “A comparison between Naturalist, vol. 159, no. 3, pp. S22–S35, 2002. cellular encoding and direct encoding for genetic neural [251] R. Lande, “Models of speciation by sexual selection on networks,” in Proceedings of the 1st Annual Conference on polygenic traits,” Proceedings of the National Academy of Genetic Programming (GECCO ’96), pp. 81–89, MIT Press, Sciences (USA), vol. 78, no. 6, pp. 3721–3725, 1981. Cambridge, Mass, USA, 1996. [252] N. Colegrave, “Sex releases the speed limit on evolution,” [234] G. S. Hornby and J. B. Pollack, “Creating high-level com- Nature, vol. 420, no. 6916, pp. 664–666, 2002. ponents with a generative representation for body-brain [253] W. J. Swanson and V. D. Vacquier, “The rapid evolution of evolution,” Artificial Life, vol. 8, no. 3, pp. 223–246, 2002. reproductive proteins,” Nature Reviews Genetics,vol. 3, no.2, [235] K. O. Stanley, “Generative and developmental systems,” in pp. 137–144, 2002. Proceedings of the Annual Conference Companion on Genetic [254] G. F. Miller, “Exploiting mate choice in evolutionary compu- and Evolutionary Computation Conference (GECCO ’09),pp. tation: sexual selection as a process of search, optimization, 3335–3354, ACM, New York, NY, USA, 2009. and diversification,” in Selected Papers from AISB Workshop [236] G. S. Hornby and J. B. Pollack, “The advantages of generative on Evolutionary Computing, pp. 65–79, Springer, Berlin, grammatical encodings for physical design,” in Proceedings of Germany, 1994. the IEEE Conference on Evolutionary Computation (CEC ’01), [255] G. F. Miller, The Mating Mind: How Sexual Choice Shaped vol. 1, pp. 600–607, IEEE, 2001. Human Nature, Anchor, Harpswell, Me, USA, 2001. 28 Journal of Artificial Evolution and Applications [256] J. H. Gillespie, Population Genetics: A Concise Guide, Johns Hopkins University Press, Baltimore, MD, USA, 2004. [257] J. F. Crow and M. Kimura, An Introduction to Population Genetics Theory, Harper and Row, New York, NY, USA, 1970. [258] H. A. Orr, “Fitness and its role in evolutionary genetics,” Nature Reviews Genetics, vol. 10, no. 8, pp. 531–539, 2009. [259] E. Sober, “The two faces of fitness,” in Thinking about Evolution: Historical, Philosophical, and Political Perspectives, pp. 309–321, Cambridge University Press, Cambridge, UK, [260] Y. Jin, “A comprehensive survey of fitness approximation in evolutionary computation,” Soft Computing,vol. 9, no.1,pp. 3–12, 2005. [261] M. B. Gerstein, C. Bruce, J. S. Rozowsky, et al., “What is a gene, post-ENCODE? History and updated definition,” Genome Research, vol. 17, no. 6, pp. 669–681, 2007. [262] T. R. Gingeras, “Origin of phenotypes: genes and transcripts,” Genome Research, vol. 17, no. 6, pp. 682–690, 2007. [263] K. Scherrer and J. Jost, “The gene and the genon concept: a functional and information-theoretic analysis,” Molecular Systems Biology, vol. 3, p. 87, 2007. [264] P. F. Stadler, S. J. Prohaska, C. V. Forst, and D. C. Krakauer, “Defining genes: a computational framework,” Theory in Biosciences, vol. 128, no. 3, pp. 165–170, 2009. [265] E. Birney, J. A. Stamatoyannopoulos, A. Dutta, et al., “Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project,” Nature, vol. 447, no. 7146, pp. 799–816, 2007. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Artificial Evolution and Applications Hindawi Publishing Corporation

Evolvability and Speed of Evolutionary Algorithms in Light of Recent Developments in Biology

Loading next page...
 
/lp/hindawi-publishing-corporation/evolvability-and-speed-of-evolutionary-algorithms-in-light-of-recent-XwVlCxRbj5
Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2010 Ting Hu and Wolfgang Banzhaf. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
1687-6229
DOI
10.1155/2010/568375
Publisher site
See Article on Publisher Site

Abstract

Hindawi Publishing Corporation Journal of Artificial Evolution and Applications Volume 2010, Article ID 568375, 28 pages doi:10.1155/2010/568375 Review Article Evolvability and Speed of Evolutionary Algorithms in Light of Recent Developments in Biology 1, 2 1 Ting Hu and Wolfgang Banzhaf Department of Computer Science, Memorial University, St. John’s, NL, Canada A1B 3X5 Norris Cotton Cancer Center, Dartmouth-Hitchcock Medical Center, Lebanon, NH 03756, USA Correspondence should be addressed to Ting Hu, Ting.Hu@Dartmouth.edu Received 15 September 2009; Accepted 24 February 2010 Academic Editor: Franz Rothlauf Copyright © 2010 T. Hu and W. Banzhaf. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Biological and artificial evolutionary systems exhibit varying degrees of evolvability and different rates of evolution. Such quantities can be affected by various factors. Here, we review some evolutionary mechanisms and discuss new developments in biology that can potentially improve evolvability or accelerate evolution in artificial systems. Biological notions are discussed to the degree they correspond to notions in Evolutionary Computation. We hope that the findings put forward here can be used to design computational models of evolution that produce significant gains in evolvability and evolutionary speed. 1. Introduction natural evolution has improved profoundly in biology. This progress has, to a large degree, not been incorporated yet into The field of Evolutionary Computation (EC) has seen enor- computational models of evolution and therefore cannot be mous progress since it was founded in the Sixties and harvested for applications. We have argued that adopting Seventies of the 20th century [1–11], inspired by the new knowledge about natural evolution generated in areas evolutionary processes observed in the living world. such as molecular genetics, cell biology, developmental In EC, candidate solutions to optimization or learning biology, and evolutionary biology would substantially benefit problems are represented by structures similar to gene EC [16, 17]. sequences and their phenotypic expressions. The ensemble The question then arises what the most important and of such solutions is referred to as a population. Evolutionary revolutionary discoveries are in biology in recent times, and operators, such as mutation, recombination, and selection, how they can be sufficiently abstracted to provide material are applied to this population. Solutions gradually improve for computational models. As the number of scientists by repeating a variation-selection cycle through numerous working in the areas mentioned above is now higher than iterations of the evolutionary process. Essentially a search at any other time of the past and can be estimated to be well method, EC, often produces well-performing solutions to over a million, it becomes nontrivial to select those aspects complex optimization and learning problems arising from of evolution that will have the most impact in computational various areas, to the point where its problem solving models. A number of books have appeared in recent years capability mirrors or even exceeds that of humans [12]. Yet, that provide some guidance in this quest (see, e.g., [18–25]). EC is not without weaknesses, and new algorithmic variants Here we restrict ourselves to mainly review the con- are constantly being introduced, studied, and applied. cepts of evolvability and the speed of evolution. This is The fundamental idea of EC was gleaned from biology, motivated by the fact that EC approaches often suffer from and more specifically, from Darwin’s theory of evolution by progressive slow-down of evolutionary speed. While under natural selection [13] as embodied in the Neo-Darwinian some circumstances appreciated as convergence to a global synthesis [14, 15]. In the past decades, however, knowledge of optimum, for many real-world tasks convergence and the 2 Journal of Artificial Evolution and Applications corresponding slow-down of fitness improvements as well Altenberg [46] describes evolvability from the viewpoint as the reduction in the diversity of solutions is more a of EC as the ability of a genetic operator or representation predicament than an advantage. This is especially true for scheme to produce offspring fitter than their parents. In difficult problems where there is no hope to find an optimal biology, Kirschner and Gerhart [47] consider evolvability as solution, but where good solutions would already provide an organism’s capacity to generate heritable and selectable a benefit in the application. As a result, the development phenotypic variation. An explicit comparison between evolv- of systems that show continued evolutionary potential, ability in biological and computational systems has been open-ended evolution,asithas been termed,has gained performed by Wagner and Altenberg [45]. In their view, prominence. evolvability must be seen as the ability of random variants Open-ended evolution is a hallmark of Life. Thus, one to produce occasional improvements, which would depend alternative route to explore this topic in computation is critically on the plasticity of the genotype-phenotype map. via an Artificial Life approach [26]. And so a number of The authors emphasize “variability” determined by the artificial systems have been designed in the meantime with genotype-phenotype map as the propensity to vary, rather the aim to simulate organic life in silico,suchas Tierra than variation itself. Marrow [48] suggests that evolvability [27], Avida [28], and Evita [29]. In these systems, computer means the capability to evolve, and this characteristic should code is regarded as “digital organisms” with CPU time be relevant to both natural and artificial evolutionary the “energy” resource and memory the “material” resource. systems. He discusses a number of important contributions Digital organisms evolve through interactions with their on this topic in both biology and EC and raise some open neighbors and competition for resources. Fitness is not an questions for further research. explicit notion in these systems. However, this is still only Recently, a growing number of evolutionary biologists an initial step towards understanding and realizing open- and computer scientists have shown interest in this topic. ended evolution in artificial systems, as Bedau and Brown In an evolutionary system, many properties of a population [30] report that the long-term capability to adaptation are considered related to evolvability, including facilitation seems to be missing from these systems compared to real of extradimensional bypass and robustness against genotypic organisms (for an example of long-term capabilities in variation [49, 50], redundancy, flexibility during develop- simple organisms, see [31]); that is, this type of artificial mental processes [47], and mutation rate adaptation [51]. evolution lacks evolvability in the “long-term”. The notion of evolutionary capacitance has also been used in In EC, the situation is even exacerbated by the existence this context ([52], and references therein). of an explicitly defined fitness function, often in the form The detection and measurement of evolvability is an of a simple scalar. However, while it is the holy grail of intriguing and nontrivial problem. Phenotypic fitness is computational models of evolution to achieve continued directly observable and serves as a selection criterion. evolutionary potential, which has—to the best of our However, as a potential to generate better fitness and a knowledge—not been reached to date, progress has still capability for adaptive evolution, evolvability is a different been achieved by studying more limited concepts like that type of observable, which is more difficult to observe and of evolvability. In a nutshell, the hope is that by relating to quantify. Although a formal methodology on measuring properties of natural evolutionary systems to mechanisms evolvability has not yet been agreed upon in the literature, used by Nature to achieve them, we might learn enough to some empirical methods have been proposed nevertheless. design algorithmic mechanisms that exhibit similar features. Nehaniv [53] proposes the perspective of using evolu- So let us start by looking more closely at evolvability and the tionary system complexity to describe and measure evolv- rate of evolution. ability. He defines the exhibited evolvability as an observable outcome generated by evolvability and measures evolvability by the rate of increasing complexity of evolutionary entities 1.1. Evolvability. In the process of evolution, genotypic vari- in an evolutionary system. Wagner proposes to simply mea- ation explores new evolutionary material, the corresponding sure the number of nonneutral 1-step mutation variations in phenotypic variation provides adaptive characteristics, and a biological system of particular relevance to RNA evolution stabilization operators like selection preserve improvements in order to quantify evolvability [54]. As one can see, over previous generations. The cooperation of these activities Nehaniv’s definition entailing more complex entities will in is what allows evolution to work. Thus, the core mechanism general also lead to a larger number of nonneutral 1-step of evolution is to assemble the forces of these operations that variations and thus increase this measure of evolvability. yield adaptive improvements implying the evolvability of an Another perspective on evolvability is provided by Earl evolutionary system. A growing number of efforts have been and Deem [55] who suggest that evolvability can be selected dedicated to understanding [25, 32–38] and enhancing [39– for by variation in the environment. By observing genetic 45] evolvability. changes in protein evolution, they find that rapid or dramatic While the concept of evolvability is still very much environmental change generates strong selection pressure under discussion, we will adopt a definition that is equally for evolvability. Thus, high evolvability can be detected applicable to natural and artificial systems. and favored by such selection pressure. For an artificial Definition 1. Evolvability is the capability of a system to evolutionary system Reisinger et al. concur when they generate adaptive phenotypic variation and to transmit it via propose an indirect encoding representation to improve an evolutionary process. evolvability [56, 57]. A gradually changing fitness function Journal of Artificial Evolution and Applications 3 is designed to measure evolvability of representations and variation. If k = k , the fixation of these two types of changes a s to evolve a population that is adaptive under different is at the same rate, a special case indicating, for example, environments. Furthermore, as the pace of change of the pseudogenes. To summarize, measuring a large k /k ratio a s fitness function increases, stronger selection pressure for suggests that adaptation has been generated (and fixed) at a evolvability is imposed. high rate. This measurement has been widely applied in the analysis of adaptive molecular evolution and is accepted as a general method for measuring the rate of gene sequence 1.2. Rate of Evolution. Related to the theme of evolvability is evolution in biology. that of the rate of evolution. Evolvability defines how likely a Other than at the molecular level, Worden has defined system can generate adaptive phenotypic variations whereas the concept of genetic information in the phenotype (GIP) the rate of evolution describes how fast this evolutionary in his work on the speed limit for evolution [63]. GIP is process can proceed. The rate of evolution is a fascinating meant to be a measure of the amount of genetic information topic in evolutionary biology and has caused many debates expressed in observable phenotype, and he uses the rate of already since Darwin’s time. Darwin himself held the view increasing GIP to describe the rate of evolution. He proposes of phyletic gradualism, hypothesizing that most evolution that GIP measurement can be applied in both biology and occurs uniformly, gradually moulded by selective conditions. EC. Others were of a different opinion, and Eldredge and Gould As we can see from these examples, both phenotypic proposed the theory of punctuated equilibria [58]. According effects and genotypic effects have to be taken into account to this idea, evolution occurs through bursts of innovation when measuring the rate of evolution. followed by long periods of stasis, a major challenge to In artificial systems used for EC, the goal of evolution Darwin’s orthodoxy. is much more specific than in nature: to find the solutions to a given problem. The rate of evolution in EC, therefore, Definition 2. Rate of Evolution is a quantitative measure usually refers to the speed of solving a specific problem, of the changesobservableinanevolutionarysystemover for example, to the speed of fitness improvements or the generational (or otherwise appropriately defined) time- speed of approaching a fixed objective. The ability to define scales. explicit phenotypic fitness is one of the most distinguishing In biology, the rate of evolution has different defini- features that differentiate EC from natural evolution. For the measurement of the rate of evolution, however, it offers a tions and measures depending on the underlying objects examined, for instance, gene sequences, proteins, organisms, trap: to go entirely phenotypic, since in order to investigate and so forth. In molecular biology, the rate of evolution the performance of a computational model, the rate of usually describes the rate of mutants being preserved as evolution is mostly measured by the speed of fitness function advantageous, that is, those that can generate phenotypic improvements. Other ad hoc methods are also utilized in EC, improvements. This is observed by looking at the fixation of like the efficiency of algorithms and CPU time. alleles in genes. Biologists use the k /k ratio to measure the Another method, however at a deeper level than sim- a s rate of gene sequences evolution [59–62]. It is known that ple fitness function improvement, deserves mentioning: Bedau and Packard [64], for instance, propose a method some changes to a gene sequence may lead to differences in the amino acid sequence of an encoded protein while others for visualizing evolutionary adaptation. This method is will not, due to the degenerate code employed for trans- useful to identify and measure the capability of creating lation. Therefore, such a measure can be used to compare adaptation during evolutionary processes. It is based on two homologous protein-coding gene sequences of related calculating evolutionary activity statistics of components in species. The k /k ratio resulting from a measurement of an evolutionary system, such as the numbers of particular a s the number of nonsynonymous (amino acid) substitutions genes (or alleles) in each generation and the persistence per nonsynonymous site (k ) to the number of synonymous of these genes (or alleles) during evolution. During a substitutions per synonymous site (k ) characterizes the rate decade of extensive development, the notion of evolution- of evolution between these two sequences. Since k measures ary activity has been applied to various scales of genetic neutral evolution (without considering functional improve- components, including alleles, allele tokens, phenotypic equivalence classes of alleles, and whole genotypes, in ments under selection pressure), the k /k ratio reflects a s the amount of adaptive evolution against the background both artificial evolutionary systems and in the biosphere. amount of variation. Note that this is an approximation since In their more recent work, these authors emphasize two there are nonsynonymous changes in amino acid sequences aspects for evolutionary adaptation: the extent and the that do not change the function of the protein in which they intensity of evolutionary activity [65, 66]. The extent of appear. evolutionary activity refers to how much of an adaptive In case k /k > 1, fixation of nonsynonymous substitu- structure is present in an evolutionary system, while the a s tions is faster than that of synonymous substitutions, which intensity concerns the capability of generating new adaptive structures. The measures of cumulative evolutionary activity means that positive selection fixes amino acid changes faster than silent changes. Mostly, however, one finds k /k < 1, the and mean cumulative evolutionary activity characterize the a s case where deleterious substitutions are eliminated by puri- extent of a system’s evolutionary adaptation. On the other fying selection (negative selection), and the rate of fixation hand, new activity is a measure of the intensity of a of amino acid changes is smaller than the background rate of system’s evolutionary adaptation. Evolutionary activity can 4 Journal of Artificial Evolution and Applications be quantified and visualized during evolutionary adaptation. organismal evolution, since they provide abundant material Its derivative is the concentration of a component’s current for mutation and selection to generate new gene functions presence, and its second derivative can be argued to reflect in a modular way. By studying the recent nucleotide the rate of evolution at a particular time. Evolutionary substitutions in human evolution, Hawks et al. [73]find activity is also claimed to be a straightforward method for that, as a population becomes more adapted to its current studying evolvability [65]. The argument is that, since a environment, the rate of adaptive evolution slows down. system with high evolvability can create highly adaptive vari- However, a growing population size can provide the potential ation, the quantification of evolvability can be achieved by for rapid adaptive innovation. Thus, enlarging the popula- measuring the levels of extent and intensity of evolutionary tion size and changing environmental conditions can both activity. promote the rate of adaptive evolution. Kashtan et al. [74] confirm in a recent report that a varying environment can 1.3. Observations. It has been observed both in the pale- speed up evolution in an artificial evolutionary system. Other ontological record [67] and, more recently, through studies properties and techniques on the acceleration of evolution of molecular evolutionary systems [68] that the rate of have been also investigated in biology and computing. evolution in biological systems is greatly varying. At times This review discusses evolvability and methods for selective sweeps pass through a population that all but accelerating artificial evolution by drawing ideas from com- plex natural systems. Notions from biology are introduced wipe out certain less advantageous alleles, while at other times seemingly nothing happens in terms of evolutionary and their potential in designing new algorithms in EC is changes. Thus we can legitimately speak of an acceleration discussed. The review is organized along the work-flow of evolutionary algorithms. Section 2 starts with the character- of evolution under certain conditions, and of a slow-down under others. istics of populations; variation operations are investigated In EC, on the other hand, the state of the art can be in Section 3, separated into genotypic variation, phenotypic summarized by the observation that under most conditions, variation, and the transformations between them. Selection algorithms tend to show exponential decay in progress is discussed in Section 4, together with notions of fitness. The review concludes with a summary in Section 5. toward an optimum with often a painfully slow convergence for a large part of runs, or, alternatively, a premature con- vergence of the algorithm to the detriment of the produced 2. Population solution, resulting in a stagnation of the search algorithm before it has reached an acceptable outcome. This has been The general idea of EC is to adopt mechanisms of evolution realized to be related to the record dynamics shown in many from nature. In Darwin’s theory of evolution, both the natural and human systems [69, 70]. Record dynamics refers notion of variation and of natural selection are based on to the slow-down of records, for instance, in competition natural populations. However, populations simulated in a sports events, where after some time records become more computer are usually simplified from their natural counter- and more difficult to break, due to the unchanging human parts. A notable difference between natural and simulated physiology and the limitations this physiology imposes on population systems is that no identical individuals exist achieving certain targets. in a natural population, whereas this is allowed and most Contrast that with the world of natural evolution, where often the case with simulated populations. Tiny variances are there is always a way to beat previous opponents, and to considered an essential aspect of natural populations as they evolve in another direction that allows to increase fitness lead to the large diversity in natural evolution via amplifying in some unforeseen ways. Surely, the implicit definition of effects produced by selection. Hence, more details should be fitness plays an important role here, as it allows enormous taken into account also in computational populations that flexibility in achieving function. Further, the fact that the will ultimately allow a better differentiation of individuals. environment is permanently changing can be expected to be Because the representation chosen for individuals and the a key contributor to the evolvability in natural environments. size parameter of a population can affect the performance Finally, the ability of living tissue (an intentionally vague of a computational model, it is an essential step in EC to term) to assemble in a hierarchical fashion, starting from determine these features of the simulated population. atoms and molecules upward into ecosystems, provides building blocks and interactions of great richness that allows evolution to progress at different speeds, and notably to 2.1. Representation. The first step for setting up evolution accelerate under favorable conditions. with a population is to decide on the representation of A number of detailed observations on the factors that evolutionary individuals. Each individual should be encoded can accelerate evolution in the living world have been as a candidate solution to a given problem, which sub- made in the past. Simon [71] raises the “nearly completely sequently determines the search space of the algorithm. decomposable” property in multicellular organisms and Therefore, choosing a representation is important because proposes it to be an important property that can lead to it predicates the input to the search process that should faster fitness increases. In research on yeast genes, Gu et al. produce a satisfactory output. Here, we highlight a two [72] report rapid evolution of gene expression and regulatory biological mechanisms, a protection mechanism for robust divergence after gene duplication. Gene (and segmental) information preservation, and a communication mechanism duplication events contribute substantially to genomic and for information interaction between different molecules. Journal of Artificial Evolution and Applications 5 2.1.1. Robustness and Redundancy. Living systems may seem Table 1: Mechanisms responsible for creating redundancy and antiredundancy at the cellular level. (Adapted from Krakauer and wasteful and luxurious to computer scientists. The most Plotkin [86].) distinguishing aspects of biology compared to other natural sciences are complexity and diversity, which are indeed of Redundancy Antiredundancy central concern to biologists. In the face of cruel com- Overlapping reading petitive circumstances, organisms show great redundancy Gene duplication frames and resilience. Redundancy exists at different levels in Nonconservative codon natural organisms, including the genomic, transcriptomic, Neutral codon usage bias and phenotypic levels, all for the benefit of the robustness of the organism. — Gene silencing We adopt Wagner’s definition for robustness here. Polyploidy Haploidy Single regulatory element Multiple regulatory Definition 3 (robustness). The robustness of a biological or for n genes elements for n genes engineering system is its capability to continue functioning Chaperone and heat shock in the face of genetic or environmental perturbations [25]. proteins Checkpoint genes inducing Checkpoint genes In biology, the genome of an organism is defined as apoptosis promoting repair the information encoded in DNA sequences and inherited Telomerase induction Loss of telomerase from generation to generation. The double helix structure of DNA sequences itself is a form of protective redundancy Dominance Incomplete dominance of genetic information. Genomes carry genes and other Autophagy — noncoding DNA sequences. A gene is a string of base pairs mRNA surveillance — grouped by a function that is embodied in a protein or Bulk transmission Bottlenecks in transmission polypeptide (protein fragment). Noncoding DNA sequences, Molecular quality control — formerly called “junk DNA”, are not expressed as proteins, tRNA suppressor molecules — although they might be transcribed into RNA and involved Modularity — in manufacturing proteins or controlling that process. All in all, genes are only quite small a fraction of the entire Multiple organelle copies Single organelle copies genome [75], with more than 98% of the human genome, Serial metabolic pathways Parallel metabolic pathways for instance, being noncoding DNA sequences [76]. Fur- Uncorrelated gene Correlated gene expression thermore, even a gene sequence itself is divided into exons expression and introns, where exons directly determine the protein DNA error repair Loss of error repair amino acid sequence but introns do not. Nevertheless, these noncoding DNA sequences are not useless. Recent biological discoveries show that they play an important role in the ncRNA. About 98% of all transcribed sequences in humans regulation of gene transcription [77]. Regulation mecha- are of this type [84]. Although many of the functions of these nisms will be discussed later in Section 3.3.1.Wrenetal. noncoding sequences are unclear, the high complexity of the [78] find that tandem-repeat polymorphisms in genes are transcriptome hints at its importance in the mechanisms of quite common, and that such polymorphisms can enhance organizing gene expression in a robust way [85]. the ability of some genes to respond rapidly to fluctuating Krakauer and Plotkin [86] go further and propose the selection pressure. The mechanism of gene duplication will new concept of antiredundancy. In their opinion antiredun- be discussed in detail in Section 3.1.1. Moreover, diploid dancy emerges as does redundancy in cells, and natural organisms have two copies of each chromosome, one copy organisms would be able to modify the redundancy proper- inherited from each parent. Recent research has also found ties of genotypes during evolution. Table 1 shows a summary that a large number of DNA segments appear in more of observed mechanisms responsible for both redundancy than two copies. Copy Number Variations (CNVs) in human and antiredundancy at the cellular level. Mechanisms for and other mammalian genomes discovered lately account redundancy mask the phenotypic effect of mutations and for a substantial amount of genetic variation other than allow mutants to stay in populations, while mechanisms for single nucleotide polymorphisms (SNPs) [79–82]. CNVs and antiredundancy enhance the efficiency of local selection to SNPs are considered to substantially contribute to genotypic remove damaged components. variation, a phenomenon that will be discussed in detail later Going even further, we finally arrive at the phenotype: in Section 3.1.1. redundancy at the phenotypic level lies in an organism’s Further down the line toward the phenotype is the robustness against intrinsic or environmental changes. With transcriptome which describes the set of all transcribed low robustness, a species will gradually decline and finally go RNAs in cells. In the human transcriptome, the proportion of extinct due to lethal mutations because random mutations transcribed nonprotein-coding sequences is large and shows in the genome usually cause deleterious changes with a great complexity [83]. Substantially more DNA is transcribed potential to destroy the offspring. than is translated, and only a small proportion of mRNAs are It seems that robustness and evolvability have a con- translated into proteins. The rest is called noncoding RNA or tradictory relationship to each other. When a system has 6 Journal of Artificial Evolution and Applications high robustness in its genome, it can be tolerant to intrinsic redundancy into our algorithms to make them resilient or environmental changes, but that should leave it less against changes while improving adaptivity. Such capabilities evolvable, as variation would be masked, and vice versa. In certainly complicate the algorithms but may be worthwhile recent contributions, Wagner [50, 54] resolves this apparent if the resulting robustness can generate higher evolvabil- contradiction. He distinguishes robustness and evolvability ity when applying intense pressure to produce adaptive as quantities at both the genotypic and the phenotypic responses. Evolution might even be accelerated because levels. If one considers genotype, the more robust a genetic the system has a quick and robust reply to evolutionary sequence is, the less innovation this sequence will produce. pressures. With the growth of computational power available However, robustness and evolvability are characteristics of today ideas like these can be more easily explored than an entire system and if investigated at phenotypic level before. show a strong correlation. A system with high phenotypic robustness harbors a great number of “neutral” variations 2.1.2. Molecular Interaction. Natural living systems are that have no functional effects. These neutral variations remarkably diverse starting from so simple organisms as do not change phenotypic function during relatively static bacteria to highly complex creatures such as primates. evolutionary periods but may be able to generate adaptation This diversity is not the result of vastly different chemical later under certain genetic or environmental changes. Thus, constituents of organisms. In fact, many species carry out a system with high phenotypic robustness simply masks similar metabolic, cell division and replication processes changes but provides great potential for phenotypic inno- under similar assembly principles [98]. The differences that vation in the future, for example, if conditions change and distinguish species are caused by the arrangement and previously neutral changes suddenly have an effect. This is distribution of basic building blocks [99] and molecular the core of the argument that high robustness and high interactions contribute significantly to these organizational evolvability are in fact correlated in nature [54], and this mechanisms. has been supported in many subsequent research [87–91]. Molecular interactions in a cell happen between the Specifically, Draghi et al. [92]gofurtherand forfirsttime same type of molecules, such as protein-protein interactions, quantify the effects of robustness/neutrality on adaptation in or between different types of molecules, such as protein- an evolving population. They suggest a complex relationship DNA or RNA-protein interactions. Signals can also be between robustness and evolvability, which depends on the sent between and responded to by cells in multicellular topology of the genotype network. Their results indicate that organisms. Molecular interactions can be triggered by energy if the genotype space has no epistatic effects, a more robust supply, for example, in metabolic pathways, chains of interac- population will have less evolvability. With epistasis, on the tions catalyzed by enzymes, or triggered by external stimuli, other hand, they find a nonmonotonic relationship between for example, signaling pathways that enable communication robustness and evolvability, that is, evolvability is the highest through the cell membrane [100]. Proteins are not only at an intermediate level of robustness. a product enabling various organismal structures but also Redundancy is wide-spread in natural organisms as an work as control factors in various processes from the efficient protection mechanism against internal or environ- synthesis of a cell, metabolism, gene regulation, to sexual mental changes, whereas in EC models components that reproduction. do not seem to be immediately relevant are often consid- Metabolism is a key process to maintain the growth and ered superfluous. In recent years, however, representation reproduction of cells. The metabolic network of a cell is an redundancy has arisen as a by-product of computational elaborated set of numerous chemical reactions catalyzed by evolution and has attracted increasing interest from EC enzymes [101]. Different types and amounts of enzymes are researchers. produced according to different energy supplies, and these enzymes will determine different metabolic pathways by Definition 4 (representations redundancy). In genetic and their catalysis. In the process of gene expression, the function evolutionary algorithms, representations are redundant if the achieved can be controlled by molecular interactions. For number of genotypes exceeds the number of phenotypes instance, the process of how a parsimonious bacterium [93]. responds to food supplies during metabolism shows a simple genetic switch mediated by molecular interactions. Since Rothlauf and Goldberg [93] examine the effects of the metabolic pathways of bacteria are much simpler than redundant representations on the performance of an EC those of multicellular organisms, the regulation of gene system both theoretically and empirically and propose that expression is more easily understandable in bacteria. The redundant representations can increase the reliability and phenomenon of enzyme induction [22] describes the adapta- efficiency of EC models. Specifically in genetic programming, tion of a bacterium to material supplies by producing varying representation redundancy is usually identified as introns (or amounts of enzyme. What triggers this production and noneffective, neutral code) [1] in programs. Researchers have how does this mechanism work? The Jacob-Monod model investigated both the positive and negative effects of introns (shown in Figure 1) first described the regulation mechanism [94–97], and a positive relation between neutral code and of inhibiting or repressing genes by inhibitory proteins, evolvability in genetic programming has been suggested. The called repressors in bacteria. The binding of lactose to a important role of redundancy in evolvability has now been repressor enables the production of RNAs by removing the realized. We might, therefore, consider designing protective repressor from its binding sites on the gene sequence where Journal of Artificial Evolution and Applications 7 Now No lactose My landing Lactose ican’t present present site is blocked! bind! RNA polymerase Lactose Repressor RNA Repressor polymerase No RNA made Makes RNA Lac gene Lac gene (a) (b) Figure 1: The genetic switch in the Jacob-Monod model. A specific repressor protein acts as a switch. When it binds to a DNA site near the gene encoding beta-galactosidase, the RNA polymerase protein cannot bind nor can it synthesize RNA from the gene. The gene is turned off. When lactose is present, it binds to the repressor and keeps it from the DNA site. The gene turns on. (Adapted from Kirschner and Gerhart [22].) RNA polymerase can bind. However, this is not a simple makes eukaryotic cells well conserved but enormously adap- on-off switch model. The continuity lies in the binding tive to generate new phenotypes in changing environments duration which determines the rate of protein synthesis. [103]. Therefore, if more sugar is absorbed during metabolism, Computational models have already been used to analyze more protein is synthesized by RNA translation. This simple and understand complex multi-input/output and higher- sugar metabolism model captures the mechanism of how order signaling systems have been examined in bioinfor- a repressor affects gene function. The enzyme here works matics [104]. In contrast, current EC models are mostly as a trigger for the protein synthesis process under various limited to representing evolutionary material based on the molecular interactions. In addition, most enzyme effects infrastructure of natural organisms, while disregarding the are sensitive to ambient temperature [102], which is an vast potential of interaction mechanisms for regulation and important parameter to control metabolic interactions. signaling at both the molecular and cellular levels. The Signaling and cellular responses to signals are complex. absence of such mechanisms in EC, however, points to These responses are controlled by a plethora of positive and significant research opportunities in this area. negative feedback loops. The presence of feedback compli- cates the simple picture of a linear pathway but is an essential 2.2. Population Size. After the encoding of an individual part of the signaling process [98]. This makes signaling is determined, a population is set up. Several features of pathways involving molecular or cellular communication a a computational population are tightly connected to its network-like structure, with complex regulatory processes evolutionary capabilities, the most relevant of which is at work. The cellular infrastructure of eukaryotic organisms population size itself. is only a few times larger than that of bacteria, but the In nature, different species have different population complexity of their signaling network control differs greatly, sizes, a characteristic that plays an important role in evolu- by orders of magnitude. The linkage between various parts tion. In the living world it is common that smaller groups of the gene expression apparatus in eukaryotic organisms constituting species evolve faster, though smaller groups have is weakened by a far less precisely defined control than a higher probability of becoming extinct, while species with that found in prokaryotic cells [47]. For instance, geometric larger populations evolve slower and can stay unchanged for requirements for binding sites are significantly relaxed in relatively long periods. However, neither a small nor a large eukaryotic gene regulation. A repressor does not have to population size is unconditionally beneficial in evolution. bind at the exact position of a target but needs only to The relation between them should be understood in different bind in the neighborhood. By lowering constraints for scenarios. cooperation, such a weak linkage also enables potential The study of population genetics was formulated by interactions between different gene sequences. Signaling Fisher [105], Haldane [106], and Wright [107]. It focuses between cells is possible only after a sufficiently large number on gene frequency changes in populations under the effects of repressors participate simultaneously. A single signal may of natural selection, mutation, genetic drift, and population incur a very complex response [49]. Allosteric proteins, size fluctuation. In this field, scientists have examined which have multiple sites for interaction, also make gene the role of population size in molecular evolution using expression more flexible because they have different sites mathematical analysis. The rate of molecular evolution for different functions. Regulatory decisions on which genes is usually measured by the nonsynonymous to synony- are transcribed when, where, and under what circumstances mous substitution ratio k /k , discussed in Section 1.2. a s 8 Journal of Artificial Evolution and Applications Decades ago Kimura [108] proposed a strong dependency populations will also favor evolving robustness by increasing of the rate of molecular evolution on population size. genetic drift pressure and a buffering mechanism of hiding More recently, Gillespie [109, 110] has conjectured that mutations from being reduced by selection. This hypothesis there is only a very weak dependency on population size. is supported by Elena et al. [117]. Among the different Somewhat in the middle between these opinions, Ohta authors, there is agreement that the effect of population size, [111] finds population size to be related to the rate of either large or small, varies in different models. evolution under particular assumptions regarding mutation In general, the size of populations in EC is orders of types. The nearly neutral theory of molecular evolution magnitude lower than the size of populations of many proposed by Kimura and Ohta [112, 113] predicts that naturally occurring species, especially those of simpler there is a substantial number of nearly neutral mutations organisms like bacteria. The commonly adopted population (including slightly deleterious and slightly advantageous size in EC varies from tens to thousands, with a few ones) in molecular evolution, and that these contribute exceptions. Genetic Programming generally uses relatively to evolution by providing potential for future phenotypic large populations, due to the more complex and nonlinear innovation. Ohta [111] predicts that population size affects fitness landscape than can be found in other branches of the rate of evolution under various mutation scenarios. If the EC family. However, the size of GP populations run is most mutations are deleterious, a smaller population can limited by resource constraints in the range of hundreds evolve faster, because the chance of a slightly deleterious of thousands. Some of these algorithms have to run on mutant being favored by selection is greater within a smaller parallel machines or on GPUs, since the evaluation of a large population and these nearly neutral mutations bring genetic population of individuals requires enormous computational variation and may further trigger phenotypic innovation. In power (see, e.g., [118–120]). An order of magnitude like contrast, if mutations are mostly advantageous, the rate of that of humankind, a billion individuals, is unheard of in evolution in a larger population is greater. If most mutations EC approaches, which already points to a vast potential for are neutral, the evolution rate is nearly independent of doing research on EC in the future. A whole landscape of EC population size. Since in general random mutations are more methods might emerge with populations that are large. deleterious than advantageous in natural systems, species As a result of the use of small population sizes in the EC with a small population size usually evolve faster. community, efforts have been dedicated to the optimization A number of studies focus on testing the relation between of population size [121], since a high correlation between population size and evolution rate by using comparisons. population size and the performance of an EC algorithm Island endemic species usually have small population sizes is presumed. The challenge is that adapting population because they are restricted to a limited geographical region. size is problem-specific and to date it is still unclear how Woolfit and Bromham [114] study species on islands in to estimate the relation among various EC parameters. In support of the effect of population size on the rate of general, current work on this topic concentrates on two tasks: molecular evolution. They compare island endemic species (i) initializing a proper population size prior to a run, and to closely related species on a nearby mainland and find (ii) adjusting population size during a run. Most theoretical that island endemic species have a significantly higher work on population size initialization is based on Goldberg’s nonsynonymous to synonymous nucleotide substitution component decomposition approach and the notion of ratio than their counterparts on the mainland. This result Building Blocks [122, 123]. With many other publications, indicates that a decrease in the population size will lead these contributions propose to choose the population size to an increase in the rate of evolution. Wright et al. [115] according to the “hardness” of a specific problem. They study tropical species which are generally regarded to have state a general principle in setting population size: the more a rapid molecular evolution rate due to several factors such difficult a problem is, the more diversity is required and the as latitude and climate. It is believed that tropical organisms larger the population should be. possess great species richness and dynamics with small but In the meantime, it has been found that even for a highly diverse populations [116]. However, there are also specific problem the requirements for population size may exceptions in that increasing population size can accelerate differ during different stages of evolution. As a result, evolution as well. By studying the recent rapid molecular empirical methods for adjusting population size during a evolution in human genomes, Hawks et al. [73] suggest that run have been proposed, such as the Genetic Algorithm with if a population is highly adapted to a current environment, Variable Population Size (GAVaPS) proposed by Arabas et evolution will become stagnant. Under these circumstances a al. [124], the parameter-less GA by Harik and Lobo [125], growing population size can provide the potential for rapid the Adaptive Population size Genetic Algorithm (APGA) by adaptive innovation. Thus, enlarging the population size Back et al. [126], and the Population Resizing on Fitness under chaotic environments can promote the rate of adaptive Improvement GA (PRoFIGA) by Eiben et al. [127]. However, evolution. mechanisms for dynamically adjusting population size in Population size is also involved in research on genome EC are much simpler than those found in nature, in that a robustness. Visser et al. [85] postulate that the population fluctuating population size still has little to do with mutation size should be sufficiently large for selection to be effective and selection patterns in different evolutionary stages. This to evolve the robustness of a system. Small populations relation requires further exploration as it seems to be a have difficulty to achieve this robustness. In a different promising indicator for population size adjustment during study Krakauer and Plotkin [86] find, however, that small arun. Journal of Artificial Evolution and Applications 9 3. Variation What triggers mutation and what is the relation between mutation and selection? Does selection pressure indeed Mutation and recombination operators are a main aspect generate new mutations or simply allow existing mutants of evolvability, since they generate the necessary variation to be fixed faster than before? Research on mutation under among individuals that later can be acted on by cumulative selection has received wide interest since Darwin’s time, but selection processes. Due to the complex mapping process controversies have arisen regarding the effect of selection from genotypic to morphological level in biology, genotypic pressure on mutation, and different models have been and phenotypic variation will be discussed separately. proposed in the meantime [129]. It is now believed that it is impossible to separate any form of mutation from the effect of selection. In order to investigate “directed” 3.1. Genotypic Variation. Genotypic variation generally mutation pathways Roth and Andersson [130]define adap- means changes to DNA sequences in both protein-coding tive mutations as fitter mutations that arise under selective and noncoding regions in the form of point mutation and conditions. In subsequent work [131–133], they propose a gene rearrangement. Gene sequences are highly conserved gene duplication-amplification model to study the mutage- against lethal changes that would likely lead to destructive nesis stimulated by enhancement of selection. In addition, consequences otherwise because a tiny mutant at the genetic a recent study by Weinreich et al. [134] on the effects levelcan causeagreatchangeinfunction[22]. In contrast, of Darwinian selection on random mutation argues that changes to the regulatory or noncoding part of sequences are environmentalselection canmakesomemultistep mutation considered more able to increase adaptability and plasticity pathways unaccessible. By studying “five point mutations” of a system. In this section, we will discuss the general form in a lactamase allele that can increase bacterial resistance of mutation first and then gene duplication as the most to an antibiotic, several mutation pathways are in principle important form of rearrangement, followed by a comparison possible for these mutations. After calculating the different between point mutation and gene rearrangement. probabilities of these pathways, their experimental results show that under intramolecular interactions that increase the fitness of proteins, only a small number of pathways are 3.1.1. Mutation. Although there can be many definitions of really accessible. This is quite an interesting result because mutation, we here adopt one that emphasizes the primary mutations might be channeled by some unknown fitness- difference to recombination, namely, that works with mate- increasing principle(s) and the resulting proteins might rial from just one individual organism. be reproducible and even predictable. These feedback and interaction mechanisms may reduce the harm that mutations Definition 5 (mutation). Mutation is the process that creates could bring to an organism. This point of view also conforms new genetic material from the addition or multiplication to Kirschner and Gerhart’s definition of evolvability [47], of stochasticity in various forms to some original genetic which they define as “the ability to reduce the potential material of an organism. deleterious mutations and the ability to reduce the number of mutations needed to produce phenotypically novel traits”. If mutations can be channeled, fewer changes might be needed Point Mutation. Searching for the essential driving force of evolution has been a central topic in evolutionary biology. to generate a required adaptation and, therefore, evolvability Since Darwin declared that natural selection is the main force would be improved by this reduction in cost of mutations. In EC, mutation is regarded as an important exploratory of evolution, controversies have arisen on different aspects of this explanation. In modern biology, the two main schools operator. Artificial evolutionary search should be good at of thought are selectionism and neutralism [128]. Some both exploring suitable genetic novelty and maintaining suc- scientists argue that genotypic variation is maintained by cessive improvements. Holland [7] discusses this principle selection, which is the central perspective of neo-Darwinians. as the tension between “exploration” and “exploitation”. The mutation rate is important to keep this balance, and it has Other evolutionists insist that high genotypic variation can be explained as a result of neutral mutations. In either case, already been studied as an evolvable parameter contributing mutation is accepted as a major mechanism to generate to evolvability. Bedau et al. [51, 135]divideevolutionary adaptation conceptually into two stages: the novelty stage, genotypic variation. Mutation canhappenanywhereonaDNAsequence, where an evolving system enhances its adaptability against that is, in either coding or noncoding regions, and may a changing environment, and the memory stage, where the consequently cause functional, regulatory, or structural evolving system is building up this adaptability through changes, or no changes at all. The neutralist hypothesis is incremental improvements. By providing a simple two- that the majority of observed sequence variation stored in the dimensional model, Bedau et al. postulate that the mutation population is neutral. This is due to the compensating mech- rate should increase during the novelty stage and decrease anisms of biological systems [128]. Most new mutations are during the memory stage. This fluctuation of mutation rates is able to keep the balance between evolutionary novelty deleterious, a few are advantageous, and many are neutral. However, most of the extant polymorphism observed in and memory and thus increases the evolvability of adaptive populations is the neutral variants. Deleterious mutations systems. have been purged and advantageous mutations have swept However, compared to natural evolutionary systems, through the populations. genotypic variation in computational systems is somewhat 10 Journal of Artificial Evolution and Applications different loci, with the consequence that duplicate genes appear in one chromosome while the other turns out to Crossover contain pseudogenes. Retroposition happens when an mRNA is retrotranscribed into a complementary DNA (cDNA) and then inserted into the original genome. Besides such gene duplication, duplication at other scales in cells has been discovered recently [138, 140], including segmental duplication and whole-genome duplication. Here, we (a) only consider gene duplication. The main products of gene duplication are called paralogous genes, a type of homologous genes. Homologous genes have two main categories, paralogs Transcription and RNA splicing mRNA and orthologs. Paralogs are results of gene duplication and code for proteins with different functions. Orthologs are the Reverse transcription products of speciation events and the proteins they code for serve similar functions. cDNA Once a gene duplication has occurred, a complex fixation Randomly insert process on the duplicate genes takes place. Purifying selection into genome and gene conversion are the main pressures affecting the sur- vival of duplicate genes [141]. Most duplicate genes become Parental gene pseudogenes after one or more mutations disable them in different chromosome and no promoting function is yielded. However, multiple (b) copies of identical genes can, after duplication, promote functional redundancy against fatal changes. The process Figure 2: Two common modes of gene duplication. (a) Unequal of pseudogenization is reported to occur in the early stages crossover, which results in a recombination event in which the two of a rapid evolution [142] process, with evidence of many recombining sites lie at nonidentical locations in the two parental pseudogenes found in the human genome. Other duplicate DNA molecules. (b) Retroposition, which occurs when a message genes are changed by selection pressure and functional RNA(mRNA)isretrotranscribedtocomplementary DNA(cDNA) divergence. Subfunctionalization and neofunctionalization and then inserted into the genome. Squares represent exons and bold lines represent introns. (Adapted from Zhang [139].) are the two main mechanisms of functional divergence [139]. In subfunctionalization of two gene duplicates, shown in Figure 3,eachcopyadoptsadifferent aspect of the function arbitrary and not as adaptive. First, the fixation process of the original gene. Both copies will be stably maintained of mutations is not simulated appropriately in most EC because both aspects of the function are indispensable. algorithms, because all changes to individual sequences are Subfunctionalization leads to functional specialization by mostly translated into phenotypic properties. Recovery or dividing multifunctional genes once the newly emerged repair mechanisms are usually not applied to individuals genes perform better. Alternatively, some relatively new suffering deleterious mutations, which make those individ- function can also evolve after gene duplication [143], and this uals unfavored during the selection process. Second, the process is called neofunctionlization.Thishas been termed selection-driven mutation pathways found in natural systems the Dykhuizen-Hartl Effect [144] earlier, where a random are an interesting direction to explore for computational mutation is preserved in the duplicated gene by reducing models and should be considered in future research in EC. selection pressure due to functional redundancy that results from gene duplication. Such mutations may accumulate and Gene Duplication. Gene duplication is an important mech- induce a genetic function change depending on conditions anism creating new genes and new genetic subsystems. of the (dynamic) environment. New adaptive functions may This mechanism has been recognized to generate abundant thus be generated and later preserved during evolution. By genetic material and contributes substantially to biological possibly creating novel functions and allowing evolution under fewer constraints, neofunctionlization is an important evolution [136]. A large number of duplicate genes have been discovered to exist in vertebrate genomes [137], and consequence of gene duplication. a repeated number of whole genome duplications have In brief, the mechanism of gene duplication contributes been established as key events in evolutionary history [138]. substantially to genomic and organismal evolution. It pro- In modern biology, gene duplication and its subsequent vides abundant material for mutation and selection and function-specialized divergence are widely believed to be a allows to specialize function or generate completely new major reason for functional novelty. functions. The acceleration of protein sequence evolution Gene duplication is usually generated by unequal after gene duplication has recently been confirmed in research on yeast genes by Gu et al. [72]. The authors crossover or retroposition [139] (see Figure 2). Unequal crossover is similar to but different from normal crossover use an additive expression distance between duplicate genes that occurs when two chromosomes exchange a propor- to measure the rate of expression divergence, and rapid tion of DNA at the same locus in base pair sequences. evolution of gene expression as well as regulatory divergence Unequal crossover happens if this exchange occurs in after gene duplication is observed. Journal of Artificial Evolution and Applications 11 A1 A2 evolvability, possibly even more than simple point mutations Expressed in T1 and T2 [153]. Recent development of technology has now facilitated the shift in focus from a locus-based analysis to a genome- wide assessment of genotypic variation [79, 154]. Gene duplication Genetic rearrangements rather than point mutations can maintain the connective information carried by gene sequences. Because genes form networks of functional control, rearrangement is better able to preserve internal Complementary structures. Genetic changes are highly constrained by gene degenerate mutations sequences and gene rearrangements occur far more fre- quently than point mutations. The ubiquity of Copy Number Variations (CNVs) has been realized recently in mammalian genomes by different Expressed in T1 Expressed in T2 groups of biologists, such as Sebat et al. [81], Iafrate et al. [80], and Tuzun et al. [82]. CNV is regarded as a Figure 3: Division of expression after gene duplication. Squares predominant type of genotypic variation leading to vast represent genes, closed ovals represent cis-acting elements that phenotypic diversity in mammalia. CNVs show that large regulate gene transcription, and open ovals represent deactivated segments of DNA, with sizes from thousand to millions of cis-elements. Consider a gene that is expressed in tissues T1 and T2, base pairs, can vary in copy number of genes. This variation with a cis-acting regulatory element A1 controlling the expression in T1 and A2 controlling the expression in T2. Following gene canleadtoprotein dosage differentiation in the expression duplication, one daughter gene might lose the A1 element whereas of genes, and CNV is therefore regarded as being responsible the other gene might lose A2, so that each is expressed in only one for a significant proportion of phenotypic variation [79]. of the two tissues. (Adapted from Zhang [139].) The mechanisms that create CNV have not yet been clearly understood, but some hypotheses have been proposed in the literature. Fredman et al. [155]and Shaw andLupski[156] propose that CNV might be the result of large segmental gene One key idea how gene duplication can speed up duplications or nonhomologous recombination events. evolution is Altenberg’s constructional selection [145, 146]. Recent bioinformatics research uses statistical and com- The idea is that gene duplication enriches the genome with putational tools to analyze chromosomal evolution by a genes that are good at increasing fitness when duplicated. comparison of genome-rearrangements between sequences This is a second-order effect that can be considered to of related species [157]. Although the biochemical mech- contribute to evolvability. For a more general review, see anisms of gene rearrangement are still far from being [147]. fully understood, we believe it is time to start using such In summary, the mechanism of gene duplication can rearrangement operations in computational models in EC. considerably increase evolvability of a system by reducing the Particularly, the recent discovery of CNVs requires attention cost of mutations. In EC, the idea of using gene duplication by computer scientists, in order to achieve similar benefits in and deletion operators was proposed some time ago. Those EC. operators are in general based on the method of variable- length genotypes and are executed with predefined dupli- 3.1.2. Recombination. Recombination has been considered cation or deletion probability [46, 148–151]. Unfortunately, both as an exploratory and as a stabilizing operation in so far only application-oriented work has appeared with biology and in EC. Here we emphasize the origin of the different representations [152], and a common framework genetic material being used for new combinations. Due to for this concept is missing. More details of gene duplication the size of search spaces, both effects are possible. in biology should be taken into account to benefit compu- tational evolution. In particular, the question of how gene Definition 6 (recombination). Recombination is a process duplication reduces the limitations of mutation and selection that generates combinations of existing genetic material from and in the process promotes evolvability needs to be studied. a multitude of organismic sources. Is there a way to implement functional specialization and innovation through gene duplication in EC? Recombination is regarded as an important force shaping genomes and phenotypes. Since some highly efficient and Point Mutation versus Gene Rearrangement. A point muta- accurate computational methods can be used in biology, tion occurs when a base on a DNA sequence is changed into analysis of gene recombination has made much progress by another base at the same locus. Gene rearrangement is a way of comparing aligned genome sequences. These com- change in the order of a DNA sequence on a chromosome. parisons facilitate a better understanding of several aspects This change can be an inversion, translocation, addition, or of genetic and evolutionary biology, notably genotypic and deletion of genes. Earlier research focused mostly on Single phenotypic variation and genome structures [158]. Nucleotide Polymorphisms (SNPs) in genomes due to the Recombination exchanges genetic material between two enormous complexity of genetic sequence analysis, but gene DNA sequences swapping strands between one or multiple rearrangements have always been believed to contribute to crossover points. Recombination can occur on homologous 12 Journal of Artificial Evolution and Applications or nonhomologous sequences. The former is more promi- value. Different from natural recombination mechanisms, nent in research because it is more common and efficient most adaptive recombination rate proposals simply react to in generating adaptation in nature. Generally, research on the current status of the search, in order to escape from recombination focuses on prevalent eukaryotic organisms local optima. However, rate adaptation in biology is much rather than prokaryotes, which do not have the sex property. more complex and suggests other models for computation. Unequal crossover is fairly rare and may lead to duplication For instance, the rate may vary among different individ- or loss of some genes (discussed in Section 3.1.1) and other uals or in different modules serving subfunctions in the results [159]. Combination events can take place between genome. Such function-specific recombination rates could different gene sequences, as in intergenic recombination,or also consider the method of “compartmentalization” for between alleles on the same gene sequence, as in intragenic modularity (Section 3.2.2). The notion of epistatic clustering recombination [158]. Despite various forms of recombina- in contributing to evolution of evolvability has recently tion, their outcome is crossover at one or multiple points and been studied [170]. Genetic linkage patterns between dif- a swapping of fragments of genetic sequences. ferent loci are claimed to affect recombination rates, and Kondrashov [160]proposesinhis deterministic mutation the simultaneous optimization of different recombination hypothesis that sexual recombination can remove deleterious rates on different traits would be realized by a method genes. It is generally believed that most nonneutral muta- called epistatic clustering. Evolvability would be improved tions are slightly deleterious. Kondrashov suggests that sexual through coevolution of trait clustering and recombination recombination can distinguish individuals with cumulative, mechanisms. slightly deleterious mutations, and the ensuing selection pressure can eventually remove those disadvantaged muta- 3.2. Phenotypic Variation. As mentioned in Section 2.1.2, tions. Further, Hadany and Beker [161] strengthen this despite their vast phenotypic differences, metabolic processes perspective in their research on the evolution of obligatory and cell structures in bacteria and humans are quite similar sex. Their model supports that sexual recombination offers [22]. What, then, makes humans so different morpho- both short-term and long-term advantages to sexually logically from other organisms? It is the regulation and reproducing individuals and has a positive effect on the reuse of these structural elements in different combina- physiological fitness of an organism. tions that generates different complex phenotypic outcomes. The rate of recombination can significantly affect the rate Unfortunately, the relation between genotypic variation and of adaptation. It is usually higher than the rate of mutation, phenotypic variation is still not fully clarified in current which implies that recombination introduces much less biological opinion. Since selection acts on phenotypes rather lethality to an evolutionary population than mutation. than on genotypes, phenotypic variation should be used to Instead, it advances evolution remarkably by stabilizing explain the immense diversity among organisms. Here, we adaptive traits from parents to offspring. This contributes discuss several aspects of phenotypic variation. We leave the to evolvability in the same way as other purifying selection discussion of the mapping process between genotype and does because the bounds on epistatic interactions between phenotype that controls the direction of phenotypic changes loci get progressively strengthened through selection over resulting from genotypic variation to Section 3.3.1. generations. By drawing a recombination map of the human genome, Kong et al. [162] discovered that recombination 3.2.1. Conservation and Relaxation. According to Kirschner rates vary in different regions of the genome. This variation and Gerhart, evolution possesses two important features: is duetosuchfunctionalfeaturesasgenedensity,other conservation at the molecular level and relaxation at the gene properties, and frequency of sequence repetitions. anatomical and physiological level [22]. By conservation it Recombination rates are also different in autosomes between is meant that the genetic components of organisms tend to different sexes. Recombination contributes to producing maintain relatively stable structures; relaxation refers to the both genotypic and phenotypic variation and is able to less constrained phenotypic diversification of organisms. The repair DNA double strand breaks. Sexual reproduction is an authors state that conservation on the genotypic level reduces important outcome of recombination. the constraints on the phenotypic level. In EC, recombination operations are considered an In Darwin’s evolutionary theory, all organisms have essential search strategy. Chromosome coding is much more evolved from the same ancestor. After primal initialization flexible in computation than in nature, and thus, various and evolution, genetic structures of organisms are highly recombination techniques have been proposed and studied, conserved during the course of billions of years [101]. including double-parent and multiparent crossover [163], This can well explain why the number of human genes is fixed-length chromosome and variable-length chromosome only a few times that of bacterial genomes but significant crossover [164, 165], and homologous and nonhomologous anatomical and behavioral differences exist between them. crossover [166–168]. High recombination rates are usually The surprisingly small number of genes in humans and other also adopted in computation because of its perceived effi- complex organisms demonstrates that the great diversity ciency in generating beneficial genetic and phenotypic varia- and complexity at the anatomical and physiological levels tion. Elsewhere, adaptive recombination rates are proposed have to rely on and organize/reuse limited genetic material. to strike a balance between exploration and exploitation When certain organisms need to improve their adaptivity [169]. In most of these adaptive recombination rate schemes, in order to survive in a new environment, the regulation modification of recombination rates is based on fitness system only has to recombine existing mechanisms for the Journal of Artificial Evolution and Applications 13 generation of adaptive functions, which requires little or Definition 7 (modularity). In a complex system, modularity no new genetic material [47]. Not only are gene sequences refers to the property that a loose horizontal coupling exits highly conserved, but also the core processes of coordination between the entities at the same level of this system [172]. of the genetic material are well conserved since the time they Simon [71, 173] further defines that “a system is nearly initially emerged [22]. These conserved core processes are decomposable if it consists of a hierarchy of components, such used repeatedly for different purposes and functions under that, at any level of the hierarchy, the rate of interaction different circumstances, at different times, with different within components at that level is much higher than the rates genetic material. The Baldwin Effect [171] explains that of interactions between different components”. Although this phenotypic variation is not generated out of the blue but “Near Decomposability (ND)” is attributed to a vertical through regulation of existing components in organisms: separation while modularity describes the separable property mutation simply stabilizes and extends what has already of components horizontally at the same level, they seem existed to improve somatic adaptability towards external closely related in that they both describe how a complex stimulations. system is decomposed into subsystems. This conservation mechanism can efficiently prevent Themodularitypropertyofgenotype-phenotypemap- lethal changes in genotypes and is an economic method pings has been extensively studied in gene expression. It to increase the adaptability of organisms. New material is reduces harmful pleiotropic effects of gene expression and not needed to adapt to changing environments, but few can lead to adaptive phenotypic variation. Pleiotropy is a modifications will suffice. general property of genotypic variation, expressing the fact Functional innovation is heavily constrained due to that one change at the genetic level can cause a multitude molecular interactions among various genetic components of functional changes at the phenotypic level. Pleiotropy that are involved to produce a specific trait. If the partic- can generate both advantageous and disadvantageous results. ipation of more genetic components is needed, it becomes Pleiotropy can sometimes generate unexpectedly improved harder for functions to change. In fact, relatively little genetic functions but can also be harmful or even fatal to evolution- material is required to generate all proteins of organisms. ary systems [174]. Since a gene can affect multiple functions, Under selection pressure from a changing environment, optimizing one particular function at the phenotypic level organisms have to yield adaptive phenotypic traits to survive, inevitably incurs side-effects on other functions. Bonner however, and the highly conserved core processes mentioned [175] proposes the notion of “gene nets” by grouping above are used repeatedly to generate new cooperation gene actions and their products into discrete units during among the conserved genetic material, bringing about fitter evolution. In general, for a given organism, the mapping function and behavior. Relying more on the combinatorics of from genotype to phenotype can be divided into modules components is equivalent to relaxing phenotypic variation. such that the sets of genes in one module only affect the The relaxation on phenotypic variation has been high- functions in that same module. The mapping is therefore lighted as the notion of “deconstraint” in Kirschner and decomposed into groups of independent “submappings”. Gerhart’s [47] research on evolvability which studies the Bonner finds that the phenomenon of gene nets becomes mapping from genotype to phenotype. Enhancing pheno- increasingly prevalent as organisms become more com- typic variability under changing environmental conditions plex. Wagner and Altenberg [45]investigate modularity allows nature more evolvability. Not only can deleterious in genotype-phenotype mappings from both perspectives, changes be avoided, but also nonlethal genetic and pheno- biology and EC. They interpret modularity as a means for typic variation is indeed the material from which innovation dividing phenotypic traits into different “compartments” to can be generated. reduce interference among different optimization modules. Turning again to EC: what is the role of conservation With such modularity, optimization of a function in one and relaxation in EC? First, an economic use of genomes module has no effect on functions in other modules. As a or building blocks can help to conserve genetic information. result, pleiotropy can be confined to a known set of functions Second, it can be assumed that by reducing the constraints during evolution. Figure 4 shows a simple example of this on changes to a phenotype the exploratory capability of idea of modular separation. a computational system to find better solutions can be Wagner and Altenberg [45] further propose that mod- enhanced. How such a process can be implemented in actual ularity results from evolutionary modifications in natural systems is presently unknown, but a worthwhile line of organisms. In their view, the evolution of modularity follows inquiry. two mechanisms, dissociation and integration. Dissociation is the suppression of pleiotropic effects by disconnecting 3.2.2. Modularity. Modularity is a widespread structural interactions between different modules, while integration is property of complex systems. It has attracted considerable realized by strengthening of pleiotropic connections among interest from studies of both natural and artificial evolution- traits in the same modules. Both mechanisms are driven by ary systems and is regarded as strongly related to evolvability selection pressure. [45] and the acceleration of evolution [71]. Modularity exists at various levels, for example, at the Thus, modularity can be conceptualized as an evolu- level of gene expression or embryonic development. Here we tionary mechanism to promote evolvability. It reduces the adopt the definition of modularity proposed by Simon [172] interdependence of disjoint components and consequently in his research on hierarchies in complex systems. reduces the chance of pleiotropic damage by mutation [47]. 14 Journal of Artificial Evolution and Applications F1 F2 and reusing relatively simple genotypic material will be a major force in shaping complex phenotypes also in EC. C1 C2 3.2.3. Facilitated Variation. Kirschner and Gerhart [22] BC emphasize that variation is much less random at the phenotypic level of organisms than at the genotypic level, where genetic mutations show considerable randomness. Since phenotypic variation should be favored by selection via modifying existing evolutionary components, they call this variation facilitated. Kirschner and Gerhart summarize three principles of facilitated variation. It serves (i) to reduce lethal pleiotropic effects, (ii) to increase phenotypic variation in light of G1 G2 G3 G4 G5 G6 a given number of genetic changes, and (iii) to improve genetic diversity in evolutionary populations (by reducing lethality). Evolution is not so much affected by the content of Figure 4: Example of a modular representation. Complexes C1 = genetic and protein structures but by regulation capabilities {A, B, C, D} and C2 = {E, F, G} serve to functions F1 and F2. to organize and reuse these functional parts and to decide Each character complex has a primary function, F1 for C1 and F2 the targets of such regulation. The core processes instead for C2. Only weak influences exist of C1 on F2 and vice versa. The are conserved being built in a special way, only to be linked genetic representation is modular because the pleiotropic effects of together under new circumstances like time, place, and the genes M1 = {G1, G2, G3} have primarily pleiotropic effects on the number of genetic components that may participate in the characters in C1 and M2 = {G4, G5, G6} on the characters in generating new phenotypic variation. It is clear that only complex C2. There are more pleiotropic effects on the characters adaptive phenotypic variation can be maintained during within each complex than between them. (Adapted from Wagner evolution, and the relevant product proteins mostly will have and Altenberg [45].) multiple functions for various adaptive requirements under selection. Variation in EC systems seems to be more random than It allows genotypic variation and selection to affect separate that in natural evolution. Despite the limitations in recogniz- features in a complex system and to evolve various functions ing these phenomena in biology, we should explore methods without interference [176]. Subsystems as part of an entire to reduce randomness in computational models in order to system can evolve faster to optimize their local subfunc- make evolving processes more “intelligent” and to facilitate tions individually, by decreasing crosstalk between genetic the discovery of good solutions. Some steps have been taken changes. In a study of encoding schemes in EC by Kazadi in EC literature in this light. Researchers have designed et al. [177], a compartment is defined similar to a module more sophisticated techniques to improve the adaptation in the genotype-phenotype mapping, and such compart- of algorithms. One idea was developed for Evolutionary mentalization at different levels is claimed to contribute to Strategies first and later applied to other branches in EC the acceleration of evolution. In RNA research, Manrubia [179–182]. Further, the evolution of “smarter” operators for and Briones [178] propose that the increase of molecule EC in a higher-level evolutionary process has been examined length and subsequent increase in functional complexity in metaevolution [183–186]. Amorerecentcontribution could be mediated by modular evolution. They find that looking at the effect of changing environments on variation short replicating RNA sequences with a small population in a computational framework has been the GA of Parter size can be assembled in a modular way and can create et al. [187]. complex multifunctional molecules faster than conventional evolution of complex individuals toward multiple optima. Modularity in general has been widely used in computer 3.3. Transformation from Genotype to Phenotype. It is at science and engineering by subdividing complex entities into the intersection between genotypes and phenotypes where smaller components to yield higher computational efficiency most of the mechanisms reside that allow for facilitated and has similarly played a key role in EC from the outset. variation. A subject of much study both in natural and In fact, it can be argued that the building block hypothesis artificial systems has been the genotype-phenotype map. is at its core an argument about modularity. However, In recent years, epigenetic effects, long suspected to have complex genotype-phenotype maps and other mechanisms enormous influence on the final expression of the phenotype, (like growth and development) to generate modularity in have assumed center stage in biology. Epigenetics [188]is EC are relative newcomers and it is expected that studying a rapidly developing and prominent research topic, both these mechanisms in biology will result in more sophisticated in relation to the development of healthy phenotypes as means to produce modularity in EC. Since modularity is the well as those who show deficiencies. This will constitute most universal property of phenotypes in natural systems, the second part of this section. Finally, epigenetics and a there is ample ground to expect that the economic and consequence of the amplified power of expression regulation sophisticated mechanisms used by Nature through regulating through epigenetics are the mechanisms of development Journal of Artificial Evolution and Applications 15 of multicellular organisms. These are discussed in the last Duplication Transcription Translation DNA RNA Protein subsection here, concluding the transformation of genotypes into phenotypes. Figure 5: Central Dogma. The Central Dogma of biology by Crick holds that genetic information normally flows from DNA to RNA 3.3.1. Genotype-Phenotype Mapping. In EC, mapping from to protein, which involves the mechanisms of gene replication, genotype to phenotype is often an encoding process, espe- transcription and translate. cially in evolutionary algorithms and evolutionary strategies, where the mapping mechanism is used in most cases to directly calculate a fitness function of an individual. How- ever, in nature, the mapping process is much more complex, living systems have evolved for billions of years, regulatory typically from highly conserved genotypic information to core processes in various organisms have remained mostly greatly divergent polymorphism in phenotypes. The funda- unchanged despite species divergence. By comparison of mental process in biological genotype-phenotype mapping related species from the same ancestors, such as humans is gene expression, and the most important mechanism and chimpanzees, at both the molecular and organismal in this process is regulation of gene expression, which levels King and Wilson [153] had already found in 1975 will be discussed next. Since research on transcriptional that genetic structures in these two species are almost regulation has discovered increasing evidence that RNA plays the same while at the organismal level, the anatomy, an important role in gene expression, the transcriptome, physiology, behavior and ecology of these two species are that is, the set of all transcribed RNAs, will be reviewed significantly different. This suggested to them that the then. complex adaptive evolution is produced by a combination and multiple utilization of similar, highly conserved Regulation of Gene Expression. In biology, the core processes genetic components under the control of regulatory (Section 3.2.1) of organisms are responsible for generating systems. anatomy and behavior using genetic and cell materials. These A key step in the regulation of gene expression is core processes include metabolism, gene expression, and transcription. Studies there are concentrated on two primary interaction among molecules and cells [22], which are well components: promotors and transcription factors.Promotors, conserved but still under exploration. Regulation of gene also known as cis-regulatory sequences, are responsible expression is the most important mechanism among the for regulatory transcription. Cis-regulatory sequences are core processes to facilitate organismal novelties in evolution. noncoding DNA sequences which determine when and Kirschner and Gerhart highlight the characteristics of “con- where “their” genes are transcribed by regulating access of servation” and “economy” in regulatory core processes in polymerase to transcription start sites. Transcription factors [22]. are proteins interacting with these cis-regulatory sequences Scientists have been trying to understand the process of by binding to certain sites on DNA sequences. Readers gene expression for decades. In 1956, Crick proposed the interested in more details are referred to Wray et al. [77]. Central Dogma of molecular biology, as shown in Figure 5, Transcription factors act either as activators or as repressors which describes the transmission of genetic information of gene expression. For example, if a transcription factor from DNA to protein. The circular arrow around DNA A binds to a site on a DNA sequence that is responsible symbolizes that a DNA is a template for self-replication. for generating protein B, then this factor A is regarded as The arrow from DNA to RNA indicates that an RNA is a repressor to protein B. In addition, as a protein itself, transcribed on a DNA template, and the arrow from RNA factor A also has its template gene sequence. If another to protein signifies that a protein is translated on an RNA transcription factor C can bind to this site and represses the template. generation of protein A, C acts as a repressor to A but in Subsequent biological research revealed that the process turn as an activator to the expression of protein B. These of gene expression is much more complex than such a activators and repressors can work together as a network linear flow and involves a considerable number of complex of logic control. Promotors usually contain a number of regulation operations. The Central Dogma was challenged binding sites for transcription factors, where each site can by discoveries of proteins playing an important role in only be occupied by one factor at a time. These binding regulation of gene expression and, most recently, the non- sites occupy, however, only a small fraction of sequences coding RNA control of chromosome architecture proposed and are distributed unevenly. Some binding sites of different by Mattick [189]. In this section, we concentrate on gene functions can overlap. Furthermore, binding affinities of expression regulation by proteins and will discuss RNA different materials are important for regulation as well. On effects in next section. the other hand, most transcription factors have numerous Recall the discussion of genome redundancy in target genes and use priorities in binding with any of Section 2.1.1. Coding regions on genetic sequences that them [77]. This sophisticated network endows the regulation can be expressed into proteins only occupy a small portion system with high robustness and plasticity necessary for of the entire genome in eukaryotic cells. This discovery evolution of capabilities of organisms. indicated that a huge number of regulatory elements exist Kauffman [190, 191] holds a long standing opinion in genomes that participate in generating adaptation in that gene regulation networks are dynamical systems and evolution according to changes in environments. Although that many phenotypic traits are encoded in the dynamical 16 Journal of Artificial Evolution and Applications attractors of these systems. Dynamical attractors refer to Duplication Transcription Translation DNA RNA Protein cyclic trajectories of the transformations of states of these networks and their study provides clues to the behavior and properties of gene regulatory networks. Kauffman’s point of view—namely, that the topology of a gene regulatory net- Figure 6: Eukaryotic genetic system. Expression of genes is not with an irreversible linear flow in eukaryotes, but involves frequent work largely decides cell types, cell fates, and functional states feedback and interactions among different molecules including of the cell—has been supported by a number of more recent DNA, RNA, and protein, as the dotted lines shown here. studies [192–195]. Meanwhile, simplifying computational models has been proposed to study dynamical attractors. Aldana et al. [196] model gene activities using random Boolean networks (RBN) with varying topologies. They work on evolvability and dynamics in artificial regulatory report that a network with scale-free output topology and networks is necessary. operating close to the critical regime (neither ordered nor chaotic) possesses the greatest robustness and evolvability compared to networks with other topologies and acting in The Transcriptome. The transcriptome, or collection of tran- different dynamical regimes. Further support comes from scripts, refers to all RNAs produced in a single or a group of [197–201], which again confirms Wagner’s argument that cells, working as an intermediate component of gene expres- high robustness and high evolvability can coexist in natural sion. In high-level eukaryotes such as humans, most regions systems (see Section 2.1.1). of the transcriptome are not translated into protein. What Evolution of cis-regulatory sequences as noncoding necessitates the existence of such a large number of RNAs sequences is considerably different from that of protein- in the transcriptome of high-level eukaryotes? Regulatory coding sequences and is less understood. King and Wilson function is one answer to this question. Although regulation [153] suggest that protein-coding sequences are highly of gene expression starts with the transcription step, these conserved during evolution since they were synthesized. It transcribed but nontranslated sequences or noncoding RNA is mutations on promoters that causes most morphological sequences act as regulators for translation in gene expression variation. Research on the evolution of transcriptional and currently attract increasing interest in biological research regulation has become mainstream in molecular biology [83, 212]. in recent years [77]. In particular, Roderiguez-Trelles et al. An RNA is not just a temporary medium between genes [75] find that significant substitution rate differences exist and proteins as described in the Central Dogma. In high- among different promotors, and even some neighboring level eukaryotes, the information transmission from DNA cis-regulatory promotors involved in the same regulatory to protein is not a one-way process but involves many func- network can have different evolution rates. Moreover, Stone tionalities of the transcriptome. The new perspective of gene and Wray [202] propose that local point mutations on expression proposed by Mattick [189, 212] can be described binding sites can lead to rapid evolution in gene expression, in Figure 6. Compared to a prokaryotic genetic system, an which indicates their potential of accelerating evolution. eukaryotic system has a parallel control mechanism with Wagner [203] points out that other simple changes such multiple outputs and information transfers. Rather than a as gene duplication and deletion of promotors can also simple medium of gene expression, RNA metabolism and result in rapid evolution in gene regulatory networks. interaction have been discovered playing an important role By comparing genomes, Fondon and Garner [204] dis- in gene expression regulation. cover that gene-associated tandem repeat expansions and Mattick [84] proposes that noncoding RNAs participate contractions exist and give rise to rapid morphological extensively in gene expression regulation, being present in evolution. In their experimental research, a tandem repeat about 98% of all transcriptional outputs in eukaryotes. In mutation shows both elevated purity and intensive length research on the human transcriptome, Frith et al. [83]find polymorphism among different dog breeds. Mutations on that noncoding RNAs play an important role in generating noncoding sequences can modify regulation of the target phenotypic variation. Noncoding RNAs can be classified into genes, the length of coding loci to transcribe, and the two categories: introns and other noncoding RNAs. occurrence conditions. Furthermore, they also result in Regulation of the transcriptome shows contributions morphological variation and accelerated phenotypic evolu- to evolvability and rapid evolution. Introns, an important tion. category of noncoding RNAs, are found more susceptible Since the mechanisms of regulation of gene expression to mutations than their neighboring protein-coding exons. can well explain many phenomena in evolvability and rapid Rather than having no function, as thought previously, it evolution in living systems, research on artificial regulatory was found that introns do have influence on regulation networks has now started in computer science. Several (see, e.g., [213]). The fewer constraints imposed on introns models of artificial evolution regulatory networks have been by selection offer flexibility to generate new functions and proposed such as Banzhaf et al. [205–208], Chavoya and rapid protein sequence evolution during the process of Duthen [209], Mattiussi and Floreano [210], and Nehaniv regulation, especially in connection with alternative splicing. [211]. These artificial models intend to generate regulatory The evolution of RNA communication networks may also behavior akin to that of natural systems. However, these accelerate the evolution of gene expression, as observed research efforts are still in their early stages, and more by Mattick [84]. These RNA communication networks, Journal of Artificial Evolution and Applications 17 Euchromatin Heterochromatin The main mechanisms of epigenetic control are DNA methylation and histone modification [215]. Modifications Me Ac Ac Me P Me P Me Me Me to chromatin, either on the DNA sequence itself (DNA Ac methylation) or on its surrounding proteins (histone modi- fication), affect gene expression and can be inherited from cell generation to cell generation during cell division. DNA methylation is a chemical addition to DNA sequences. Genes (a) (b) with methyl marks are repressed in expression, despite their unchanged DNA content [219]. In histone modification, the Figure 7: Euchromatin and Heterochromatin. Histone tails have tails of histone proteins are modified by different molecular three types of modification including acetylation (Ac), phosphory- attachments, for examole, acetyl, phosphoryl, and methyl lation (p), and methylation (Me). Euchromatin (a) is the loosely groups (see Figure 7). If acetyl groups are attached to the packed state that most histone tails are attached by acetyl groups. histone tails of a chromatin, it will be loosely packed, a Heterochromatin (b) is the tightly packed state that most histone state called euchromatin. In euchromatin, DNA is readable tails are attached by methyl groups. (Adapted from Jenuwein and and can be transcribed into RNA and later translated Allis [218].) into proteins. In contrast, if methyl groups are attached to histone tails, chromatin is tightly compressed, a state called heterochromatin. In the heterochromatin state, genes are inaccessible to the transcriptional machinery such as which describe interaction among different layers of RNA RNA polymerase or to transcription factors, and genes are signaling, provide a sophisticated regulatory architecture, prevented from being transcribed [220]. Other mechanisms enabling DNA-DNA, DNA-RNA, or RNA-RNA communi- recognized to be responsible for epigenetic regulation of cation, DNA methylation, chromatin generation, and RNA gene expression include chromatin remodeling, histone translation. variant composition, and noncoding RNA regulation. A Compared to natural systems, the genotype-phenotype discussion of these mechanisms can be found by Allis et al. mapping in EC is rather primitive still and a transcrip- [188]. tome is mostly missing in algorithms. The complex RNA The key feature of epigenetic mechanisms is their ability parallel information transfer framework inspires various to coordinate internal and environmental signals which applications. Based on what computational models have can collaborate to modify protein production [215]. The already achieved with artificial regulatory networks, more underlying interactions involve various molecules, such as mechanisms should be implemented, especially the newly DNA, RNA, and proteins, but the extensive feedback between discovered powerful mechanisms of transcriptome regula- these molecules is still beyond our current understanding. tion (see a step in this direction here [214]). We believe that epigenetics opens up a new field in evolvability studies for both biology and EC. Sophisticated 3.3.2. Epigenetic Mechanism. Epigenetics has become a new epigenetic feedback networks suggest a new structure for EC compared to the linear flow of computation usually research direction in evolutionary biology [21]. Literally, “epi”-genetic control lies in the regulation of gene expression employed in the literature. For instance, in dynamic opti- without changing the DNA sequence itself; so it is “beyond mization problems, not all genes responsible for different subfunctions need to be expressed all the time. We anticipate the conventional genetic” control. Epigenetic regulation arises during the processes of organism development and cell that a “controller switch” can be integrated into the genotype proliferation, triggered by intrinsic signals or environmental allowing short-term changes, where fragments of the genome stimulations [215]. Epigenetic changes are heritable in the can be turned on and off in response to external feedback. short term from cell generation to cell generation, and Such a mechanism for repression of expression has barely been used in computation. Similar multilayer adaptive these stable alterations do not involve mutations on DNA sequences. Epigenetic regulation of DNA expression lies at encoding schemes have been proposed, for example, the the heart of many complex and long-term human diseases messy Genetic Algorithm (mGA) [164] that combines short building blocks to form variable-length chromosomes to [216]. Previous research in genetics mostly focused on the increasingly cover all features of a problem, or diploid Genetic sequential information carried by DNA. However, DNA Algorithm,for example, [221] using a two-chromosome sequences are coiled up in cells in intimate complexes with representation to adapt phenotypic variation in dynamic the help of so-called histone proteins. A DNA sequence environments. However, existing work has not embedded the wrapped with histones comprises a nucleosome. Chromatin organizational epigenetic control in algorithms that would is the complex of nucleosomes in the nucleus of cells which allow significant flexibility in changing environments. We participates in the control process of gene expression. The anticipate that epigenetic mechanisms will play a crucial role in increasing the evolvability of EC algorithms. chromatin composition varies according to cell type and response to internal and external signals. The different composition of chromatin may affect expression and thus 3.3.3. Development. Evolutionary developmental biology change the produced proteins even in the absences of DNA with the subject of the relation between evolution and devel- sequence modification [217]. opment, nick-named evo-devo, has arisen as a productive 18 Journal of Artificial Evolution and Applications research direction which tries to unify concepts that have secondly at benefiting a population’s ability to diversify and been separated for a long number of years. The develop- persist. mental viewpoint provides crucial clues to many puzzles and The importance of the developmental point of view controversies that have arisen in genetics and evolutionary seems to be partially realized by the EC community. A biology in the past [222]. Vice versa, evolution is key to new area named generative and developmental systems has understanding the developmental mechanisms that have emerged and attracted studies. Artificial or computational shaped multicellular life [223]. embryogeny was first introduced to simulate the development process in silico (see, e.g., [232–234]). More recently, inspired Definition 8 (development). Development is the process by the complex mapping from genotype to phenotype, by which a multicellular organism unfolds its phenotype, computer scientists have started to allow more freedom and starting from a fertilized single-cell stadium (the zygote), to scalability when representing individuals, a topic known a mature multicellular stadium through a defined sequence as indirect encoding. With an indirect encoding scheme, a of stages that are under the control of its genome and heavily genotype does not map directly to units of structure in its influenced by its environment. phenotype, but a growth or developmental process is allowed in this mapping [235]. Various encoding methods have West-Eberhard [224] discusses the relation of develop- been proposed using, for example, hierarchical grammars ment and evolution and suggests that it is important to [236, 237], or simulating cell chemical processes [238, 239]. reexamine major themes of evolutionary biology in the Indirect encoding schemes have shown advantages over light of development. Molecular biology has extensively traditional one-to-one direct encodings [240, 241]. Indirect investigated evolution on the genotypic level, studying the encoding is a first step to simulate biological development mechanisms of gene expression and protein formation, in computational systems by allowing more freedom and the effect of mutations on genes, and other questions. It complexity in the genotype-phenotype mapping, but it is is, however, development which produces the multicellular by no means the full story of development. As evolutionary phenotypes and their variation that ultimately is screened by developmental biology continues to produce new insights, selection. So in order to examine the effectofamutationon it will be imperative for the EC community to increase its the evolution of multicellular organisms, one has to look at efforts to design new algorithms that are inspired by evo- the effects of this mutation in development. Development devo. emphasizes the time-dimension of an organism and the continuity of phenotypic changes in its interaction with the environment. 4. Selection A major focus of the field of evo-devo is to study the Although Darwin’s theory of evolution being directed pri- role of phenotypic plasticity,or developmental plasticity in marily by natural selection has been the subject of much evolution [225–228]. Phenotypic plasticity is the phenotypic argument, selection is an extremely important operation responsiveness of an organism to environmental input, and it to stabilize the functional traits already generated by some is the most universal property of the phenotype of organisms. exploratory operations [128]. Selection mechanisms are Organisms can alter their form, status, behavior, movement, divided into two types by their effects on different stages or other features in response to environmental stimuli. These of evolution. First, positive selection enhances the fixation changes mostly will not involve any modifications of their of advantageous alleles thus improving the diversity in early genome. This flexibility is a result of the development pro- stages of evolution [139, 143]. Second, negative selection, cess, with a complex mapping from genotype to phenotype. also known as stabilizing selection or purifying selection, The effect of phenotypic plasticity on the rate of evo- occurs at later stages of evolution when genetic diversity lution is a subject of debate [229]. It either can accelerate decreases when such selection eliminates deleterious alleles evolution since new and adaptive alternative phenotypes and only stabilizes specific traits [141]. The balance between are generated to match the current environment or can selection and diversity of an evolutionary population has also be considered to delay the rate of genetic changes been a critical problem, and the dynamic pressure and some since this flexibility is able to provide adaptation to an consequences of selection are still under active investigation. organism without the need to modify its genotype. The In general, selection pressure is produced by two factors, the role of phenotypic plasticity in evolution depends on which environment and mating competition, both of which will be level of evolution is studied, and under what conditions discussed next. [227]. It is, however, clear that both major properties of the phe- notype, its plasticity and its modularity (see Section 3.2.2), 4.1. Environmental Selection. Environmental selection orig- are the result of the hierarchical organization of the inates in the external surroundings and enforces the adap- development process producing the phenotype from the tivity of organisms to survive. Since Darwin environmental genotype. These characteristics both contribute a great deal selection has received extensive attention in evolutionary to the evolvability of living systems [230]. Kirschner and biology. Natural selection is an extremely important driving Gerhart [47, 231] hold the opinion that plasticity and mod- force for adaptive evolution in natural populations [242]. ularity contribute to evolvability due to their advantages, The first response of a living organisms to a changing first at providing adaptation at the individual level, and environment is somatic adaptation. A simple example of Journal of Artificial Evolution and Applications 19 somatic adaptation is human temperature compensation pressure is increasingly strong when the environment [22]. When the external temperature increases above normal, becomes uncertain. Dramatic environmental changes lead to humans will sweat to adapt to this new environment. selection for better evolvability. They consider evolvability Shivering will occur if temperature falls below a normal as a selectable trait, and facilitating environmental changes value. Somatic adaptation happens directly as an organismal can be a method to accelerate evolution. A recent simulation reaction to a changing environment and is not fixed in by Kashtan et al. [74] in a biologically realistic setting also morphological structures as an evolutionary change unless suggests that varying environments may accelerate natural some deeper adaptations caused by somatic changes can evolution. In their work, different scenarios of temporarily increase the survivability of an organism. Organisms have changing optima were used. Kashtan and Alon [246]report plenty of latent traits within their somatic adaptability; that a goal that varies in a modular way can speed up so they have a fairly high tolerance to changes in their evolution. Other work [247] takes into account the effect of environment [22]. Somatic adaptation mechanisms can only the rate of environmental change. By observing the dynamics adjust existing functions to external changes. However, if of adaptive walks under scenarios of varying speeds, they evolution acts for a long time under environmental selection, find that environments with varying rates of change have changes may be stabilized by mutations in the germ-line after noticeably different effects on the fixation of beneficial somatic adaptation has tested them through promotion of mutations, the substitution time required, and the final survivability of the organism. phenotypic variation. Selection can act at different levels depending on its In EC, selection strategies are considered affecting search targets [47]. These might be individual selection, individual- capability significantly during an evolutionary process. and-clade selection, or clade selection. At the individual Different selection strategies have been proposed and the level, the selection process has the fewest constraints since dynamics of selection pressure has been studied extensively it directly affects phenotypic function fitness, and fewer [248, 249]. Since the effects of environmental selection on mutational changes are required for a new adaptive trait. An the evolution of evolvability have been recognized, further individual can also interact with others in a clade, such as research on the dynamics of selection is required. Moreover, through recombination, and survive under selective pressure somatic adaptation might be considered when applying as a member in this clade. At the highest level, selection selection. Group-based selection methods should also be can happen on the level of an entire clade given large studied for varying selection pressure, so that a balance environmental impact, and the entire clade can, as a whole, between the development of a minority and of the entire escape from extinction. Some small groups of the lineage population can be dynamically achieved. might go extinct, but the entire line will be able to survive even if it might have to go through population bottlenecks. This idea has drawn more attentions in some subsequent 4.2. Sexual Selection. Sexual selection was proposed by works [243, 244]. Darwin as the pressure away from the possibility of mating Interactions between different species may also cause failure. Two forms of sexual selection pressure are met by environmental changes. Phillips and Shine [245]reportan mature high-level animals: the battle between male individ- interesting phenomenon on species invasion. Toxic cane uals who fight, and the competition through mating choice toads induced morphological changes among a species of made by females. Fisher [105]proposeda runaway process, snakes in Australia. Generally, native natural ecosystems can where a male trait and female preference for it can both be devastated by the invasion of new species. At upon the evolve dramatically over time until finally checked by severe arrival of an invasive species, the number of native organisms counter-selection. In modern biology, scientists pay much may decrease. However, as these native organisms adapt attention to these sex-based competitions that can generate towards the invaders, the impact of the invasion declines and evolve several kinds of traits in high-level organisms. and a new balance is achieved. Morphological changes For instance, Kirkpatrick and Ravigne [250] find that some are fixed subsequently. Complex natural ecosystems possess secondary sexual characteristics among individuals of the communities with highly frequent and dense interactions same sex can trigger rapid speciation. between species as well as between species-specific functional Sexual selection happens at the interspecies level and traits within a species. affects reproductive fitness of individuals. Reproductive Environmental selection is now widely accepted as con- fitness is the probability of successfully generating offspring. tributing significantly to natural evolution, and has entered Sexual selection has two main forms: intrasexual selection the mainstream of studies in evolvability. As a potential to and intersexual selection. Intrasexual selection is known generate adaptation, evolvability is difficult to observe and as the combat between competitive male individuals, and to select for. However, there is increasing research arguing usually occurs in the form of a fight. Intersexual selection that evolvability is selectable and environmental selection is based on the choice made by the opposite sex. Male sec- can improve the evolution of evolvability. In the real world, ondary sexual characteristics and female mating preferences the environment is changing constantly and fixes beneficial can affect each other and evolve cooperatively [251]. This mutations, and there is a growing acceptance that a changing joint selection pressure, combined with natural selection, is environment is a key ingredient to studying evolvability. a powerful force for rapid evolution. Selection pressure is a critical operator to control an evolu- Recent research in biology has connected sexual selection tionary process. Earl and Deem [55] suggest that selection to the acceleration of evolution. Colegrave [252] finds that 20 Journal of Artificial Evolution and Applications the rate of adaptation can be increased by sex mechanisms among evolutionary components, internal or external to because sexual selection allows a rapid adaptive response these organisms during a long, continuing evolutionary under changing conditions by fixing beneficial mutations. process [170]. In reality, the fitness of individuals in a Swanson and Vacquier [253] observe that rapid evolution system can vary a great deal. Moreover, a large-scale quality emerges in reproductive proteins. This rapid evolution is differentiation exists in almost every natural evolutionary forced by three main selective factors: sperm competition, system, and these vastly diverse evolution systems exhibit sexual selection, and sexual conflict. Sperm competition is substantial evolvability. Since selection and evaluation act quite fierce in that each sperm will compete with billions of directly on observable phenotypic functions but evolv- others to fuse with the only egg, and this competition exists ability only provides the potential for better functions, in multiple steps for the sperm. Sexual selection happens selection and evaluation for evolvability are not observable when different eggs have varying affinities for a special allele directly. of a sperm-surface protein, and only the egg with the highest Since EC has been widely applied in many areas of affinity is most likely to bind to this sperm. Sexual conflict industry and academia, fitness evaluation arises as a difficult means that only one egg can be fused with the sperm to avoid problem because it is usually very CPU-intensive. In the polyspermy such that only one embryo is fertilized. These current literature, two main methods of fitness evaluation types of mechanisms add considerable selection pressure to are employed, absolute fitness and relative fitness. Absolute reproductive proteins and thus trigger rapid evolution in fitness of each individual usually refers to its value of a certain regions of these proteins. specified fitness function. Relative fitness compares different The concept of mating choice was already applied in individuals and gives a rank to each individual to produce EC decades ago by Miller [254, 255]. Some coevolutionary a record of winners. This latter method is good at sup- algorithms have been proposed to simulate mechanisms pressing exceptionally good individuals, thus, helping an from sexual selection by constructing subgroups which can evolutionary system to escape from premature convergence. affect each other cooperatively to evolve in parallel. As more In fact, evaluating the fitness of each individual is usually and more knowledge has been accumulated by biologists difficult for many optimization problems in the real world on the complex process of sexual selection, especially on because explicit fitness can be hard to define and expensive the advantages that sex mechanisms contribute to the to calculate. As a result, fitness approximation has been acceleration of speciation and evolution, this knowledge proposed with differing levels of approximation, including should be better incorporated in EC. “problem approximation”, “functional approximation”, and “evolutionary approximation”. Jin [260] has surveyed these approaches. They are sensitive to training data and to varying 4.3. Fitness Evaluation. Fitness evaluation measures behavior constraints of different models; so a common framework or function of individuals or species. In nature, fitness of would be required. Moreover, Reisinger and Miikkulainen an individual or species is implicit and subject to natural [56] propose an evolvable representation and an evaluation selection, whereas in EC, fitness is mostly based on numerical strategy to exert indirect selection pressure on evolvability. values of an individual as solution to a given problem, and In their work, a systematically changing fitness function this fitness is explicit. is adopted according to a special evolvable representation that can reflect efficiently how genetic changes restructure Definition 9. Fitness is the measure to quantify an evolution- phenotypic variation. Thus, evolvability can be evaluated ary individual/component with regard to its ability to survive through the way such a systematic structure can expand in and reproduce in a certain environment. phenotypes. These approaches might provide a good starting In nature, adaptable species survive by passing different point to simulate the implicit adaptive fitness evaluation challenges, and less fit species may become extinct during from nature, a method that has good prospects for detecting evolution. Adaptability lies not only in the currently existing evolvability in EC. adaptivity to the environment but also in the capability to generate more adapted offspring. In essence, fitness of 5. Conclusion natural organisms is implicit and is subject to natural selection. Empirically, biologists use mathematical methods Since Darwin proposed his theory of natural evolution based to quantify fitness. Individual fitness usually refers to the on heritable variation and natural selection, an enormous viability of an individual, that is, its probability to survive research effort has been dedicated to revealing the intricacies [256]. Moreover, individuals having more offspring can be of the processes involved. In modern biology, a host of details considered as fitter ones since their genetic information is about mechanisms of evolution and factors that can affect more likely to be preserved. Other than at the individual evolution have been revealed. Besides understanding the level, in classic population genetics literature [257], the geno- history of evolution, biologists are currently paying attention type fitness quantifies the frequency changes of a genotype to the capability of organisms to evolve and to the evolution in a population during transformation from one generation of such capability in an open-ended natural evolutionary to the next. Various measures have been proposed in the process. Varying evolution rates among different species or biological literature (see [258, 259] for detailed reviews). different regions of genetic material in an organism attract The above implicit fitness in natural organisms empha- researchers’ interest under the aspect of the acceleration sizes evolvability under intricate pressures from interactions of evolution. Meanwhile, in artificial evolutionary systems, Journal of Artificial Evolution and Applications 21 one is also working on improving the power of systems by [12] “Annual “HUMIES” Awards for Human Competitive Results,”ProducedbyGenetic andEvolutionaryComputa- studying more intelligent and adaptive mechanisms. tion held by GECCO: Genetic and Evolutionary Compu- Evolvability, as the capability to generate adaptation by tation Conference, ACM, New York, NY, USA, since 2004, producing fitter offspring via evolutionary operations, has http://www.humancompetitive.org. received considerable interest in recent research in both [13] C. Darwin, On the Origin of Species by Means of Natural biology and EC. Substantial work has been published on this Selection, John Murray, London, UK, 1859. topic in both areas, and we have tried to cover many of the [14] S. J. Gould, The Structure of Evolutionary Theory,Harvard factors that contribute to evolvability in this review. After University Press, Cambridge, Mass, USA, 2002. some phenomena of rapid evolution were found in nature, [15] E. Mayr and W. B. Provine, Eds., The Evolutionary Synthesis, acceleration of evolution also became an hot research topic, Harvard University Press, Cambridge, Mass, USA, 2nd not the least because an increase in the speed of artificial edition, 1998. evolution would greatly benefit applications. [16] W. Banzhaf, G. Beslon, S. Christensen, et al., “Guidelines: More is to come. As a result of the unrelenting progress in from artificial evolution to computational evolution: a biology, a lively discussion has now ensued as to what a gene research agenda,” Nature Reviews Genetics,vol. 7, no.9,pp. really is [261–264]. There were times when the notion of a 729–735, 2006. gene was simple. But with the advent of alternative splicing, [17] J. Timmis, M. Amos, W. Banzhaf, and A. Tyrrell, “Going back intron activity during regulation, iRNA, and other wonderful to our roots: second generation biocomputing,” International intricacies [265], the simple life of geneticists seems to be Journal of Unconventional Computing, vol. 2, no. 4, pp. 349– over. In this review, our perspective was much more limited 378, 2006. by discussing only two very narrow aspects of evolution, its [18] S. Carroll, J. Grenier, and S. Weatherbee, From DNA to speed and the issue of evolvability. We hope that the ideas Diversity, Blackwell, Oxford, UK, 2005. discussed here can inspire new methods and applications of [19] G. Cochran and H. Harpending, The 10,000 Year Explosion, EC. Basic Books, New York, NY, USA, 2009. [20] G. Forgacs and S. A. Newman, Biological Physics of the Developing Embryo, Cambridge University, Cambridge, UK, Acknowledgments [21] E. Jablonka and M. J. Lamb, Evolution in Four Dimensions: Funding by NSERC under the Discovery Grant Program Genetic, Epigenetic, Behavioral, and Symbolic Variation in the RGPIN 283304-07 is gratefully acknowledged. The authors History of Life, MIT Press, Cambridge, Mass, USA, 2005. thank Y. P. Chen and T. Weise for valuable discussions. [22] M. W. Kirschner and J. C. Gerhart, The Plausibility of Life, Yale University Press, New Haven, Conn, USA, 2005. References [23] M. Novak, Evolutionary Dynamics, Harvard University Press, Cambridge, Mass, USA, 2006. [1] W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, [24] S. R. Palumbi, The Evolution Explosion, Norton, New York, Genetic Programming—An Introduction: On the Automatic NY, USA, 2002. Evolution of Computer Programs and Its Applications,Morgan [25] A. Wagner, Robustness and Evolvability in Living Systems, Kaufmann, San Francisco, CA, USA, 1998. Princeton University Press, Princeton, NJ, USA, 2005. [2] K. A. De Jong, Evolutionary Computation: A Unified [26] M. A. Bedau, J. S. McCaskill, N. H. Packard, et al., “Open Approach, MIT Press, Cambridge, Mass, USA, 2006. problems in artificial life,” Artificial Life, vol. 6, no. 4, pp. 363– [3] A.Eiben andJ.Smith, Introduction to Evolutionary Comput- 376, 2000. ing, Springer, Berlin, Germany, 2003. [27] T. S. Ray, “An approach to the synthesis of life,” in Artificial [4] D.B.Fogel, Evolutionary Computation: Toward a New Life II, pp. 371–408, Addison-Wesley, Reading, Mass, USA, Philosophy of Machine Intelligence, IEEE Press, Orlando, Fla, USA, 1995. [28] C. Adami and C. T. Brown, “Evolutionary learning in the 2D [5] L.J.Fogel,A.J.Owens,and M. J. Walsh, Artificial Intelligence artificial life system “avida”,” in Artificial Life IV, pp. 377–381, through Simulated Evolution, John Wiley & Sons, New York, MIT press, Cambridge, Mass, USA, 1994. NY, USA, 1966. [29] M. A. Bedau, E. Snyder, and C. T. Brown, “A comparison of [6] D.E.Goldberg, Genetic Algorithms in Search, Optimization evolutionary activity in artificial evolving systems and in the and Machine Learning, Addison-Wesley, Reading, Mass, USA, biosphere,” in Proceedings of the 4th European Conference on Artificial Life, pp. 125–134, Springer, 1997. [7] J. H. Holland, Adaptation in Natural and Artificial Systems, [30] M. A. Bedau and C. T. Brown, “Visualizing evolutionary University of Michigan Press, Ann Arbor, Mich, USA, 1975. activity of genotypes,” Artificial Life, vol. 5, no. 1, pp. 17–35, [8] J.R.Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press, Cambridge, Mass, USA, 1992. [31] J. E. Barrick,D.S.Yu, S. H. Yoon,etal., “Genomeevolution and adaptation in a long-term experiment with Escherichia [9] M. Mitchell, An Introduction to Genetic Algorithms,MIT coli,” Nature, vol. 461, no. 7268, pp. 1243–1247, 2009. Press, Cambridge, Mass, USA, 1998. [32] P. Alberch, “From genes to phenotype: dynamical systems [10] I. Rechenberg, Evolution Strategy, Frommann-Holzboog, and evolvability,” Genetica, vol. 84, no. 1, pp. 5–11, 1991. Stuttgart, Germany, 1973. [33] J. F. Y. Brookfield, “Evolution: the evolvability enigma,” [11] H.-P. Schwefel, Evolution and Optimum Seeking: The Sixth Generation, John Wiley & Sons, New York, NY, USA, 1995. Current Biology, vol. 11, no. 3, pp. R106–R108, 2001. 22 Journal of Artificial Evolution and Applications [34] A. J. R. Carter, J. Hermisson, and T. F. Hansen, “The role of [56] J. Reisinger and R. Miikkulainen, “Selecting for evolvable epistatic gene interactions in the response to selection and representations,” in Proceedings of the Genetic and Evolution- the evolution of evolvability,” Theoretical Population Biology, ary Computation Conference (GECCO ’06), vol. 2, pp. 1297– vol. 68, no. 3, pp. 179–196, 2005. 1304, ACM, 2006. [35] J. L. Hendrikse, T. E. Parsons, and B. Hall-grimsson, “Evolv- [57] J. Reisinger, K. O. Stanley, and R. Miikkulainen, “Towards ability as the proper focus of evolutionary developmental an empirical measure of evolvability,” in Proceedings of the biology,” Evolution and Development, vol. 9, no. 4, pp. 393– Genetic and Evolutionary Computation Conference Workshop 401, 2007. Program (GECCO ’05), pp. 257–264, ACM, 2005. [36] D. Houle, “Comparing evolvability and variability of quanti- [58] N. Eldredge and S. J. Gould, “Punctuated equilibria: an tative traits,” Genetics, vol. 130, no. 1, pp. 195–204, 1992. alternative to phyletic gradualism,” in Models in Paleobiology, pp. 82–115, Freeman Cooper, 1972. [37] K. F. Liem, “Key evolutionary innovations, differential diver- sity, and symecomorphosis,” in Evolutionary Innovations,pp. [59] W. H. Li, C. I. Wu, and C. C. Luo, “A new method 147–170, University of Chicago Press, Chicago, Ill, USA, for estimating synonymous and nonsynonymous rates of nucleotide substitution considering the relative likelihood [38] P. D. Sniegowski and H. A. Murphy, “Evolvability,” Current of nucleotide and codon changes,” Molecular Biology and Biology, vol. 16, no. 19, pp. R831–R834, 2006. Evolution, vol. 2, no. 2, pp. 150–174, 1985. [39] G. Bell, “Evolutionary genetics: the evolution of evolution,” [60] T. Miyata and T. Yasunaga, “Molecular evolution of mRNA: Heredity, vol. 94, no. 1, pp. 1–2, 2005. a method for estimating evolutionary rates of synonymous and amino acid substitutions from homologous nucleotide [40] R. Dawkins, “The evolution of evolvability,” in Artificial Life: The Quest for a New Creation, pp. 201–220, Addison-Wesley, sequences and its application,” Journal of Molecular Evolution, vol. 16, no. 1, pp. 23–36, 1980. Reading, Mass, USA, 1989. [41] A. G. Jones, S. J. Arnold, and R. Burger, “The mutation [61] M. Nei and T. Gojobori, “Simple methods for estimating matrix and the evolution of evolvability,” Evolution, vol. 61, the numbers of synonymous and nonsynonymous nucleotide substitutions,” Molecular Biology and Evolution, vol. 3, no. 5, no. 4, pp. 727–745, 2007. pp. 418–426, 1986. [42] L. Partridge and N. H. Barton, “Evolving evolvability,” Nature, vol. 407, no. 6803, pp. 457–458, 2000. [62] Z. Yang and J. R. Bielawski, “Statistical methods for detecting [43] M. Pigliucci, “Perspectives: is evolvability evolvable?” Nature molecular adaptation,” Trends in Ecology and Evolution, vol. 15, no. 12, pp. 496–503, 2000. Reviews Genetics, vol. 9, pp. 75–82, 2008. [63] R. P. Worden, “A speed limit for evolution,” Journal of [44] M. Radman, I. Matic, and F. Taddei, “Evolution of evolvabil- ity,” Annals of the New York Academy of Sciences, vol. 870, pp. Theoretical Biology, vol. 176, no. 1, pp. 137–152, 1995. 146–155, 1999. [64] M. A. Bedau and N. H. Packard, “Measurement of evolution- [45] G. P. Wagner and L. Altenberg, “Complex adaptations and ary activity, teleology, and life,” in Artificial Life II, pp. 431– the evolution of evolvability,” Evolution,vol. 50, no.3,pp. 461, Addison-Wesley, Reading, Mass, USA, 1992. 967–976, 1996. [65] M. A. Bedau, “Quantifying the extent and intensity of adap- [46] L. Altenberg, “The evolution of evolvability in genetic tive evolution,” in Proceedings of the Genetic and Evolution programming,” in Advances in Genetic Programming, pp. 47– Computation Conference Workshop Program (GECCO ’99), 74, MIT Press, Cambridge, Mass, USA, 1994. pp. 34–37, Morgan Kaufmann, San Francisco, CA, USA, [47] M. Kirschner and J. Gerhart, “Evolvability,” Proceedings of the 1999. National Academy of Sciences (USA), vol. 95, no. 15, pp. 8420– [66] M. J. Raven and M. A. Bedau, “General framework for 8427, 1998. evolutionary activity,” in Proceedings of the 7th European [48] P. Marrow, “Evolvability: evolution, computation, and biol- Conference on Artificial Life, pp. 676–685, Springer, 2003. ogy,” in Proceedings of the Genetic and Evolutionary Compu- [67] G. G. Simpson, Tempo and Mode in Evolution, Columbia tation Conference Workshop on Evolvability (GECCO ’99),pp. University Press, New York, NY, USA, 1944. 30–33, Morgan Kaufmann, San Francisco, CA, USA, 1999. [68] Z. D. Blount, C. Z. Borland, and R. E. Lenski, “Historical [49] M. Conrad, “The geometry of evolution,” BioSystems, vol. 24, contingency and the evolution of a key innovation in an no. 1, pp. 61–81, 1990. experimental population of Escherichia coli,” Proceedings of [50] A. Wagner, “Robustness, evolvability, and neutrality,” FEBS the National Academy of Sciences (USA), vol. 105, no. 23, pp. Letters, vol. 579, no. 8, pp. 1772–1778, 2005. 7899–7906, 2008. [51] M. A. Bedau and N. H. Packard, “Evolution of evolvability [69] L. P. Oliveira, H. J. Jensen, M. Nicodemi, and P. Sibani, via adaptation of mutation rates,” BioSystems,vol. 69, no.2- “Record dynamics and the observed temperature plateau in 3, pp. 143–162, 2003. the magnetic creep-rate of type-II superconductors,” Physical [52] J. Masel, “Evolutionary capacitance may be favored by Review B, vol. 71, Article ID 104526, 2005. natural selection,” Genetics, vol. 170, no. 3, pp. 1359–1371, [70] P. Sibani, M. R. Schmidt, and P. Alstrøm, “Fitness opti- 2005. mization and decay of extinction rate through biological [53] C. L. Nehaniv, “Measuring evolvability as the rate of evolution,” Physical Review Letters, vol. 75, no. 10, pp. 2055– 2058, 1995. complexity increase,” in Proceedings of the 7th Artificial Life Workshop, pp. 55–57, MIT Press, 2000. [71] H. A. Simon, “Near decomposability and the speed of [54] A. Wagner, “Robustness and evolvability: a paradox evolution,” Industrial and Corporate Change,vol. 11, no.3, pp. 587–599, 2002. resolved,” Proceedings of the Royal Society B, vol. 275, no. 1630, pp. 91–100, 2008. [72] X. Gu, Z. Zhang, and W. Huang, “Rapid evolution of expres- [55] D. J. Earl and M. W. Deem, “Evolvability is a selectable trait,” sion and regulatory divergences after yeast gene duplication,” Proceedings of the National Academy of Sciences of USA, vol. Proceedings of the National Academy of Sciences (USA), vol. 101, no. 32, pp. 11531–11536, 2004. 102, no. 3, pp. 707–712, 2005. Journal of Artificial Evolution and Applications 23 [73] J. Hawks, E. T. Wang, G. M. Cochran, H. C. Harpending, [91] R. Wilds, S. A. Kauffman, and L. Glass, “Evolution of complex and R. K. Moyzis, “Recent acceleration of human adaptive dynamics,” Chaos, vol. 18, no. 3, Article ID 033109, 2008. evolution,” Proceedings of the National Academy of Sciences [92] J. A. Draghi, T. L. Parsons, G. P. Wagner, and J. B. Plotkin, (USA), vol. 104, no. 52, pp. 20753–20758, 2007. “Mutational robustness can facilitate adaptation,” Nature, vol. 463, no. 7279, pp. 353–355, 2010. [74] N. Kashtan, E. Noor, and U. Alon, “Varying environments can speed up evolution,” Proceedings of the National Academy [93] F. Rothlauf and D. E. Goldberg, “Redundant representations of Sciences (USA), vol. 104, no. 34, pp. 13711–13716, 2007. in evolutionary computation,” Evolutionary Computation, [75] F. Rodriguez-Trelles, R. Tarrio, and F. J. Ayala, “Evolution vol. 11, no. 4, pp. 381–415, 2003. of cis-regulatory regions versus codifying regions,” Interna- [94] W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, “The evolu- tional Journal of Developmental Biology,vol. 47, no.7-8,pp. tion of size and shape,” in Advances in Genetic Programming, 665–673, 2003. vol. 3, pp. 163–190, MIT Press, Cambridge, Mass, USA, 1999. [76] S. Asthana, W. S. Noble, G. Kryukov, C. E. Grant, S. Sunyaev, [95] S. Luke, “Code growth is not caused by introns,” in and J. A. Stamatoyannopoulos, “Widely distributed noncod- Proceedings of the Genetic and Evolutionary Computation ing purifying selection in the human genome,” Proceedings of Conference (GECCO ’00), pp. 228–235, Morgan Kaufmann, the National Academy of Sciences (USA), vol. 104, no. 30, pp. San Francisco, CA, USA, 2000. 12410–12415, 2007. [96] P. Nordin, F. Francone, and W. Banzhaf, “Explicitly defined [77] G. A. Wray, M. W. Hahn, E. Abouheif, et al., “The evolution introns and destructive crossover in genetic programming,” of transcriptional regulation in eukaryotes,” Molecular Biol- in Advances in Genetic Programming, vol. 2, pp. 111–134, MIT ogy and Evolution, vol. 20, no. 9, pp. 1377–1419, 2003. Press, Cambridge, Mass, USA, 1996. [97] T. Yu and J. Miller, “Neutrality and the evolvability of boolean [78] J. D. Wren,E.Forgacs,J.W.FondonIII,etal., “Repeat polymorphisms within gene regions: phenotypic and evolu- function landscape,” in Proceedings of the 4th European tionary implications,” American Journal of Human Genetics, Conference on Genetic Programming (EuroGP ’01), vol. 2038 vol. 67, no. 2, pp. 345–356, 2000. of Lecture Notes in Computer Science, pp. 204–217, Springer, [79] J. L. Freeman, G. H. Perry, L. Feuk, et al., “Copy number vari- ation: new insights in genome diversity,” Genome Research, [98] M. Beckerman, Molecular and Cellular Signaling, Springer, Berlin, Germany, 2005. vol. 16, no. 8, pp. 949–961, 2006. [80] A. J. Iafrate, L. Feuk, M. N. Rivera, et al., “Detection of large- [99] F. Jacob, “Evolution and tinkering,” Science, vol. 196, no. 4295, pp. 1161–1166, 1977. scale variation in the human genome,” Nature Genetics, vol. 36, no. 9, pp. 949–951, 2004. [100] J. Cohen, “Bioinformatics—an introduction for computer scientists,” ACM Computing Surveys, vol. 36, no. 2, pp. 122– [81] J. Sebat, B. Lakshmi, J. Troge, et al., “Large-scale copy number 158, 2004. polymorphisminthe humangenome,” Science, vol. 305, no. 5683, pp. 525–528, 2004. [101] J. D. Watson, N. H. Hopkins, J. W. Roberts, J. A. Steitz, and A. M. Weiner, Molecular Biology of the Gene, Benjamin- [82] E. Tuzun, A. J. Sharp, J. A. Bailey, et al., “Fine-scale structural Cummings, 5th edition, 2004. variation of the human genome,” Nature Genetics, vol. 37, no. 7, pp. 727–732, 2005. [102] B. Alberts, D. Bray, J. Lewis, M. Raff,K.Roberts,and J. D. Watson, Molecular Biology of the Cell, Garland, New York, NY, [83] M. C. Frith, M. Pheasant, and J. S. Mattick, “The amazing USA, 4th edition, 2002. complexity of the human transcriptome,” European Journal of Human Genetics, vol. 13, no. 8, pp. 894–897, 2005. [103] M. Z. Ludwig, N. H. Patel, and M. Kreitman, “Functional analysis of eve stripe 2 enhancer evolution in Drosophila, [84] J. S. Mattick, “Non-coding RNAs: the architects of eukaryotic rules governing conservation and change,” Development, vol. complexity,” EMBO Reports, vol. 2, no. 11, pp. 986–991, 2001. 125, no. 5, pp. 949–958, 1998. [85] J. A. de Visser, J. Hermisson, G. P. Wagner, et al., “Perspective: [104] S. R. Neves and R. Iyengar, “Modeling of signaling networks,” evolution and detection of genetic robustness,” Evolution, vol. BioEssays, vol. 24, no. 12, pp. 1110–1117, 2002. 57, no. 9, pp. 1959–1972, 2003. [105] R. A. Fisher, Genetical Theory of Natural Selection, Clarendon, [86] D. C. Krakauer and J. B. Plotkin, “Redundancy, antiredun- Oxford, UK, 1930. dancy, and the robustness of genomes,” Proceedings of the [106] J. B. S. Haldane, The Causes of Evolution,Harperand Row, National Academy of Sciences (USA), vol. 99, no. 3, pp. 1405– New York, NY, USA, 1932. 1409, 2002. [107] S. Wright, “Evolution in Mendelian populations,” Genetics, [87] J. D. Bloom, S. T. Labthavikul, C. R. Otey, and F. H. Arnold, vol. 16, no. 2, pp. 97–159, 1931. “Protein stability promotes evolvability,” Proceedings of the [108] M. Kimura, “Evolutionary rate at the molecular level,” National Academy of Sciences (USA), vol. 103, no. 15, pp. 5869–5874, 2006. Nature, vol. 217, no. 5129, pp. 624–626, 1968. [109] J. H. Gillespie, “The role of population size in molecular [88] B. C. Daniels,Y.-J. Chen,J.P.Sethna,R.N.Gutenkunst, and C. R. Myers, “Sloppiness, robustness, and evolvability in evolution,” Theoretical Population Biology, vol. 55, no. 2, pp. 145–156, 1999. systems biology,” Current Opinion in Biotechnology, vol. 19, no. 4, pp. 389–395, 2008. [110] J. H. Gillespie, “Is the population size of a species relevant to its evolution?” Evolution, vol. 55, no. 11, pp. 2161–2169, [89] S. F. Elena and R. Sanjuan, “The effect of genetic robustness on evolvability in digital organisms,” BMC Evolutionary 2001. Biology, vol. 8, no. 1, article 284, 2008. [111] T. Ohta, “Population size and rate of evolution,” Journal of Molecular Evolution, vol. 1, no. 4, pp. 305–314, 1972. [90] R. C. McBride, C. B. Ogbunugafor, and P. E. Turner, “Robustness promotes evolvability of thermotolerance in an [112] T. Ohta, “The nearly neutral theory of molecular evolution,” RNA virus,” BMC Evolutionary Biology, vol. 8, no. 1, article Annual Review of Ecology and Systematics, vol. 23, no. 1, pp. 231, 2008. 263–286, 1992. 24 Journal of Artificial Evolution and Applications [113] T. Ohta and M. Kimura, “On the constancy of the evolution- adaptive mutation controversy,” Annual Review of Microbi- ary rate of cistrons,” Journal of Molecular Evolution, vol. 1, no. ology, vol. 60, pp. 477–501, 2006. 1, pp. 18–25, 1971. [130] J. R. Roth and D. I. Andersson, “Adaptive mutation: how [114] M. Woolfit and L. Bromham, “Population size and molecular growth under selection stimulates Lac+ reversion by increas- evolution on islands,” Proceedings of the Royal Society B, vol. ing target copy number,” Journal of Bacteriology, vol. 186, no. 272, no. 1578, pp. 2277–2282, 2005. 15, pp. 4855–4860, 2004. [115] S. Wright, J. Keeling, and L. Gillman, “The road from Santa [131] H. Hendrickson, E. S. Slechta, U. Bergthorsson, D. I. Rosalia: a faster tempo of evolution in tropical climates,” Andersson, and J. R. Roth, “Amplification-mutagenesis: evi- Proceedings of the National Academy of Sciences (USA), vol. dence that “directed” adaptive mutation and general hyper- 103, no. 20, pp. 7718–7722, 2006. mutability result from growth with a selected gene amplifica- [116] G. C. Stevens, “The latitudinal gradient in geographical tion,” Proceedings of the National Academy of Sciences (USA), range: how so many species coexist in the tropics,” The vol. 99, no. 4, pp. 2164–2169, 2002. American Naturalists, vol. 133, no. 2, pp. 240–256, 1989. [132] E. Kugelberg, E. Kofoid, A. B. Reams, D. I. Andersson, and J. [117] S. F. Elena, C. O. Wilke, C. Ofria, and R. E. Lenski, “Effects R. Roth, “Multiple pathways of selected gene amplification of population size and mutation rate on the evolution of during adaptive mutation,” Proceedings of the National mutational robustness,” Evolution, vol. 61, no. 3, pp. 666–674, Academy of Sciences (USA), vol. 103, no. 46, pp. 17319–17324, [118] W. Banzhaf, S. Harding, W. Langdon, and G. Wilson, “Accel- [133] E. S. Slechta, K. L. Bunny, E. Kugelberg, E. Kofoid, D. erating genetic programming through graphics processing I. Andersson, and J. R. Roth, “Adaptive mutation: general units,” in Genetic Programming Theory and Practice VI,pp. mutagenesis is not a programmed response to stress but 229–248, Springer, Berlin, Germany, 2009. results from rare coamplification of dinB with lac,” Proceed- [119] H. Juille ´ and J. B. Pollack, “Massively parallel genetic ings of the National Academy of Sciences (USA), vol. 100, no. programming,” in Advances in Genetic Programming, vol. 2, 22, pp. 12847–12852, 2003. pp. 339–357, MIT Press, Cambridge, Mass, USA, 1996. [134] D. M. Weinreich, N. F. Delaney, M. A. DePristo, and D. [120] W. B. Langdon, “A many threaded CUDA interpreter for L. Hartl, “Darwinian evolution can follow only very few genetic programming,” in Proceedings of the 13th European mutational paths to fitter proteins,” Science, vol. 312, no. Conference on Genetic Programming (EuroGP ’10), vol. 6021 5770, pp. 111–114, 2006. of Lecture Notes in Computer Science, pp. 146–158, Springer, [135] M. A. Bedau and R. Seymour, “Adaptation of mutation rates in a simple model of evolution,” in Complex Systems: Mecha- [121] F. G. Lobo and C. F. Lima, “A review of adaptive population nism of Adaptation, pp. 179–183, IOS Press, Amsterdam, The sizing schemes in genetic algorithms,” in Proceedings of Netherlands, 1994. the Workshops on Genetic and Evolutionary Computation [136] S. Ohno, Evolution by Gene Duplication, Springer, Berlin, (GECCO ’05), pp. 228–234, ACM, 2005. Germany, 1970. [122] D. E. Goldberg, “Sizing populations for serial and parallel [137] M. Nei, “Gene duplication and nucleotide substitution in genetic algorithms,” in Proceedings of the 3rd International evolution,” Nature, vol. 221, no. 5175, pp. 40–42, 1969. Conference on Genetic Algorithms, pp. 70–79, Morgan Kauf- [138] J. Spring, “Genome duplication strikes back,” Nature Genet- mann, San Francisco, CA, USA, 1989. ics, vol. 31, no. 2, pp. 128–129, 2002. [123] D. E. Goldberg, K. Deb, and J. H. Clark, “Genetic algorithms, [139] J. Zhang, “Evolution by gene duplication: an update,” Trends noise, and the sizing of populations,” Complex Systems, vol. in Ecology and Evolution, vol. 18, no. 6, pp. 292–298, 2003. 6, no. 4, pp. 333–362, 1992. [140] J. A. Bailey, Z. Gu, R. A. Clark, et al., “Recent segmental [124] J. Arabas, Z. Michalewicz, and J. Mulawka, “GAVaPS—a duplications in the human genome,” Science, vol. 297, no. genetic algorithm with varying population size,” in Proceed- 5583, pp. 1003–1007, 2002. ings of the 1st IEEE Conference on Evolutionary Computation [141] M. Nei and A. P. Rooney, “Concerted and birth-and-death (CEC ’94), vol. 1, pp. 73–78, IEEE, 1994. evolution of multigene families,” Annual Review of Genetics, [125] G. R. Harik and F. G. Lobo, “A parameter-less genetic vol. 39, no. 15, pp. 121–152, 2005. algorithm,” in Proceedings of the Genetic and Evolutionary [142] P. M. Harrison, H. Hegyi, S. Balasubramanian, et al., Computation Conference (GECCO ’99), pp. 258–267, Morgan “Molecular fossils in the human genome: identification and Kaufmann, San Francisco, CA, USA, 1999. analysis of the pseudogenes in chromosomes 21 and 22,” [126] T. Back, A. E. Eiben, and N. A. L. van der Vaart, “An empirical Genome Research, vol. 12, no. 2, pp. 272–280, 2002. study on GAs “with-out parameters”,” in Proceedings of the [143] J. Zhang, H. F. Rosenberg, and M. Nei, “Positive Darwinian 6th International Conference on Parallel Problem Solving from selection after gene duplication in primate ribonuclease Nature (PPSN ’00), vol. 1917 of Lecture Notes in Computer genes,” Proceedings of the National Academy of Sciences (USA), Science, pp. 315–324, Springer, 2000. vol. 95, no. 7, pp. 3708–3713, 1998. [127] A. E. Eiben, E. Marchiori, and V. A. Valko, ´ “Evolutionary [144] D. Dykhuizen and D. L. Hartl, “Selective neutrality of 6PGD algorithms with on-the-fly population size adjustment,” in allozymes in E. coli and the effects of genetic background,” Proceedings of the 8th International Conference on Parallel Genetics, vol. 96, no. 4, pp. 801–817, 1980. Problem Solving from Nature (PPSN ’04), vol. 3242 of Lecture Notes in Computer Science, pp. 41–50, Springer, 2004. [145] L. Altenberg, “Knowledge representation in the genome: new [128] M. Nei, “Selectionism and neutralism in molecular evolu- genes, exons, and pleiotropy,” Genetics, vol. 110, p. s41, 1985. tion,” Molecular Biology and Evolution, vol. 22, no. 12, pp. [146] L. Altenberg, “Genome growth and the evolution of the 2318–2342, 2005. genotype-phenotype map,” in Proceedings of the Evolution [129] J. R. Roth, E. Kugelberg, A. B. Reams, E. Kofoid, and D. and Biocomputation Workshop, vol. 899 of Lecture Notes in I. Andersson, “Origin of mutations under selection: the Computer Science, pp. 205–259, Springer, 1995. Journal of Artificial Evolution and Applications 25 [147] D. Sankoff, “Gene and genome duplication,” Current Opinion Problem Solving from Nature (PPSN ’92), pp. 269–278, in Genetics and Development, vol. 11, no. 6, pp. 681–684, North-Holland, 1992. 2001. [166] W. B. Langdon, “Size fair and homologous tree genetic pro- [148] F. Gruau, “Genetic micro programming of neural networks,” gramming crossovers,” Genetic Programming and Evolvable in Advances in Genetic Programming, pp. 495–518, MIT Press, Machines, vol. 1, no. 1, pp. 95–119, 2000. Cambridge, Mass, USA, 1994. [167] P. Nordin, W. Banzhaf, and F. Francone, “Efficient evolution [149] J. R. Koza, “Gene duplication to enable genetic programming of machine code for cisc architectures using blocks and to concurrently evolve both the architecture and work- homologous crossover,” in Advances in Genetic Programming, performing steps of a computer program,” in Proceedings of vol. 3, pp. 275–299, MIT Press, Cambridge, Mass, USA, 1999. the 14th International Joint Conference on Artificial Intelli- [168] R. Poli and N. F. Mcphee, “Exact schema theory for gp gence, pp. 695–717, Morgan Kaufmann, San Francisco, CA, and variable-length gas with homologous crossover,” in USA, 1995. Proceedings of the Genetic and Evolutionary Computation [150] K. Schmitt, “Using gene duplication and gene deletion Conference (GECCO ’01), pp. 104–111, Morgan Kaufmann, in evolution strategies,” Tech. Rep., Collaborative Research San Francisco, CA, USA, 2001. Center 531, University of Dortmund, Dortmund, Germany, [169] M. Srinivas and L. M. Patnaik, “Adaptive probabilities 2004. of crossover and mutation in genetic algorithms,” IEEE [151] M. Schutz, “Other operations: gene duplication and dele- Transactions on Systems, Man and Cybernetics, vol. 24, no. 4, tion,” in Handbook of Evolutionary Computation, pp. 8–15, pp. 656–667, 1994. Oxford University Press, Oxford, UK, 1997. [170] J. W. Pepper, “The evolution of evolvability in genetic linkage [152] R. Calabretta, S. Nolfi, D. Parisi, and G. P. Wagner, “Dupli- patterns,” BioSystems, vol. 69, no. 2-3, pp. 115–126, 2003. cation of modules facilitates the evolution of functional [171] G. G. Simpson, “The Baldwin effect,” Evolution, vol. 7, no. 2, specialization,” Artificial Life, vol. 6, no. 1, pp. 69–84, 2000. pp. 110–117, 1953. [153] M. C. King and A. C. Wilson, “Evolution at two levels in [172] H. A. Simon, “The organization of complex systems,” in humans and chimpanzees,” Science, vol. 188, no. 4184, pp. Hierarchy Theory: The Challenge of Complex Systems,pp. 1– 107–116, 1975. 27, George Braziller, 1973. [154] J. H. Moore, F. W. Asselbergs, and S. M. Williams, “Bioin- [173] H. A. Simon, “The architecture of complexity,” Proceedings formatics challenges for genome-wide association studies,” of the American Philosophical Society, vol. 106, pp. 467–482, Bioinformatics, vol. 26, no. 4, pp. 445–455, 2010. 1962. [155] D. Fredman, S. J. White, S. Potter, E. E. Eichler, et al., [174] L. Altenberg, “The schema theorem and Price’s theorem,” “Complex SNP-related sequence variation in segmental in Fundations of Genetic Algorithms, vol. 3, pp. 23–49, MIT genome duplications,” Nature Genetics,vol. 36, no.8,pp. Press, Cambridge, Mass, USA, 1995. 861–866, 2004. [175] J. T. Bonner, The Evolution of Complexity by Means of Natural [156] C. J. Shaw and J. R. Lupski, “Implications of human genome Selection, Princeton University, Princeton, NJ, USA, 1988. architecture for rearrangement-based disorders: the genomic [176] J. E. G. Leigh, “Levels of selection, poten-tial conflicts, and basis of disease,” Human Molecular Genetics, vol. 13, no. 1, their resolution: the role of the “common good”,” in Levels of pp. R57–R64, 2004. SelectioninEvolution, pp. 15–30, Princeton University Press, [157] D. Sankoff, “Rearrangements and chromosomal evolution,” Princeton, NJ, USA, 1999. Current Opinion in Genetics and Development,vol. 13, no.6, [177] S. T. Kazadi, D. Lee, R. Modi, J. Sy, and W. Lue, “Levels of pp. 583–587, 2003. compartmentalization in artificial evolution,” in Proceedings [158] D. Posada, K. A. Crandall, and E. C. Holmes, “Recombina- of the Genetic and Evolutionary Computation Conference tion in evolutionary genomics,” Annual Review of Genetics, (GECCO ’00), pp. 841–848, Morgan Kaufmann, San Fran- vol. 36, pp. 75–97, 2002. cisco, CA, USA, 2000. [159] G. P. Smith, “Evolution of repeated DNA sequences by [178] S. C. Manrubia and C. Briones, “Modular evolution unequal crossover,” Science, vol. 191, no. 4227, pp. 528–535, and increase of functional complexity in replicating RNA 1976. molecules,” RNA, vol. 13, no. 1, pp. 97–107, 2007. [160] A. S. Kondrashov, “Deleterious mutations and the evolution [179] T. Back, “Self-adaptation in genetic algorithms,” in Proceed- of sexual reproduction,” Nature, vol. 336, no. 6198, pp. 435– ings of the 1st European Conference on Artificial Life: Toward 440, 1988. a Practice of Autonomous Systems, pp. 263–271, MIT Press, [161] L. Hadany and T. Beker, “Sexual selection and the evolution 1992. of obligatory sex,” BMC Evolutionary Biology, vol. 7, no. 1, [180] N. Hansen and A. Ostermeier, “Completely derandomized article 245, 2007. self-adaptation in evolution strategies,” Evolutionary Compu- [162] A. Kong, D. F. Gudbjartsson, J. Sainz, et al., “A high- tation, vol. 9, no. 2, pp. 159–195, 2001. resolution recombination map of the human genome,” [181] R. Hinterding, Z. Michalewicz, and A. Eiben, “Adaptation Nature Genetics, vol. 31, no. 3, pp. 241–247, 2002. in evolutionary computation: a survey,” in Proceedings of the [163] A. E. Eiben, “Multiparent recombination in evolutionary IEEE Conference on Evolutionary Computation (CEC ’97),pp. computing,” in Advances in Evolutionary Computing: Theory 65–69, IEEE, 1997. and Applications, pp. 175–192, Springer, Berlin, Germany, [182] H. Kita, “A comparison study of self-adaptation in evolution 2003. strategies and real-coded genetic algorithms,” Evolutionary [164] D. E. Goldberg, B. Korb, and K. Deb, “Messy genetic Computation, vol. 9, no. 2, pp. 223–241, 2001. algorithms: motivation, analysis, and first results,” Complex [183] B. Edmonds, “Meta-genetic programming: co-evolving the Systems, vol. 3, no. 5, pp. 493–530, 1989. operators of variation,” Turkish Journal of Electrical Engineer- [165] I. Harvey and I. Harvey, “The SAGA cross: the mechanics of ing and Computer Sciences, vol. 9, pp. 13–29, 2001. recombination for species with variable length genotypes,” [184] W. Kantschik, P. Dittrich, M. Brameier, and W. Banzhaf, in Proceedings of the 2nd International Conference on Parallel “Empirical analysis of different levels of meta-evolution,” 26 Journal of Artificial Evolution and Applications in Proceedings of the Congress on Evolutionary Computation [201] I. Shmulevich, S. A. Kauffman, and M. Aldana, “Eukaryotic (CEC ’99), vol. 3, pp. 2086–2093, IEEE Press, 1999. cells are dynamically ordered or critical but not chaotic,” Proceedings of the National Academy of Sciences (USA), vol. [185] L. Spector and A. Robinson, “Genetic programming and 102, no. 38, pp. 13439–13444, 2005. autoconstructive evolution with the push programming language,” Genetic Programming and Evolvable Machines, vol. [202] J. R. Stone and G. A. Wray, “Rapid evolution of cis-regulatory 3, no. 1, pp. 7–40, 2002. sequences via local point mutations,” Molecular Biology and [186] J. Tavares, P. Machado, A. Cardoso, F. B. Pereira, and E. Costa, Evolution, vol. 18, no. 9, pp. 1764–1770, 2001. “On the evolution of evolutionary algorithms,” in Proceedings [203] A. Wagner, “Evolution of gene networks by gene duplica- of the 7th European Conference on Genetic Programming tions: a mathematical model and its implications on genome (EuroGP ’04), vol. 3003 of Lecture Notes in Computer Science, organization,” Proceedings of the National Academy of Sciences pp. 389–398, Springer, 2004. (USA), vol. 91, no. 10, pp. 4387–4391, 1994. [187] M. Parter, N. Kashtan, and U. Alon, “Facilitated variation: [204] J. W. Fondon III and H. R. Garner, “Molecular origins of how evolution learns from past environments to generalize rapid and continuous morphological evolution,” Proceedings to new environments,” PLoS Computational Biology, vol. 4, of the National Academy of Sciences (USA), vol. 101, no. 52, no. 11, article e1000206, 2008. pp. 18058–18063, 2004. [188] C. D. Allis, T. Jenuwein, and D. Reinberg, Eds., Epigenetics, [205] W. Banzhaf, “Artificial regulatory networks and genetic Cold Spring Harbor Laboratory, Woodbury, NY, USA, 2007. programming,” in Genetic Programming Theory and Practice, [189] J. S. Mattick, “Challenging the dogma: the hidden layer of pp. 43–62, Kluwer Academic Publishers, Dordrecht, The non-protein-coding RNAs in complex organisms,” BioEssays, Netherlands, 2003. vol. 25, no. 10, pp. 930–939, 2003. [206] W. Banzhaf, “On the dynamics of an artificial regulatory [190] S. A. Kauffman, “Metabolic stability and epigenesis in networks,” in Proceedings of the 7th European Conference on randomly constructed genetic nets,” Journal of Theoretical Artificial Life: Advances in Artificial Life, vol. 2801 of Lecture Biology, vol. 22, no. 3, pp. 437–467, 1969. Notes in Artificial Intelligence, pp. 217–227, Springer, 2003. [191] S. A. Kauffman, Origins of Order: Self-Organization and [207] W. Banzhaf and P. D. Kuo, “Network motifs in natural Selection in Evolution, Oxford University Press, Oxford, UK, and artificial transcriptional regulatory networks,” Journal of Biological Physics and Chemistry, vol. 4, no. 2, pp. 85–92, [192] R. Albert and H. G. Othmer, “The topology of the regulatory interactions predicts the expression pattern of the segment [208] P. Kuo, W. Banzhaf, and A. Leier, “Network topology and polarity genes in Drosophila melanogaster,” Journal of Theo- the evolution of dynamics in an artificial genetic regulatory retical Biology, vol. 223, no. 1, pp. 1–18, 2003. network model created by whole genome duplication and [193] C. Espinosa-Soto, P. Padilla-Longoria, and E. R. Alvarez- divergence,” BioSystems, vol. 85, no. 3, pp. 177–200, 2006. Buylla, “A gene regulatory network model for cell-fate deter- [209] A. Chavoya and Y. Duthen, “Evolving an artificial regulatory mination during Arabidopsis thaliana flower development network for 2D cell patterning,” in Proceedings of the IEEE that is robust and recovers experimental gene expression Symposium on Artificial Life (ALIFE ’07), pp. 47–53, IEEE, profiles,” Plant Cell, vol. 16, no. 11, pp. 2923–2939, 2004. Honolulu, Hawaii, USA, 2007. [194] S. Huang, G. Eichler, Y. Bar-Yam, and D. E. Ingber, “Cell [210] C. Mattiussi and D. Floreano, “Analog genetic encoding for fates as high-dimensional attractor states of a complex gene the evolution of circuits and networks,” IEEE Transactions on regulatory network,” Physical Review Letters, vol. 94, no. 12, Evolutionary Computation, vol. 11, no. 5, pp. 596–607, 2007. Article ID 128701, 2005. [211] C. L. Nehaniv, “Self-replication, evolvability and asyn- [195] S. Huang and D. E. Ingber, “Shape-dependent control of chronicity in stochastic worlds,” in Stochastic Algorithms: cell growth, differentiation, and apoptosis: switching between Foundations and Applications, pp. 126–169, Springer, Berlin, attractors in cell regulatory networks,” Experimental Cell Germany, 2005. Research, vol. 261, no. 1, pp. 91–103, 2000. [212] J. S. Mattick, “RNA regulation: a new genetics?” Nature [196] M. Aldana, E. Balleza, S. Kauffman, and O. Resendiz, Reviews Genetics, vol. 5, no. 4, pp. 316–323, 2004. “Robustness and evolvability in genetic regulatory networks,” [213] C. P. Mack and G. K. Owens, “Regulation of smooth muscle Journal of Theoretical Biology, vol. 245, no. 3, pp. 433–448, α-actin expression in vivo is dependent on CArG elements within the 5 and first intron promoter regions,” Circulation [197] E. Balleza, E. R. Alvarez-Buylla, A. Chaos, S. Kauffman, I. Research, vol. 84, no. 7, pp. 852–861, 1999. Shmulevich, and M. Aldana, “Critical dynamics in genetic [214] A. Gonc ¸ alves and E. Costa, “A computational model of regulatory networks: examples from four kingdoms,” PLoS gene regulatory networks and its topological properties,” One, vol. 3, no. 6, article e2456, 2008. in Proceedings of the 11th International Conference on the [198] M. Nykter, N. D. Price, M. Aldana, et al., “Gene expression Simulation and Synthesis of Living Systems (ALIFE ’08),pp. dynamics in the macrophage exhibit criticality,” Proceedings 204–211, 2008. of the National Academy of Sciences (USA), vol. 105, no. 6, pp. [215] R. Jaenisch and A. Bird, “Epigenetic regulation of gene 1897–1900, 2008. expression: how the genome integrates intrinsic and environ- [199] R. Serra, M. Villani, A. Graudenzi, and S. A. Kauffman, “Why mental signals,” Nature Genetics, vol. 33, pp. 245–254, 2003. a simple model of genetic regulatory networks describes the [216] S. B. Baylin and J. G. Herman, “DNA hypermethylation in distribution of avalanches in gene expression data,” Journal of Theoretical Biology, vol. 246, no. 3, pp. 449–460, 2007. tumorigenesis: epigenetics joins genetics,” Trends in Genetics, vol. 16, no. 4, pp. 168–174, 2000. [200] R. Serra, M. Villani, and A. Semeria, “Genetic network models and statistical properties of gene expression data in [217] C. D. Allis, T. Jenuwein, and D. Reinberg, “Overview and knock-out experiments,” Journal of Theoretical Biology, vol. concepts,” in Epigenetics, pp. 23–61, Cold Spring Harbor 227, no. 1, pp. 149–157, 2004. Laboratory, Woodbury, NY, USA, 2007. Journal of Artificial Evolution and Applications 27 [218] T. Jenuwein and C. D. Allis, “Translating the histone code,” [237] S. K. Sharma and G. W. Irwin, “Fuzzy coding of genetic Science, vol. 293, no. 5532, pp. 1074–1080, 2001. algorithms,” IEEE Transactions on Evolutionary Computation, [219] A. P. Wolffe and M. A. Matzke, “Epigenetics: regulation vol. 7, no. 4, pp. 344–355, 2003. through repression,” Science, vol. 286, no. 5439, pp. 481–486, [238] W. Banzhaf and J. Miller, “The challenge of complexity,” in 1999. Frontiers in Evolutionary Computation, pp. 243–260, Kluwer [220] S. I. S. Grewal and D. Moazed, “Heterochromatin and Academic Publishers, Dordrecht, The Netherlands, 2004. epigenetic control of gene expression,” Science, vol. 301, no. [239] D. Roggen and D. Federici, “Multi-cellular development: is 5634, pp. 798–802, 2003. there scalability and robustness to gain?” in Proceedings of the [221] A. S. Uyar and A. E. Harmanci, “A new population based 8th International Conference on Parallel Problem Solving from adaptive domination change mechanism for diploid genetic Nature (PPSN ’04), vol. 3242 of Lecture Notes in Computer algorithms in dynamic environments,” Soft Computing, vol. Science, pp. 391–400, 2004. 9, no. 11, pp. 803–814, 2005. [240] S. Harding and W. Banzhaf, “Artificial development,” in [222] W. Arthur, “The effect of development on the direction Organic Computing, pp. 201–220, Springer, Berlin, Germany, of evolution: toward a twenty-first century consensus,” 2008. Evolution and Development, vol. 6, no. 4, pp. 282–288, 2004. [241] S. Harding, J. F. Miller, and W. Banzhaf, “Self modifying cartesian genetic programming: fibonacci, squares, regres- [223] R. A. Raff, The Shape of Life: Genes, Development, and the Evolution of Animal Form, University of Chicago Press, sion and summing,” in Proceedings of the 12th European Chicago, Ill, USA, 1996. Conference on Genetic Programming (EuroGP ’09), vol. 5481 [224] M. West-Eberhard, Developmental Plasticity and Evolution, of Lecture Notes in Computer Science, pp. 133–144, Springer, Oxford University Press, Oxford, UK, 2003. Tubingen, Germany, 2009. [225] A. D. Bradshaw, “Unravelling phenotypic plasticity—why [242] A. P. Hendry, “Evolutionary biology: the power of natural should we bother?” New Phytologist, vol. 170, no. 4, pp. 644– selection,” Nature, vol. 433, no. 7027, pp. 694–695, 2005. 648, 2006. [243] L. Ancel-Meyers, F. D. Ancel, and M. Lachmann, “Evolution [226] T. J. Garland and S. A. Kelly, “Phenotypic plasticity and of genetic potential,” PLoS Computational Biology, vol. 1, no. experimental evolution,” The Journal of Experimental Biology, 3, pp. 236–243, 2005. vol. 209, no. 12, pp. 2344–2361, 2006. [244] A. Crombach and P. Hogeweg, “Evolution of evolvability in [227] T. D. Price, A. Qvarnstr, and D. E. Irwin, “The role of pheno- gene regulatory networks,” PLoS Computational Biology, vol. typic plasticity in driving genetic evolution,” Proceedings of 4, no. 7, article e1000112, 2008. the Royal Society B, vol. 270, no. 1523, pp. 1433–1440, 2003. [245] B. L. Phillips and R. Shine, “Adapting to an invasive species: [228] F. Valladares, D. Sanchez-Gomez, and M. A. Zavala, “Quan- toxic cane toads induce morphological change in Australian snakes,” Proceedings of the National Academy of Sciences titative estimation of phenotypic plasticity: bridging the gap between the evolutionary concept and its ecological (USA), vol. 101, no. 49, pp. 17150–17155, 2004. applications,” Journal of Ecology, vol. 94, no. 6, pp. 1103– [246] N. Kashtan and U. Alon, “Spontaneous evolution of mod- 1116, 2006. ularity and network motifs,” Proceedings of the National [229] W. T. Wcislo, “Behavioral environments and evolutionary Academy of Sciences (USA), vol. 102, no. 39, pp. 13773–13778, change,” Annual Review of Ecology and Systematics, vol. 20, 2005. pp. 137–169, 1989. [247] S. Collins, J. de Meaux, and C. Acquisti, “Adaptive walks [230] J. Draghi and G. P. Wagner, “Evolution of evolvability in a toward a moving optimum,” Genetics, vol. 176, no. 2, pp. developmental model,” Evolution, vol. 62, no. 2, pp. 301–315, 1089–1099, 2007. 2008. [248] T. Blickle and L. Thiele, “A comparison of selection schemes [231] J. C. Gerhart and M. W. Kirschner, Cells, Embryos, and used in evolutionary algorithms,” Evolutionary Computation, Evolution, Blackwell Science, Oxford, UK, 1997. vol. 4, no. 4, pp. 361–394, 1996. [232] P. Bentley and S. Kumar, “Three ways to grow designs: [249] D. E. Goldberg and K. Deb, “A comparative analysis of a comparison of embryogenies for an evolutionary design selection schemes used in genetic algorithms,” in Foundations problem,” in Proceedings of the Genetic and Evolutionary of Genetic Algorithms 1, pp. 69–93, Morgan Kaufmann, San Mateo, CA, USA, 1991. Computation Conference Workshop on Evolvability (GECCO ’99), pp. 35–43, Morgan Kaufmann, San Francisco, CA, USA, [250] M. Kirkpatrick and V. Ravigne, “Speciation by natural and 1999. sexual selection: models and experiments,” The American [233] F. Gruau, D. Whitley, and L. Pyeatt, “A comparison between Naturalist, vol. 159, no. 3, pp. S22–S35, 2002. cellular encoding and direct encoding for genetic neural [251] R. Lande, “Models of speciation by sexual selection on networks,” in Proceedings of the 1st Annual Conference on polygenic traits,” Proceedings of the National Academy of Genetic Programming (GECCO ’96), pp. 81–89, MIT Press, Sciences (USA), vol. 78, no. 6, pp. 3721–3725, 1981. Cambridge, Mass, USA, 1996. [252] N. Colegrave, “Sex releases the speed limit on evolution,” [234] G. S. Hornby and J. B. Pollack, “Creating high-level com- Nature, vol. 420, no. 6916, pp. 664–666, 2002. ponents with a generative representation for body-brain [253] W. J. Swanson and V. D. Vacquier, “The rapid evolution of evolution,” Artificial Life, vol. 8, no. 3, pp. 223–246, 2002. reproductive proteins,” Nature Reviews Genetics,vol. 3, no.2, [235] K. O. Stanley, “Generative and developmental systems,” in pp. 137–144, 2002. Proceedings of the Annual Conference Companion on Genetic [254] G. F. Miller, “Exploiting mate choice in evolutionary compu- and Evolutionary Computation Conference (GECCO ’09),pp. tation: sexual selection as a process of search, optimization, 3335–3354, ACM, New York, NY, USA, 2009. and diversification,” in Selected Papers from AISB Workshop [236] G. S. Hornby and J. B. Pollack, “The advantages of generative on Evolutionary Computing, pp. 65–79, Springer, Berlin, grammatical encodings for physical design,” in Proceedings of Germany, 1994. the IEEE Conference on Evolutionary Computation (CEC ’01), [255] G. F. Miller, The Mating Mind: How Sexual Choice Shaped vol. 1, pp. 600–607, IEEE, 2001. Human Nature, Anchor, Harpswell, Me, USA, 2001. 28 Journal of Artificial Evolution and Applications [256] J. H. Gillespie, Population Genetics: A Concise Guide, Johns Hopkins University Press, Baltimore, MD, USA, 2004. [257] J. F. Crow and M. Kimura, An Introduction to Population Genetics Theory, Harper and Row, New York, NY, USA, 1970. [258] H. A. Orr, “Fitness and its role in evolutionary genetics,” Nature Reviews Genetics, vol. 10, no. 8, pp. 531–539, 2009. [259] E. Sober, “The two faces of fitness,” in Thinking about Evolution: Historical, Philosophical, and Political Perspectives, pp. 309–321, Cambridge University Press, Cambridge, UK, [260] Y. Jin, “A comprehensive survey of fitness approximation in evolutionary computation,” Soft Computing,vol. 9, no.1,pp. 3–12, 2005. [261] M. B. Gerstein, C. Bruce, J. S. Rozowsky, et al., “What is a gene, post-ENCODE? History and updated definition,” Genome Research, vol. 17, no. 6, pp. 669–681, 2007. [262] T. R. Gingeras, “Origin of phenotypes: genes and transcripts,” Genome Research, vol. 17, no. 6, pp. 682–690, 2007. [263] K. Scherrer and J. Jost, “The gene and the genon concept: a functional and information-theoretic analysis,” Molecular Systems Biology, vol. 3, p. 87, 2007. [264] P. F. Stadler, S. J. Prohaska, C. V. Forst, and D. C. Krakauer, “Defining genes: a computational framework,” Theory in Biosciences, vol. 128, no. 3, pp. 165–170, 2009. [265] E. Birney, J. A. Stamatoyannopoulos, A. Dutta, et al., “Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project,” Nature, vol. 447, no. 7146, pp. 799–816, 2007.

Journal

Journal of Artificial Evolution and ApplicationsHindawi Publishing Corporation

Published: Jun 2, 2010

References