Inevitability versus contingency in evolution

This third post on evolution will deal with the question of contingency. I will partly recycle, and partly expand upon, my recent comments at Larry Moran's blog.

The question here is this: If evolution had to start all over again, would it result in the same kinds of organisms or in very different ones? Because we cannot actually rerun evolution on this planet, it could also be rephrased as what we would expect life on other planets to look like, either very similar to that on Earth or totally different. But because we cannot see life from other planets either, for now all we have is informed speculation either way.

The discussion is often simplified as being about two sides. On one side are those who are convinced that life as it is currently developed on Earth is inevitable, either for some teleological or religious reason (with humans as a necessary aspect of divine purpose) or because of a strong confidence in the power of selection to force organisms into a few optimal shapes. On  the other are those who believe that life as it is currently developed on Earth is only one of a myriad of completely different and equally probable outcomes, either because that would make life on other worlds more interesting or because they stress the importance of random mutations and genetic drift over that of selection.

But it is obvious that there is really more of a gradient between two poles. Dividing the spectrum into four more or less arbitrary sections, we could imagine the following positions:

1. Things had to turn out pretty much as they are in pretty much every detail, humans included, either because that is what god set evolution up to achieve or because that is what the universe works towards. If there is life on other Earth-like planets, it would by logical extension be expected to include the familiar green plants, insectoids, arachnoids, fish, lizards, mammals, and ultimately Star Trek style humanoids very similar to us except in minor details.
2. Things had to turn out pretty much as they are, at least in broad strokes, because there is a strictly limited number of working solutions to evolutionary challenges and because unused opportunities do not long remain so. If there is life on other Earth-like planets, it would by logical extension be expected to look vaguely familiar but with numerous differences in detail. There will most likely be something like worms, arthropods, molluscs and vertebrates but with different body plans than their counterparts on earth (e.g. vertebrates have six legs and the spine on the front/underside of the body). Land plants on other planets might be red or brown and have different sexuality than ours but they would be constrained to look somewhat like the plants on Earth, with root, stem and leaves. Sentient beings on other planets would hardly be humanoids but they would still be under some constraints: above a certain size and thus necessarily vaguely vertebrate-like or aquatic, with some appendages for manipulation of objects, some means of communication, etc. It is also unlikely that their ethics and society would be utterly incomprehensible to us because there is also a strictly limited number of working solutions to social challenges.
3. Evolution is strongly contingent on the material it has to work on, and over the long time scale of evolution this will lead to a butterfly effect. Consequently, the existence of any particular group of organisms that we are now familiar with is a fluke. If there is life on other Earth-like planets, it would by logical extension be expected to look quite different. In any individual case, there may not actually be anything remotely comparable to arthropods, to vertebrates, or to vascular plants, but there might be completely different groups of organisms that we can hardly imagine at the moment. Perhaps there are worlds where due to a meteorite impacting at the wrong time arthropods never evolved; perhaps there are even worlds covered in sponge-like plants with gently floating balloon-animals above them. It is even possible that life on most planets never gets more complex than bacteria. Sentience might be another complete fluke and arise hardly anywhere else in the universe, but if it does the sentient organisms would have to be expected to appear very alien to us.
4. Evolution is strongly contingent on the material it has to work on,  and over the long time scale of evolution this will lead to a butterfly effect. Consequently, the existence of any and all groups of organisms that we are now familiar with is a fluke. If there is life on other Earth-like planets, it would by logical extension be expected to look so different that we may at first not even be able to recognize it as life. Sentience might be another complete fluke and arise hardly anywhere else in the universe, but if it did the sentient organisms, their societies and their morals would have to be expected to be completely alien and incomprehensible to us.

Although to a certain degree restricted to speculation, we can probably immediately reject the extreme positions #1 and #4. As for the first, there is no evidence whatsoever for the existence of a god or universal intelligence guiding evolution towards humans. As for the last, even the most optimistic science fiction author will at some point have to acknowledge that not everything that is imaginable is also technically feasible, and that limits the types of organisms (and societies) that can actually exist in reality.

So the question is whether one is more convinced by the second or the third scenario. Personally, I lean towards the second, meaning I assume that life on other planets will probably look reasonably familiar. I would expect to find more or less the same general types of organisms to pop up everywhere - worms, something like arthropods, something vaguely like vertebrates, herbivores and carnivores, flying animals, plants with roots, stems and leaves - although not necessarily in the same chronological order and the same relative diversity.

What convinces me of a certain inevitability is that (1) we can observe so many cases of parallel evolution or convergence on Earth* and (2) if there is an unexploited resource or unused habitat it provides an enormous opportunity to those who can utilize or move into it.

There are two counter-arguments. The first one claims that the groups that have converged on the same solution already had the same prerequisites in place when they diverged: it is really contingency, the parallels are simply contingent on something in the common ancestor. Of course one can always say that but it would effectively mean that any possible observation is interpreted as fitting the contingency thesis, thus making it unscientific. And in some cases one would have to claim that all the contingency was already played out at the stage where animals were limbless tiny worms, which is really stretching the idea.

The second is to point at a few cases where no convergence happened, where something evolved only once. Obvious examples are vertebrates, land plants and eukaryotes. The problem here is that I do not see how any of the relevant characters or adaptations can be considered difficult. Vertebrates have an interior skeleton - starting with a worm evolving a bit of cartilage may not be that hard a thing to do, and if you are the first to achieve it the world is your oyster. Land plants have learned to grow on dry land. Yes, that is tough but you have a lot of time to get it right as long as the land is still empty. Eukaryotes have invented a few extra membrane folds and multiple linear chromosomes, and acquired one or two types of endosymbiontic bacteria (and that latter event happened several times in the history of life so at least that part cannot be rare and difficult).

Because these cases are far outweighed by the many cases of striking convergence, it seems more plausible to explain the few singular cases with the first successful group distorting the fitness landscape: once the first lineage of algae evolved an epidermis, vascular tissue and roots, all late-comers would find the dry land occupied by competitors that are already too well-adapted for them to even try.

But that does not mean that, say, red algae couldn't become land plants if the green algae had not already taken that seat, nor would it mean that such hypothetical red land plants would have any morphospace available that does not include leaves, shoots and roots. A large plant without roots wouldn't work. There is simply a limited number of solutions that are viable, and that is why they will always turn up again. Most likely also on other planets, given enough time.

Footnote

*) Just to mention a few: Streamlined body shape in swimming animals; legs; eyes; the major internal organs such as kidneys, brain, etc (although I will admit that echinoderms have a very aberrant body plan compared with the parallels between all other mobile animals); bills; wings; eusociality; maternal care; the various pollination syndromes; the various dispersal syndromes; heterospory; leaves (yes, invented at least four times independently in the land plants); secondary growth; parasitism; and of course rather stunning examples such as dodders and Cassytha or cacti and euphorbs.