iGTP - short for Gene Tree Parsimony - is a highly specialised software for the inference of species trees from gene trees using parsimony criteria. It offers the criteria Minimising Deep Coalescences (MDC) to deal with incomplete lineage sorting and Minimising Gene Duplications as well as Minimising Gene Duplications and Losses for when dealing with gene families.
Input data are all the gene trees in Newick format in one text file. Several alleles or individuals in the same species should have the exact same name, which is rather unusual. Most other species tree programs use allele assignment tables.
Sadly it failed to work on Ubuntu at home; it started but did not actually do anything after importing the gene trees. Yesterday I finally gave iGTP a try on Windows. It worked fine with exactly the same gene tree file as I tried on Ubuntu, further demonstrating that there is something wrong with the Linux version I downloaded.
So how did it go? The program is fast and simple to use, but there are two little problems. First, it produced a very different tree for the same dataset than did *BEAST and ASTRAL, and the solutions found by the latter two make considerably more sense in the light of the morphology and biogeography of the species in question. I am afraid in this case MDC may have been mislead.
Second, iGTP has an integrated tree viewer that I found both unnecessarily complicated and, sorry to be so frank, ugly. Complicated in that it is some kind of fancy 3d viewer where a simple FigTree/TreeView style representation would be more helpful, and ugly in that the semi-transparent white font on an intensely blue background hurt my eyes. Seriously, I could barely make out the species names.
MDC is known to find the wrong answer under a very specific set of circumstances (just like parsimony on a single character matrix has a problem with long branch attraction), and maybe that is what is happening here. I am sure there are many cases where it will work better; in fact in the past I have often found that MDC and *BEAST would produce meaningful results where algorithmic/distance based approaches similar to ASTRAL would produce nonsense. But in this specific case I will leave it at having tried out how iGTP works.
Will update the big species tree post to include these new observations.
The blog of a systematic botanist of German origin, now working in Australia. It covers botany, phylogenetics, cladistics, science in general, freethought, and occasionally sillier issues.
Showing posts with label species trees. Show all posts
Showing posts with label species trees. Show all posts
Saturday, April 2, 2016
Sunday, March 27, 2016
Species tree method update: ASTRAL
I have just updated and restructured my long reference blog post on species tree methods. The occasion is that I have tried out ASTRAL for the first time today, so I decided to add it to the list.
TL;DR:
The only inputs are a text file with all the gene trees in Newick format and, if there are multiple alleles per species, an allele assignation table. In that case, also make sure to download the "multiple individuals" version here instead of the master version. The program needs Java and is run through the command line.
ASTRAL is fast and very flexible, and the results I got are meaningful although without branch lengths, limiting the downstream use cases. It is available for all three major operating systems and worked seamlessly on my Linux computer.
Downsides are the aforementioned command line operation, at least for people who are unfamiliar with that, and in my eyes the feeling that the developers are uncomfortable with and may not trust their current multiple individuals version ("The code for handling multiple individuals is in its infancy and might not work well yet").
Apart from adding ASTRAL and iGTP, I reorganised the long blog post to be a list of software tools in contrast to a list of methods. I hope it will continue to be useful to people searching the internet for an overview and comparison of species tree methods.
TL;DR:
The only inputs are a text file with all the gene trees in Newick format and, if there are multiple alleles per species, an allele assignation table. In that case, also make sure to download the "multiple individuals" version here instead of the master version. The program needs Java and is run through the command line.
ASTRAL is fast and very flexible, and the results I got are meaningful although without branch lengths, limiting the downstream use cases. It is available for all three major operating systems and worked seamlessly on my Linux computer.
Downsides are the aforementioned command line operation, at least for people who are unfamiliar with that, and in my eyes the feeling that the developers are uncomfortable with and may not trust their current multiple individuals version ("The code for handling multiple individuals is in its infancy and might not work well yet").
Apart from adding ASTRAL and iGTP, I reorganised the long blog post to be a list of software tools in contrast to a list of methods. I hope it will continue to be useful to people searching the internet for an overview and comparison of species tree methods.
Friday, February 26, 2016
Species trees from SNPs with SVD quartets
Today I have tried for the first time to use the SVD quartet method for inferring species trees from Single Nucleotide Polymorphism (SNP) data. This also marks my first foray into the new PAUP test versions.
In case it is unclear what this is about, the situation is that you have several samples per species and genome-wide SNPs for each of them, and you want to infer the species tree, that is a phylogeny that has species instead of your samples as the OTUs, under the assumption of the coalescent model. This post thus ties in with my previous posts on species tree methods for multi-locus data and the Bayesian approach to species trees from SNPs.
And just like those it will not at all be about the theory but about how to do it in practice. For those who fancy dozens of pages of mathematical formulas, the developers of the method, Chifman & Kubatko, have provided the background on arxiv. For the moment, however, I just wanted to try it out.
The newest version of the SVD quartet method is implemented in PAUP, so head over there and download it. I am using Linux, so I have to use the command line version, but that is okay. The main work of setting up a nexus file cannot be avoided anyway.
In case it is unclear what this is about, the situation is that you have several samples per species and genome-wide SNPs for each of them, and you want to infer the species tree, that is a phylogeny that has species instead of your samples as the OTUs, under the assumption of the coalescent model. This post thus ties in with my previous posts on species tree methods for multi-locus data and the Bayesian approach to species trees from SNPs.
And just like those it will not at all be about the theory but about how to do it in practice. For those who fancy dozens of pages of mathematical formulas, the developers of the method, Chifman & Kubatko, have provided the background on arxiv. For the moment, however, I just wanted to try it out.
The newest version of the SVD quartet method is implemented in PAUP, so head over there and download it. I am using Linux, so I have to use the command line version, but that is okay. The main work of setting up a nexus file cannot be avoided anyway.
Monday, December 15, 2014
Trying to use SNAPP
I really should rewrite my post on species tree methods one of these days. But at the moment I do not have the energy, and so I will simply write a few words on my experience with SNAPP.
Anybody who happens upon my species tree methods post will see that despite being more of a parsimony guy myself I have a lot of praise for BEAST. It is fast (for a Bayesian method) and very user-friendly. "Normal" BEAST is for standard gene tree phylogenetics, *BEAST or starBEAST is the add-on for species tree analyses based on multiple independent genes, and the still fairly novel SNAPP is the add-on for species tree analyses based on Single Nucleotide Polymorphism (SNP) data.
With genomic sequencing, SNPs are only going to become more important for the study of closely related organisms. If you have species that are very recently derived, any individual gene sequence is probably going to be extremely similar between them. This means that the approach of inferring species trees from the reconciliation of multiple gene trees is unlikely to work: instead of gene trees you are likely to get gene "combs", simply unresolved relationships.
There will still be thousands of little individual mutations differentiating your study specimens, but they will be distributed all across their genomes. This is why they are called SNPs: Single Nucleotide Polymorphisms each surrounded by conserved sequence regions.
The idea of SNAPP is now to use the SNPs from multiple samples per species directly to infer the species phylogeny, without any intermediate steps like alignments or gene trees. For this, it uses the coalescent model and the usual Bayesian Marcov Chain Monte Carlo approach. This sounds very attractive, especially after the good experiences with BEAST, and also very rigorous.
Unfortunately, so far my attempts at using SNAPP have been rather frustrating. There are three main issues:
Anybody who happens upon my species tree methods post will see that despite being more of a parsimony guy myself I have a lot of praise for BEAST. It is fast (for a Bayesian method) and very user-friendly. "Normal" BEAST is for standard gene tree phylogenetics, *BEAST or starBEAST is the add-on for species tree analyses based on multiple independent genes, and the still fairly novel SNAPP is the add-on for species tree analyses based on Single Nucleotide Polymorphism (SNP) data.
With genomic sequencing, SNPs are only going to become more important for the study of closely related organisms. If you have species that are very recently derived, any individual gene sequence is probably going to be extremely similar between them. This means that the approach of inferring species trees from the reconciliation of multiple gene trees is unlikely to work: instead of gene trees you are likely to get gene "combs", simply unresolved relationships.
There will still be thousands of little individual mutations differentiating your study specimens, but they will be distributed all across their genomes. This is why they are called SNPs: Single Nucleotide Polymorphisms each surrounded by conserved sequence regions.
The idea of SNAPP is now to use the SNPs from multiple samples per species directly to infer the species phylogeny, without any intermediate steps like alignments or gene trees. For this, it uses the coalescent model and the usual Bayesian Marcov Chain Monte Carlo approach. This sounds very attractive, especially after the good experiences with BEAST, and also very rigorous.
Unfortunately, so far my attempts at using SNAPP have been rather frustrating. There are three main issues:
- SNAPP appears to be rather capricious as to whether it will run at all or whether it will fall over. The only machine on which I can get it to run consistently is our family computer, a Linux machine. On the Windows machine at work it is also very consistent in that it always error messages and crashes.
- BEAST in general is known to have a problem with missing data although it can at least be tricked into accepting an allele missing for a species. Still, the same problem applies to SNAPP; a colleague had to throw out most of the data and samples he had to get missing data below ca. 5% before he could do an analysis. In my dataset that is just not possible, I'd be left with too few SNPs.
- Finally, SNAPP is really. Really. Really. Slow. A hundred SNPs, no problem, I can run a decent analysis over a day. Five hundred SNPs? Forget it. Our high performance computing cluster at work did 1,000 generations over a few hours, and I need it to do at least ten million generations; do the math. At home I just tried a dataset reduced to 200 SNPs, and it seems as if it will finish in three months. All that sounds like First World Problems, but the thing is, the whole point of genome-wide SNPs is that you have thousands of them. A SNAPP analysis of my whole dataset is just not going to happen, even if I did not have the missing data issue on top of it.
Thursday, July 18, 2013
*BEAST, species trees in general, and Bayesian versus parsimony
Alexei Drummond is visiting Canberra this week and today he has given a BEAST workshop hosted by our ANU/CSIRO Centre for Biodiversity Analysis. I have participated in the workshop and as one might expect I am most interested in the species tree analyses one can do with *BEAST. It was very rewarding and great fun. A few learnings I am taking from this, mostly as a kind of supplement to my species tree post from April, and a few thoughts:
The official position is that one can run an analysis with only one sample per species but it is not advisable because at least two samples are needed to estimate population sizes. From what people who have run their own analyses tell me, two samples are generally not enough for good results either.
That has an interesting consequence: because one should preferably have several samples per species but computing time explodes with larger sample numbers, these analyses are then only realistic for limited numbers of species. Unless, that is, you are prepared to assemble a ridiculously large dataset even for small studies and run your analysis on a supercomputer for a few weeks.
There are methodological alternatives (see my post linked above) but of course they have their own weaknesses. And as one of the course instructors pointed out, the parsimony based methods I like so much may come without such warnings and happily give me a species tree for every data set I throw at them, but they may give me a false sense of security where *BEAST would honestly show me how uncertain the result is.
There is a way around the problem of *BEAST not accepting missing data (i.e. one locus missing for a species): one can make a dummy sample of the species for that locus and fill the whole sequence with Ns. The analysis will then run without an error but it may take longer to mix.
It was confirmed that *BEAST will run with only one locus. This and the previous point are really important because it means that the program is even more flexible than I had assumed so far, apart from being reasonably fast and user friendly for a Bayesian phylogenetics tool.
---
It is funny to observe how different scientists see different methodological issues, and how everybody is convinced that the one they have experienced is the most important one. The same instructor mentioned above was really concerned about the problem of non-Bayesian methods giving me one best tree even if there might be two more or less two equally good tree islands in the landscape. A parsimony method, for example, would pick one of them but a properly done (!) Bayesian analysis would sample over both and then show the uncertainty.
Apart from the fact that one can implement measures of certainty also for non-Bayesian methods (and of course that has been done), Bayesian approaches come with their own host of issues. They are slow, there are all the controversies around prior selection, and they demand an enormous up front investment on part of the end user. What is gamma? What is theta? What chain heat should I chose? How do I know what is a realistic prior for any of these dozens of items? What substitution model to chose? How to evaluate whether the run has been sufficiently long? Why do I have to learn how to use at least four different programs for one measly analysis?
Of course, some would argue that one should not do an analysis if one is not willing to do it right, but that steep learning curve is definitely also reducing the accessibility of science and sometimes borders on Herrschaftswissen (a term for which there might not be a good English translation). Not everybody interested in the phylogenetic relationships of one genus can be expected to become one of the world experts in Bayesian phylogenetics.
And again, sometimes I find it good to know where the computer has its hands... Bayesian analyses come with a huge and ever increasing number of variables and, importantly, assumptions that one has to accept. Parsimony analyses, on the other hand, have one simple assumption: of two possible explanations, the simpler one is to be preferred. It is very clear what actually happens inside the computer when you use them. So, sometimes you need a Sojus to get somewhere, but in other situations you would be better served just taking your bicycle.
Still, if you have the right data and can defend your assumptions, then BEAST is the most flexible and most sophisticated tool one may find.
---
Finally, do not try to conduct any significant MCMC run on an ASUS Eee PC Seashell series. Good for conferences and field trips, not so good for doing science.
The official position is that one can run an analysis with only one sample per species but it is not advisable because at least two samples are needed to estimate population sizes. From what people who have run their own analyses tell me, two samples are generally not enough for good results either.
That has an interesting consequence: because one should preferably have several samples per species but computing time explodes with larger sample numbers, these analyses are then only realistic for limited numbers of species. Unless, that is, you are prepared to assemble a ridiculously large dataset even for small studies and run your analysis on a supercomputer for a few weeks.
There are methodological alternatives (see my post linked above) but of course they have their own weaknesses. And as one of the course instructors pointed out, the parsimony based methods I like so much may come without such warnings and happily give me a species tree for every data set I throw at them, but they may give me a false sense of security where *BEAST would honestly show me how uncertain the result is.
There is a way around the problem of *BEAST not accepting missing data (i.e. one locus missing for a species): one can make a dummy sample of the species for that locus and fill the whole sequence with Ns. The analysis will then run without an error but it may take longer to mix.
It was confirmed that *BEAST will run with only one locus. This and the previous point are really important because it means that the program is even more flexible than I had assumed so far, apart from being reasonably fast and user friendly for a Bayesian phylogenetics tool.
---
It is funny to observe how different scientists see different methodological issues, and how everybody is convinced that the one they have experienced is the most important one. The same instructor mentioned above was really concerned about the problem of non-Bayesian methods giving me one best tree even if there might be two more or less two equally good tree islands in the landscape. A parsimony method, for example, would pick one of them but a properly done (!) Bayesian analysis would sample over both and then show the uncertainty.
Apart from the fact that one can implement measures of certainty also for non-Bayesian methods (and of course that has been done), Bayesian approaches come with their own host of issues. They are slow, there are all the controversies around prior selection, and they demand an enormous up front investment on part of the end user. What is gamma? What is theta? What chain heat should I chose? How do I know what is a realistic prior for any of these dozens of items? What substitution model to chose? How to evaluate whether the run has been sufficiently long? Why do I have to learn how to use at least four different programs for one measly analysis?
Of course, some would argue that one should not do an analysis if one is not willing to do it right, but that steep learning curve is definitely also reducing the accessibility of science and sometimes borders on Herrschaftswissen (a term for which there might not be a good English translation). Not everybody interested in the phylogenetic relationships of one genus can be expected to become one of the world experts in Bayesian phylogenetics.
And again, sometimes I find it good to know where the computer has its hands... Bayesian analyses come with a huge and ever increasing number of variables and, importantly, assumptions that one has to accept. Parsimony analyses, on the other hand, have one simple assumption: of two possible explanations, the simpler one is to be preferred. It is very clear what actually happens inside the computer when you use them. So, sometimes you need a Sojus to get somewhere, but in other situations you would be better served just taking your bicycle.
Still, if you have the right data and can defend your assumptions, then BEAST is the most flexible and most sophisticated tool one may find.
---
Finally, do not try to conduct any significant MCMC run on an ASUS Eee PC Seashell series. Good for conferences and field trips, not so good for doing science.
Monday, June 10, 2013
Update on species tree methods
I have updated my earlier post on species tree methods to include three methods made available on a very useful server and to reflect what I have recently learned about PhyloNet, STEM-hy and the speed (or rather, lack thereof) of Mesquite when trying to find MDC species trees for larger datasets. I have also promoted the post to appear among the links on the right.
Wednesday, April 24, 2013
Comparison of species tree methods
Update 10 June 2013: This post originally from 24 April 2013 has been updated extensively because I have since tried out a set of new species tree methods, got STEM to run and a bit more experience with some others. I have also promoted the post to one of the "recommended phylogenetic systematics" posts on this site despite not being about theory of classification.
Update 27 March 2016: Added ASTRAL and iGTP, restructured the post to be more software-focused.
-----
I spent part of the last few days trying out different species tree methods, partly to help a colleague produce an example tree that he can use in a workshop he is planning and partly because I want to infer a species tree for one of my own projects in the next few weeks. This post was written for two reasons: as a note to myself for future reference and as a pointer for somebody who might want to infer a species tree and does not know which of the many programs to choose. A person like myself a few days ago, one could say, only if they find this post via search engine it might save them some of the frustrations I experienced.
Note that this is not a post for a methods wonk or for somebody who wants to learn about the theoretical or methodological background. It is strictly from the end user perspective, directed at those who want to know what is available, how user friendly the tools are and where to get them.
If you don't know what this is about you might want to refer to my earlier post on the topic. To summarize: these days we mostly use molecular data, in particular the DNA sequences of genes or intergenic spacers, to infer the evolutionary relationships of species. However, any individual gene phylogeny may or may not be congruent with the species phylogeny or with other gene phylogenies because each species inherits a random subset of the pre-existing allele diversity in of its ancestral species. Alternatively, discrepancies between a gene tree and the species tree or between different gene trees may also arise from introgression, rare gene flow between distinct species.
Update 27 March 2016: Added ASTRAL and iGTP, restructured the post to be more software-focused.
-----
I spent part of the last few days trying out different species tree methods, partly to help a colleague produce an example tree that he can use in a workshop he is planning and partly because I want to infer a species tree for one of my own projects in the next few weeks. This post was written for two reasons: as a note to myself for future reference and as a pointer for somebody who might want to infer a species tree and does not know which of the many programs to choose. A person like myself a few days ago, one could say, only if they find this post via search engine it might save them some of the frustrations I experienced.
Note that this is not a post for a methods wonk or for somebody who wants to learn about the theoretical or methodological background. It is strictly from the end user perspective, directed at those who want to know what is available, how user friendly the tools are and where to get them.
If you don't know what this is about you might want to refer to my earlier post on the topic. To summarize: these days we mostly use molecular data, in particular the DNA sequences of genes or intergenic spacers, to infer the evolutionary relationships of species. However, any individual gene phylogeny may or may not be congruent with the species phylogeny or with other gene phylogenies because each species inherits a random subset of the pre-existing allele diversity in of its ancestral species. Alternatively, discrepancies between a gene tree and the species tree or between different gene trees may also arise from introgression, rare gene flow between distinct species.
Subscribe to:
Posts (Atom)