P082

1:University Paris - Saclay

The democratization of Whole Genome Sequencing (WGS) methods require robust benchmark methodologies. Existing benchmark studies for M. tuberculosis rely either on simplistic in silico genomes evolved from the H37Rv reference genome, or on de novo assembled genomes which require an additional complete genome alignment step. We introduce maketube, a new method for building more realistic evolved genomes using the extensive literature describing M. tuberculosis structural variation. This includes the transposition of large Inserting Sequences (IS), large deletions and duplications. These genomes can be built from any reference, and the comprehensive evolutionary steps can be traced back to this reference. We show that maketube genomes are realistic and more representative of the diversity found in different lineages of M. tuberculosis. Using these genomes, we show that genome to genome alignment tools miss 5% of variants, introducing noise to the performance evaluation. We evaluated the performance of two recently published variant calling methodology and our in-house genomic pipeline genotube. As traditional benchmarks overestimate tools performance,  methodologies with precision and recall previously described with near perfect precision and recall missed up to 10% of variants on maketube genome. At last we studied the local impact of structural variants and found that structural variations with sequence homology negatively affected the precision and recall of variant callers.