kmtricks: Efficient construction of Bloom filters for large sequencing data collections - Inria - Institut national de recherche en sciences et technologies du numérique
Journal Articles Bioinformatics Advances Year : 2022

kmtricks: Efficient construction of Bloom filters for large sequencing data collections

Abstract

When indexing large collections of short-read sequencing data, a common operation that has now been implemented in several tools (Sequence Bloom Trees and variants, BIGSI,.) is to construct a collection of Bloom filters, one per sample. Each Bloom filter is used to represent a set of k-mers which approximates the desired set of all the non-erroneous k-mers present in the sample. However, this approximation is imperfect, especially in the case of metagenomics data. Erroneous but abundant k-mers are wrongly included, and non-erroneous but low-abundant ones are wrongly discarded. We propose kmtricks, a novel approach for generating Bloom filters from terabase-sized collections of sequencing data. Our main contributions are 1/ an efficient method for jointly counting k-mers across multiple samples, including a streamlined Bloom filter construction by directly counting, partitioning and sorting hashes instead of k-mers, which is approximately four times faster than state-of-the-art tools; 2/ a novel technique that takes advantage of joint counting to preserve low-abundant k-mers present in several samples, improving the recovery of non-erroneous k-mers. Our experiments highlight that this technique preserves around 8x more k-mers than the usual yet crude filtering of low-abundance k-mers in a large metagenomics dataset.
Fichier principal
Vignette du fichier
kmtricks.pdf (1.01 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03166007 , version 1 (11-03-2021)

Identifiers

Cite

Téo Lemane, Paul Medvedev, Rayan Chikhi, Pierre Peterlongo. kmtricks: Efficient construction of Bloom filters for large sequencing data collections. Bioinformatics Advances, 2022, ⟨10.1093/bioadv/vbac029⟩. ⟨hal-03166007⟩
290 View
429 Download

Altmetric

Share

More