You are here

Article Text

Article menu
Methods
Original article
SeqHBase: a big data toolset for family based sequencing data analysis
  1. Min He1,2,3,
  2. Thomas N Person1,
  3. Scott J Hebbring1,3,
  4. Ethan Heinzen4,
  5. Zhan Ye2,
  6. Steven J Schrodi1,3,
  7. Elizabeth W McPherson5,
  8. Simon M Lin6,
  9. Peggy L Peissig2,
  10. Murray H Brilliant1,3,
  11. Jason O'Rawe7,
  12. Reid J Robison8,
  13. Gholson J Lyon7,8,
  14. Kai Wang8,9,10
  1. 1Center for Human Genetics, Marshfield Clinic Research Foundation, Marshfield, Wisconsin, USA
  2. 2Biomedical Informatics Research Center, Marshfield Clinic Research Foundation, Marshfield, Wisconsin, USA
  3. 3Department of Computation and Informatics in Biology and Medicine, University of Wisconsin-Madison, Madison, Wisconsin, USA
  4. 4College of Science and Engineering, University of Minnesota-Twin Cities, Minnesota, Minnesota, USA
  5. 5Department of Medical Genetics Services, Marshfield Clinic, Marshfield, Wisconsin, USA
  6. 6The Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA
  7. 7Cold Spring Harbor Laboratory, Stanley Institute for Cognitive Genomics, Cold Spring Harbor, New York, USA
  8. 8Utah Foundation for Biomedical Research, Provo, Utah, USA
  9. 9Zilkha Neurogenetic Institute, University of Southern California, Los Angeles, California, USA
  10. 10Department of Psychiatry, University of Southern California, Los Angeles, California, USA
  1. Correspondence to Dr. Min He, Center for Human Genetics, Marshfield Clinic Research Foundation, 1000 N Oak Ave, Marshfield, WI 54449, USA; he.max{at}marshfieldclinic.org and Dr. Kai Wang, Zilkha Neurogenetic Institute, University of Southern California, 1501 San Pablo St, Los Angeles, CA 90089, USA; kaiwang@usc.edu

Abstract

Background Whole-genome sequencing (WGS) and whole-exome sequencing (WES) technologies are increasingly used to identify disease-contributing mutations in human genomic studies. It can be a significant challenge to process such data, especially when a large family or cohort is sequenced. Our objective was to develop a big data toolset to efficiently manipulate genome-wide variants, functional annotations and coverage, together with conducting family based sequencing data analysis.

Methods Hadoop is a framework for reliable, scalable, distributed processing of large data sets using MapReduce programming models. Based on Hadoop and HBase, we developed SeqHBase, a big data-based toolset for analysing family based sequencing data to detect de novo, inherited homozygous, or compound heterozygous mutations that may contribute to disease manifestations. SeqHBase takes as input BAM files (for coverage at every site), variant call format (VCF) files (for variant calls) and functional annotations (for variant prioritisation).

Results We applied SeqHBase to a 5-member nuclear family and a 10-member 3-generation family with WGS data, as well as a 4-member nuclear family with WES data. Analysis times were almost linearly scalable with number of data nodes. With 20 data nodes, SeqHBase took about 5 secs to analyse WES familial data and approximately 1 min to analyse WGS familial data.

Conclusions These results demonstrate SeqHBase's high efficiency and scalability, which is necessary as WGS and WES are rapidly becoming standard methods to study the genetics of familial disorders.

  • whole-genome sequencing
  • whole-exome sequencing
  • big data
  • de novo mutations
  • inherited homozygous or compound heterozygous mutations

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/jmedgenet-2014-102907

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Advances in next-generation sequencing (NGS) technologies have made it possible to systematically search for rare disease-contributing genetic variants in human genomic studies. Unlike population-based studies, rare disease-contributing variants can be enriched in families, such as trios (parents and an offspring) or other nuclear families. The availability of family data also facilitates detection of de novo mutations that may be disease-contributing1–4 and provides a valuable resource for NGS studies. Similarly, family based designs can identify many mutations contributing to recessive diseases, inherited as homozygous or compound heterozygous states. Whole-genome sequencing (WGS) and/or whole-exome sequencing (WES) data are being produced at an unprecedented rate, often on families with apparent familial diseases.5 Often, one single study generates multiple terabytes or even petabytes of sequencing data, including raw sequence reads, alignment files, variant calls and annotations. However, the design and development of efficient and scalable computational tools for analysing large sets of sequencing data have lagged far behind our ability to generate data.

    Apache Hadoop (http://hadoop.apache.org/) is an open-source infrastructure that allows for reliable and scalable distributed processing of large data sets across clusters of computers using MapReduce programming models. Apache HBase (http://hbase.apache.org/) is an open-source, distributed, versioned and non-relational database modelled after Google's Bigtable.6 Just as Bigtable leverages the distributed data storage provided by the Google File System,7 HBase provides Bigtable-like capabilities on top of Hadoop and Hadoop distributed file systems. HBase is designed to supply random read/write access to structured big data in real time. It can host very large tables, each capable of billions of rows with millions of columns. This capacity can be applied to manipulate hundreds of thousands of WGS samples, which will soon become reality, such as the 100K Genomes Project (http://www.genomicsengland.co.uk/). A number of Hadoop-based tools for processing sequencing data have been developed and applied to NGS studies, such as quality control,8 alignment,9–11 single-nucleotide polymorphism (SNP) calling,10 variant annotation12 and general workflow management.13 In addition, there are many other projects that are built based on Hadoop infrastructure for NGS studies. These include the Hadoop-BAM14 library for manipulating BAM files based on Hadoop, and SeqPig,15 which is a library based on Apache Pig (http://pig.apache.org/) for using the advanced high-level features of Pig to manipulate aligned and unaligned sequence data via writing scripts. However, there is a critical need for a reliable computational toolset that can efficiently manipulate genome-wide sets of variants, their functional annotations and every-site coverage (read depth), as well as analyse NGS data to detect disease-contributing genes based on a scalable big data infrastructure. Leveraging Apache Hadoop and HBase, we have created SeqHBase, a big data-based toolset designed for analysing large family based sequencing data to detect mutations that may be disease-contributing.

    Materials and methods

    Framework of SeqHBase

    The basic framework of SeqHBase is described in figure 1. SeqHBase manipulates coverage information (every site in the genome) as well as genetic variations and their functional annotations for use in the analysis of each pedigree. Leveraging Apache Hadoop and HBase, SeqHBase efficiently manipulates, stores and retrieves these sequencing data via MapReduce programming models. As described in figure 1, users can load three different types of sequencing data into HBase by providing BAM or pileup files generated by SAMtools16 for coverage information, VCF or vcf.gz files for genetic variations, and comma-separated values (CSV) files for annotated variants. SeqHBase then uses the MapReduce model to split the input data set into independent chunks that are processed by the map tasks in a completely parallel manner. In conjunction with a pedigree file, coverage information, genetic variations and variant annotations are extracted by the reduce tasks in a parallel manner. SeqHBase then generates de novo, inherited homozygous, or compound heterozygous mutations using an analysis engine developed based on the big data infrastructure. SeqHBase was developed in Java and it is freely available for use by academic or non-profit organisations at http://seqhbase.omicspace.org/.

    Figure 1
    Figure 1

    The basic framework of SeqHBase: the sequencing data include annotated variants, genetic variations of every whole-genome sequencing/whole-exome sequencing (WGS/WES) sample, and coverage (read depth) of each site of every WGS/WES sample. Users can load three different types of sequencing data into HBase by providing CSV files for variants, VCF or vcf.gz files for variations, BAM or pileup files generated by SAMtools for coverage. Then SeqHBase uses a MapReduce model to split the input data set into independent chunks that are processed by the map tasks in a completely parallel manner. Given a pedigree file for analysing a data set, SeqHBase extracts variant, variation and coverage information using reduce tasks in a parallel manner for each sample. Finally, SeqHBase uses inheritance information for detecting de novo, inherited homozygous or compound heterozygous mutations that may be disease-contributing in trios, nuclear families and/or extended families.

    Extraction of variant, variation and quality information from sequencing data

    Variant calling, especially INDELs, along with calling of larger deletions and insertions (>5 bp) is still a considerable challenge.17 In this work, we will not address variant calling. We assume variants are well-called and the VCF and BAM files of every sequencing sample are available. SeqHBase accepts three types of files as input: (1) annotated variant files generated by an annotation programme such as ANNOVAR18 or others. These files can be simple comma or tab-delimited text (.csv) files, where each column represents one piece of annotation information and each row represents one variant. The files are used for extracting variant information, including chromosome number, start position, end position, reference allele, alternative allele, frequency in the 1000 Genome Project19 and/or the EPS6500 project (http://evs.gs.washington.edu/EVS/), ClinVar,20 Combined Annotation Dependent Depletion (CADD) score,21 biological function and multiple diverse function-relevant scores, such as PolyPhen-2 score,22 Sorting Intolerant From Tolerant (SIFT) score23 and others; (2) VCF files for extracting variation information, including sample family ID, individual ID, called variant genotypes, coverage (read depth) and Phred quality scores; (3) BAM files for extracting coverage of each site (∼3 billion sites in a WGS) of every sequencing sample. The diverse types of extracted information are also described in table 1. In downstream analyses, the coverage information can be used to identify if no-call sites are reference-consistent with high quality or reference-inconsistent caused by low quality mapping. A specific function within SeqHBase is developed for directly generating the read depths of each site from BAM files, similar to SAMtools pileup function.

    View this table:
    Table 1

    Extracted information from three types of input files

    Detection of de novo, inherited homozygous or compound heterozygous mutations

    Detection of de novo mutations is based on input parameters provided by users, such as variant frequency in 1000 Genome Project and/or 6500 Exome Sequencing Project (ESP6500) population, minimum read depth, biological functions of interest and predicted functional deleteriousness scores (eg, PolyPhen-2). This filtering step ensures that only functionally relevant variants are examined in the steps below, and the parameters for this filtering step are adjustable by the user. Variants in parents and offspring(s) are examined for all potential de novo mutations where the affected carries a heterozygous variant and both parents carry high coverage (≥20× read depth) reference alleles. Similar to detecting de novo mutations, inherited mutations are also detected based on user-specified input parameters. Parental and affected variations are examined for all potential inherited homozygous mutations, where the affected carries a homozygous variant and both parents carry heterozygous variants. Potential compound heterozygous mutations are examined for every parent-offspring (affected) to ensure that the affected carries two different variants in the same gene region and the contributing variants are each inherited from a different parent. Furthermore, any number of unaffected siblings of the affected can be used to reduce false positive mutations, while any number of affected siblings of the affected can be used to augment the chance of detecting true disease-contributing mutations.

    Compilation of three family sequencing data sets

    To illustrate the utility of SeqHBase for manipulating sequencing data and detecting de novo, inherited homozygous and compound heterozygous mutations, we analysed three sets of sequencing data from three different family structures described in figure 2. Family 1 in figure 2 is a nuclear family that consists of two unaffected parents, two unaffected children and an affected child (female) with Rodriguez syndrome. The affected in the family had clinical features of severe micrognathia, with a normal chest, short forearms, absent fibula and clubbed feet.24 The samples of the five members in the family were sequenced using next-generation WGS technology by Illumina using a HiSeq sequencer. In addition to providing the BAM and VCF files of the five samples, Illumina also used its quality control pipeline to exclude low quality variants and provided a single nucleotide variant data set that included 3 355 802 clear single nucleotide variants carried by the affected. We then annotated the variants using ANNOVAR. Finally, we loaded the WGS data from the five samples, including the annotated variants carried by the affected, genetic variations of the five samples, and coverage information on each site from the five BAM files, into a Hadoop and HBase cluster built with 20 virtual machines (VMs). We focused on analysing rare variants (eg, variant frequencies ≤0.01 or not observed in the 1000 Genome Project and EPS6500 populations) with interesting biological functions, which included non-synonymous, stop-gain, stop-loss, splicing and frame-shift mutations. The two unaffected siblings’ data were used to systematically exclude false positive mutations by SeqHBase. For instance, if a de novo mutation was present in the affected and either of the two unaffected siblings, the mutation would be excluded from a candidate list in the resulting file, as the mutation is less likely to be a disease-contributing mutation with respect to the syndrome studied.

    Figure 2
    Figure 2

    Description of three families used in our benchmarking study. (1) Family 1 is a five-member nuclear family in which the affected individual has Rodriguez syndrome. One plausible de novo mutation and one possible compound heterozygous mutation were detected. (2) Family 2 is a four-member nuclear family where the affected individual has idiopathic haemolytic anaemia. One plausible gene with compound heterozygous mutations was detected. (3) Family 3 is a 10-member extended family with three generations where the two affecteds have an undiagnosed disease manifesting with intellectual disability, autism, attention deficit hyperactivity disorder and other symptoms. An X linked de novo mutation with the mother of the two affecteds was detected. Both affecteds inherited the mutation.

    The second data set,25 described in figure 2 as Family 2, was a WES study conducted at the Utah Foundation for Biomedical Research on a four-member nuclear family in which one child (male) was affected with idiopathic haemolytic anaemia (IHA), while his parents and a sibling were unaffected. The annotated variant data, genetic variations and coverage information at every site for the four whole exome samples were loaded into the SeqHBase framework following the procedure described above for the first data set. We analysed rare variants for interesting biological functions in the same manner as the analysis of the first WGS data set. The unaffected sibling's data were used by SeqHBase to systematically exclude false positive mutations.

    The third data set (O'Rawe et al, under review), described in figure 2 as Family 3, was a WGS study conducted at the Stanley Institute for Cognitive Genomics at Cold Spring Harbor Laboratory on a 10-member extended family with three generations. The affected individuals are two affected male siblings with severe intellectual disability, autistic behaviours, attention deficit hyperactivity disorder and very distinctive facial features. The other eight members, including grandparents, parents, an uncle, an aunt, an uncle-in-law and a first cousin, are unaffected, without any of the described symptoms. After loading the annotated variant, variation and coverage information into the SeqHBase framework, we analysed rare variants in the same way as described above for the other two data sets.

    Results

    Performance of SeqHBase

    To evaluate the efficiency and performance of SeqHBase, we used 5, 10, 15 and 20 data nodes (VMs) to test the running time of detecting de novo, inherited homozygous or compound heterozygous mutations in the affected individual from the first WGS data set (Family 1 in figure 2), which was composed of a five-member nuclear family. As predicted, the larger the number of data nodes, the shorter the running time with a nearly linear relationship between these two measures. SeqHBase takes approximately 16 s to run analysis on the five-member sequencing data set for detecting de novo and inherited homozygous (or X linked) mutations, and similar time for detecting compound heterozygous mutations. The procedure of detecting de novo and inherited homozygous (or X linked) mutations is separated from the one of detecting compound heterozygous mutations as it was a single variant-based search while the latter was a gene-based search (figure 3).

    Figure 3
    Figure 3

    SeqHBase running time in seconds when run on different numbers of data nodes. The data set for Family 1 in figure 2 was used to evaluate the performance of SeqHBase. Each data node was configured with 6 GB memory, two CPUs (2.6 GHz) and 1 TB hard disk space. Note that the performance of SeqHBase using a single data node is not evaluated due to lack of disk space to manipulate five WGS data sets within the same virtual machine.

    Analysis of a WGS data set on a five-member nuclear family

    To evaluate the functionality of SeqHBase, we first analysed a WGS data set from a five-member nuclear family that consisted of two unaffected parents, two unaffected children and an affected child with Rodriguez syndrome (Family 1 in figure 2).24 Rodriguez syndrome26 is a rare acrofacial dysostosis syndrome that is clinically distinguished from other acrofacial dysostosis syndromes, such as Miller syndrome27 and Nager syndrome,28 primarily by the severity and distribution of the limb defects.

    When first sequenced, a search for de novo mutations within a few interesting gene regions was performed. A de novo mutation in SF3B4 gene was detected and reported.24 We ran a genome-wide search for potential de novo, inherited homozygous or compound heterozygous mutations on the five-sample WGS data set for Family 1 using SeqHBase. After loading the WGS data of the five individuals into a Hadoop and HBase cluster built using 20 VMs, we collected and analysed rare variants with a coverage of ≥20× for every individual, variant frequencies (minor allele frequency)≤0.01 in the 1000 Genome Project and EPS6500 populations, and variants that were annotated as being non-synonymous, stop-gain, stop-loss, splicing or frame-shift changes. Based on the framework built using 20 VMs, SeqHBase took approximately 16 s to scan the whole genome, collect the rare variant list, and generate potential de novo and inherited homozygous (or X linked) mutations. This shows the efficiency and performance of SeqHBase for manipulating and analysing WGS data stored in big tables with multiple billions of records.

    When detecting de novo mutations, six candidate mutations were detected. One splicing mutation (chr1:149898811C>T) in SF3B4 (NM_005850:exon4:c.164-1G>A) was the most plausible candidate for an association with Rodriguez syndrome in the pedigree, as expected.29 An inherited homozygous mutation was detected in the analysis using the criteria described above, but the gene (OR56A5) has not shown any association with Nager syndrome or Rodriguez syndrome. When detecting compound heterozygous mutations, two candidates were found. These mutations are briefly summarised in table 2 and shown in more detail in online supplementary table S1. In addition, the two unaffected siblings’ data were used by SeqHBase to systematically exclude false positive mutations. For example, mutations present in both the affected and either of the two unaffected siblings were ruled out as likely disease-contributing mutations. In the procedure of detecting de novo mutations, three false positive de novo mutations were excluded, and in the procedure of detecting compound heterozygous mutations, six false positive compound heterozygous mutations were excluded. This exercise illustrated the clear value of having additional family members in ruling out potential false positive and negative variant calls.

    View this table:
    Table 2

    Brief results of family based sequencing data analysis*

    Analysis of a WES data set on a four-member nuclear family

    A WES data set of a four-member nuclear family was also available to test the functionality of SeqHBase. The nuclear family consisted of two unaffected parents, one unaffected sibling and an affected child who was diagnosed with IHA (Family 2 in figure 2). We previously analysed this data set and identified the disease-contributing gene as PKLR, which carried compound heterozygous disease-contributing mutations in the affected.25 We loaded the WES data of the four individuals into the same Hadoop and HBase cluster built using 20 VMs. We collected and analysed rare variants using the same analysis criteria as was used for analysing the first WGS data set. SeqHBase took approximately 5 s to scan the sequenced regions, collect the rare variant list, and generate potential de novo and inherited homozygous mutations. Similarly, it took approximately 5 s to detect compound heterozygous mutations.

    When detecting de novo mutations, 16 candidate mutations were detected. After a thorough literature search for the associated genes, none of them were associated with IHA, as IHA is mainly caused by mutations in genes involved in red cell metabolism (glycolysis, hexose monophosphate shunt and the purine salvage pathway).30 No homozygous (or X linked) mutations were detected in the analysis with the criteria described above. When detecting compound heterozygous mutations, two candidate mutations were detected. The affected carried compound heterozygous mutations, with a rare variant (chr1:155260382C>T) in PKLR (NM_000298:exon11:c.1706G>A:p.R569Q, NM_181871: exon11:c.1613G>A:p.R538Q) inherited from the mother and another rare variant (chr1:155264120C>G) in the same gene (NM_000298:exon7:c.1022G>C:p.G341A, NM_181871:exon7: c.929G>C:p.G310A) inherited from the father. PKLR is a known disease-contributing gene for haemolytic anaemia.31 This compound heterozygous mutation was also reported by Lyon et al,25 and it was replicated by SeqHBase using the analysis criteria described above. Another compound heterozygous mutation was not associated with the syndrome studied. The mutations are briefly summarised in table 2 and shown in more detail in online supplementary table S2. The unaffected sibling's data were used to systematically exclude false positive mutations. In the procedure of detecting de novo mutations, seven false positive de novo mutations were excluded, and in the procedure of detecting compound heterozygous mutations, one false positive compound heterozygous mutation was excluded.

    Analysis of a WGS data set on a 10-member three-generation family

    To further evaluate the functionality of SeqHBase, we analysed a large WGS data set from an extended family with three generations (Family 3 in figure 2). Two affecteds in the third generation are affected with severe intellectual disability, autistic behaviours, attention deficit hyperactivity disorder and very distinctive facial features. As with analysing the first and second sequencing data sets, the variants carried by the two affecteds were annotated using ANNOVAR and the annotated information was loaded into the same Hadoop and HBase cluster. The genetic variation information was extracted from the VCF files and loaded into the same cluster. The coverage information of every site of the 10 WGS samples was extracted from the BAM files and loaded to the cluster. We ran a genome-wide search for potential de novo, inherited homozygous or compound heterozygous mutations on the 10 WGS data sets using the criteria described in analysing the first data set. SeqHBase took approximately 80 s to collect the rare variant list and generate potential de novo and inherited mutations that might be disease-contributing. And it took similar time to generate a potential compound heterozygous mutation list.

    By detecting mutations carried by both of the two affecteds, SeqHBase generated a candidate list for disease-contributing mutations under different genetic models. When detecting de novo mutations, SeqHBase found 18 candidate mutations shared between the two affecteds. When detecting homozygous mutations, one inherited homozygous mutation in the two affecteds was detected in the analysis but the mutations in the gene are not known to be associated with a disease similar to the one studied here. Additionally, one X linked mutation (chrX:70621541T>C) located in TAF1 (NM_001286074:exon25:c.4010T>C:p.I1337T, NM_004606:exon25:c.4010T> C:p.I1337T, NM_138923:exon25:c.3947T>C:p.I1316T) was detected. Interestingly, the X linked non-synonymous mutation in TAF1 was detected as a de novo mutation arising in the mother of the two affecteds. This mutation appears to be a plausible candidate for an association with the syndrome studied in the pedigree, as the gene has been shown to be associated with X linked dystonia-parkinsonism,32 ,33 although further functional study is needed. When detecting compound heterozygous mutations, two candidate mutations, which are carried by the two affecteds individually, located in the same gene were detected. These mutations are briefly summarised in table 2 and shown in more detail in online supplementary table S3. Given the availability of the large pedigree, SeqHBase also used data from other unaffected family members to systematically exclude false positive variant calls. As a result, 14 false positive de novo mutations were excluded as false negatives in the parents of the affected. This example illustrated the power of SeqHBase to generate candidate disease-contributing mutations carried by the two affecteds.

    Discussion

    Using an Apache Hadoop and HBase framework, we developed a big data toolset for manipulating WGS or WES data. In analysis of three different types of family based sequencing data sets, SeqHBase demonstrated excellent performance with diverse functionality for rapid analysis of familial sequence data.

    Although our study mainly focused on family based data sets for candidate disease gene finding using inheritance information, the design of SeqHBase also allows handling of population-based studies with a large number of samples (eg, thousands). Big tables within HBase are capable of handling large-scale information with billions of rows and millions of columns and can be applied to manipulate millions of WGS data sets. Therefore, the design of SeqHBase can be applied to large number of family and/or population-based sequencing studies that are ongoing, such as Epi4K (http://www.epi4k.org/), UK10K (http://www.uk10k.org/) and the 100K Genomes Project (http://www.genomicsengland.co.uk/). The users simply need to ensure that their Hadoop-based framework has an adequate quantity of data nodes for sufficient data storage in the Hadoop distributed file systems. The underlying design of Hadoop MapReduce ensures scalability, flexibility and reliability to handle large data sets as well as analytical applications to process large data sets. Statistical methods for analysing large sets of population-based sequencing data will be developed within SeqHBase in the near future.

    ANNOVAR is one of many excellent annotation programmes. Other commonly used annotation programmes include snpEff (http://snpeff.sourceforge.net/), SVA (http://www.svaproject.org/), VEP (http://www.ensembl.org/info/docs/tools/vep/index.html) and others. When developing SeqHBase, we applied ANNOVAR to annotate variants in sequencing data sets. However, annotated information generated by other annotation programmes can also be applied in SeqHBase. Due to the use of open-source framework, SeqHBase is flexible enough to be deployed locally or in the cloud with ease. For example, we have successfully deployed it on Amazon's Cloud (via the Elastic MapReduce service provided by Amazon). For clinical use, sequence data may not be transferred externally to protect patient confidentiality; users can instead build in-house Hadoop clusters in individual labs or at the institutional level and deploy SeqHBase on them.

    SeqHBase stores coverage data for all sequenced positions by default, not just for positions with variant calls associated with them. We have chosen this as a design principle, which allows for users to interrogate different sets of variant calls generated for the same sample. Variant calls made on the same sequencing sample are often different based on the variant calling algorithm used, differences in parameter settings used with the same algorithm, use of family aware calling algorithms, or differences in the ensemble of samples used with multisample calling algorithms.17 Therefore, storing coverage data for all sequenced positions allows the users to switch to a different set of variant calls with relative ease. However, we also acknowledge that to save storage space, some users may include only a subset of sites, which can be achieved with user-specified coverage and allelic fraction thresholds.

    In summary, SeqHBase is a reliable big data-based computational toolset for efficiently manipulating genome-wide variants, annotations and every-site coverage in NGS studies. It uses a heuristic framework of inheritance information for detecting de novo, inherited homozygous or compound heterozygous mutations that may be disease-contributing in trios, nuclear families and/or extended families. It shows very good performance on three different examples of family based sequencing data and is scalable by virtue of its basis on MapReduce framework. SeqHBase is freely available for use by academic or non-profit organisations at http://seqhbase.omicspace.org/. Source code of SeqHBase can be downloaded after obtaining a license agreement with Marshfield Clinic Applied Sciences.

    Acknowledgments

    The authors thank Dr Rachel Stankowski and Ms Marie Fleisner in the Office of Scientific Writing and Publication at Marshfield Clinic Research Foundation for assistance with editing and preparation of this manuscript. The authors also thank Mr Ryan Frahm in the Biomedical Informatics Research Center at Marshfield Clinic Research Foundation for building 20 data nodes for this study.

    References

    1. ↵
      1. Sanders SJ,
      2. Murtha MT,
      3. Gupta AR,
      4. Murdoch JD,
      5. Raubeson MJ,
      6. Willsey AJ,
      7. Ercan-Sencicek AG,
      8. DiLullo NM,
      9. Parikshak NN,
      10. Stein JL,
      11. Walker MF,
      12. Ober GT,
      13. Teran NA,
      14. Song Y,
      15. El-Fishawy P,
      16. Murtha RC,
      17. Choi M,
      18. Overton JD,
      19. Bjornson RD,
      20. Carriero NJ,
      21. Meyer KA,
      22. Bilquvar K,
      23. Mane SM,
      24. Sestan N,
      25. Lifton RP,
      26. Günel M,
      27. Roeder K,
      28. Geschwind DH,
      29. Devlin B,
      30. State MW
      . De novo mutations revealed by whole-exome sequencing are strongly associated with autism. Nature 2012;485:237–41. doi:10.1038/nature10945
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. O'Roak BJ,
      2. Vives L,
      3. Girirajan S,
      4. Karakoc E,
      5. Krumm N,
      6. Coe BP,
      7. Levy R,
      8. Ko A,
      9. Lee C,
      10. Smith JD,
      11. Tuner EH,
      12. Stanaway IB,
      13. Vernot B,
      14. Malig M,
      15. Baker C,
      16. Reilly B,
      17. Akey JM,
      18. Borenstein E,
      19. Rieder MJ,
      20. Nickerson DA,
      21. Bernier R,
      22. Shendure J,
      23. Eichler EE
      . Sporadic autism exomes reveal a highly interconnected protein network of de novo mutations. Nature 2012;485:246–50. doi:10.1038/nature10989
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Neale BM,
      2. Kou Y,
      3. Liu L,
      4. Ma'ayan A,
      5. Samocha KE,
      6. Sabo A,
      7. Lin CF,
      8. Stevens C,
      9. Wang LS,
      10. Makarov V,
      11. Polak P,
      12. Yoon S,
      13. Maguire J,
      14. Crawford EL,
      15. Campbell NG,
      16. Geller ET,
      17. Valladares O,
      18. Schafer C,
      19. Liu H,
      20. Zhao T,
      21. Cai G,
      22. Lihm J,
      23. Dannenfelser R,
      24. Habado O,
      25. Peralta Z,
      26. Nagaswamy U,
      27. Muzny D,
      28. Reid JG,
      29. Newsham I,
      30. Wu Y,
      31. Lewis L,
      32. Han Y,
      33. Voight BF,
      34. Lim E,
      35. Rossin E,
      36. Kirby A,
      37. Glannick J,
      38. Fromer M,
      39. Shakir K,
      40. Fennell T,
      41. Garimella K,
      42. Banks E,
      43. Poplin R,
      44. Gabriel S,
      45. DePristo M,
      46. Wimbish JR,
      47. Boone BE,
      48. Levy SE,
      49. Betancur C,
      50. Sunyaev S,
      51. Boerwinkle E,
      52. Buxbaum JD,
      53. Cook EH Jr.,
      54. Devlin B,
      55. Gibbs RA,
      56. Roeder K,
      57. Schellnberg GD,
      58. Sutcliffe JS,
      59. Daly MJ
      . Patterns and rates of exonic de novo mutations in autism spectrum disorders. Nature 2012;485:242–5. doi:10.1038/nature11011
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      Epi4K Consortium; Epilepsy Phenome/Genome Project; Allen AS, Berkovic SF, Cossette P, Delanty N, Dlugos D, Eichler EE, Epstein MP, Glauser T, Goldstein DB, Han Y, Heinzen EL, Hitomi Y, Howell KB, Johnson MR, Kuzniecky R, Lowenstein DH, Lu YF, Madou MR, Marson AG, Mefford HC, Esmaeeli Nieh S, O'Brien TJ, Ottman R, Petrovski A, Ruzzo EK, Scheffer IE, Sherr EH, Yuskaitis CJ, Abou-Khalil B, Alldredge BK, Bautista JF, Berkovic SF, Boro A, Cascino GD, Consalvo D, Crumrine P, Devinsky O, Dlugos D, Epstein MP, Fiol M, Fountain NB, French J, Friedman D, Geller EB, Glauser T, Glynn S, Haut SR, Hayward J, Helmers SL, Joshi S, Kanner A, Kirsch HE, Knowlton RC, Kossoff EH, Kuperman R, Kuzniecky R, Lowenstein DH, McGuire SM, Motika PV, Novotny EJ, Ottman R, Paolicchi JM, Park K, Poduri A, Scheffer IE, Shellhaas RA, Sherr EH, Shih JJ, Sing R, Sirven J, Smith MC, Sullivan J, Lin Thio L, Venkat A, Vining EP, Von Allmen GK, Weisenberg GL, Widdess-Walsh P, Winawer WR. De novo mutations in epileptic encephalopathies. Nature 2013;501:217–21. doi:10.1038/nature12439
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Iossifov I,
      2. O'Roak BJ,
      3. Sanders SJ,
      4. Ronemus M,
      5. Krumm N,
      6. Levy D,
      7. Stessman HA,
      8. Witherspoon KT,
      9. Vives L,
      10. Patterson KE,
      11. Smith JD,
      12. Paeper B,
      13. Nickerson DA,
      14. Dea J,
      15. Dong S,
      16. Gonzalez LE,
      17. Mandell JD,
      18. Mane SM,
      19. Murtha MT,
      20. Sullivan CA,
      21. Walker MF,
      22. Waqar Z,
      23. Wei L,
      24. Willsey AJ,
      25. Yamrom B,
      26. Lee Y,
      27. Grabowska E,
      28. Dalkic E,
      29. Wang Z,
      30. Marks S,
      31. Andrews P,
      32. Leotta A,
      33. Kendall J,
      34. Hakker I,
      35. Rosenbaum J,
      36. Ma B,
      37. Rodgers L,
      38. Troge J,
      39. Narzisi G,
      40. Yoon S,
      41. Schatz MC,
      42. Ye K,
      43. McCombie WR,
      44. Shendure J,
      45. Eichler EE,
      46. State MW,
      47. Wigler M
      . The contribution of de novo coding mutations to autism spectrum disorder. Nature 2014;515:216–21. doi:10.1038/nature13908
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Chang F,
      2. Dean J,
      3. Ghemawat S,
      4. Hsieh WC,
      5. Wallach DA,
      6. Burrows M,
      7. Chandra T,
      8. Fikes A,
      9. Gruber RE
      . Bigtable: a distributed storage system for structured data. Proceedings of OSDI 2006:205–18. http://static.usenix.org/events/osdi06/tech/chang/chang_html/?em_x=22
    7. ↵
      1. Ghemawat S,
      2. Gobioff H,
      3. Leung ST
      . The Google file system. 19th Symposium on Operating Systems Principles: SOSP'03 2003. http://static.googleusercontent.com/media/research.google.com/en/us/archive/gfs-sosp2003.pdf
    8. ↵
      1. Robinson T,
      2. Killcoyne S,
      3. Bressler R,
      4. Boyle J
      . SAMQA: error classification and validation of high-throughput sequenced read data. BMC Genomics 2011;12:419. doi:10.1186/1471-2164-12-419
      OpenUrlCrossRefPubMed
    9. ↵
      1. Schatz MC
      . CloudBurst: highly sensitive read mapping with MapReduce. Bioinformatics 2009;25:1363–9. doi:10.1093/bioinformatics/btp236
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Langmead B,
      2. Schatz MC,
      3. Lin J,
      4. Pop M,
      5. Salzberg SL
      . Searching for SNPs with cloud computing. Genome Biol 2009;10:R134. doi:10.1186/gb-2009-10-11-r134
      OpenUrlCrossRefPubMed
    11. ↵
      1. Pireddu L,
      2. Leo S,
      3. Zanetti G
      . SEAL: a distributed short read mapping and duplicate removal tool. Bioinformatics 2011;27:2159–60. doi:10.1093/bioinformatics/btr325
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. O'Connor BD,
      2. Merriman B,
      3. Nelson SF
      . SeqWare Query Engine: storing and searching sequence data in the cloud. BMC Bioinformatics 2010;11(suppl 12):S2. doi:10.1186/1471-2105-11-S12-S2
      OpenUrl
    13. ↵
      1. Schonherr S,
      2. Forer L,
      3. Weissensteiner H,
      4. Kronenberg F,
      5. Specht G,
      6. Kloss-Brandstatter A
      . Cloudgene: a graphical execution platform for MapReduce programs on private and public clouds. BMC Bioinformatics 2012;13:200. doi:10.1186/1471-2105-13-200
      OpenUrlCrossRefPubMed
    14. ↵
      1. Niemenmaa M,
      2. Kallio A,
      3. Schumacher A,
      4. Klemela P,
      5. Korpelainen E,
      6. Heljanko K
      . Hadoop-BAM: directly manipulating next generation sequencing data in the cloud. Bioinformatics 2012;28:876–7. doi:10.1093/bioinformatics/bts054
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Schumacher A,
      2. Pireddu L,
      3. Niemenmaa M,
      4. Kallio A,
      5. Korpelainen E,
      6. Zanetti G,
      7. Heljanko K
      . SeqPig: simple and scalable scripting for large sequencing data sets in Hadoop. Bioinformatics 2014;30:119–20. doi:10.1093/bioinformatics/btt601
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Li H,
      2. Handsaker B,
      3. Wysoker A,
      4. Fennell T,
      5. Ruan J,
      6. Homer N,
      7. Marth G,
      8. Abecasis G,
      9. Durbin R
      . The Sequence Alignment/Map format and SAMtools. Bioinformatics 2009;25:2078–9. doi:10.1093/bioinformatics/btp352
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Fang H,
      2. Wu Y,
      3. Narzisi G,
      4. O'Rawe JA,
      5. Jimenez Barron LT,
      6. Rosenbaum J,
      7. Ronemus M,
      8. Iossifov I,
      9. Schatz MC,
      10. Lyon GJ
      . Reducing INDEL calling errors in whole-genome and exome sequencing. Genome Medicine 2014;6:89. doi:10.1186/s13073-014-0089-z
      OpenUrlCrossRefPubMed
    18. ↵
      1. Wang K,
      2. Li M,
      3. Hakonarson H
      . ANNOVAR: functional annotation of genetic variants from high-throughput sequencing data. Nucleic Acids Res 2010;38:e164. doi:10.1093/nar/gkq603
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Abecasis GR,
      2. Auton A,
      3. Brooks LD,
      4. DePristo MA,
      5. Durbin RM,
      6. Handsaker RE,
      7. Kang HM,
      8. Marth GT,
      9. McVean GA
      . An integrated map of genetic variation from 1,092 human genomes. Nature 2012;491:56–65. doi:10.1038/nature11632
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Landrum MJ,
      2. Lee JM,
      3. Riley GR,
      4. Jang W,
      5. Rubinstein WS,
      6. Church DM,
      7. Maglott DR
      . ClinVar: public archive of relationships among sequence variation and human phenotype. Nucleic Acids Res 2014;42:D980–5. doi:10.1093/nar/gkt1113
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Kircher M,
      2. Witten DM,
      3. Jain P,
      4. O'Roak BJ,
      5. Cooper GM,
      6. Shendure J
      . A general framework for estimating the relative pathogenicity of human genetic variants. Nat Genet 2014;46:310–15. doi:10.1038/ng.2892
      OpenUrlCrossRefPubMed
    22. ↵
      1. Adzhubei IA,
      2. Schmidt S,
      3. Peshkin L,
      4. Ramensky VE,
      5. Gerasimova A,
      6. Bork P,
      7. Kondrashov AS,
      8. Sunyaev SR
      . A method and server for predicting damaging missense mutations. Nat Methods 2010;7:248–9. doi:10.1038/nmeth0410-248
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Kumar P,
      2. Henikoff S,
      3. Ng PC
      . Predicting the effects of coding non-synonymous variants on protein function using the SIFT algorithm. Nat Protoc 2009;4:1073–81. doi:10.1038/nprot.2009.86
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. McPherson E,
      2. Zaleski C,
      3. Ye Z,
      4. Lin S
      . Rodriguez syndrome with SF3B4 mutation: a severe form of Nager syndrome? Am J Med Genet A 2014;164:1841–5. doi:10.1002/ajmg.a.36555
      OpenUrlCrossRef
    25. ↵
      1. Lyon GJ,
      2. Jiang T,
      3. Van Wijk R,
      4. Wang W,
      5. Bodily PM,
      6. Xing J,
      7. Tian L,
      8. Robison RJ,
      9. Clement M,
      10. Lin Y,
      11. Zhang P
      12. Liu Y,
      13. Moore B,
      14. Glessner JT,
      15. Elia J,
      16. Reimherr F,
      17. van Solinge WW,
      18. Yandell M,
      19. Hakonarson NH,
      20. Wang J,
      21. Johnson WE,
      22. Wei Z,
      23. Wang K
      . Exome sequencing and unrelated findings in the context of complex disease research: ethical and clinical implications. Discov Med 2011;12:41–55.
      OpenUrlPubMed
    26. ↵
      1. Rodríguez JI,
      2. Palacios J,
      3. Urioste M
      . New acrofacial dysostosis syndrome in 3 sibs. Am J Med Genet 1990;35:484–9. doi:10.1002/ajmg.1320350408
      OpenUrlCrossRefPubMed
    27. ↵
      1. Miller M,
      2. Fineman R,
      3. Smith DW
      . Postaxial acrofacial dysostosis syndrome. J Pediatr 1979;95:970–5. doi:10.1016/S0022-3476(79)80285-1
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Nager FR
      . [Anomalies of the labyrinth in the light of modern genetic theory]. Pract Otorhinolaryngol (Basel) 1951;13:129–45.
      OpenUrlPubMed
    29. ↵
      1. Bernier FP,
      2. Caluseriu O,
      3. Ng S,
      4. Schwartzentruber J,
      5. Buckingham KJ,
      6. Innes AM,
      7. Jabs EW,
      8. Innis JW,
      9. Schuette JL,
      10. Gorski JL,
      11. Byers PH,
      12. Andelfinger G,
      13. Siu V,
      14. Lauzon J,
      15. Fernandez BA,
      16. McMillin M,
      17. Scott RH,
      18. Racher H; FORGE Canada Consortium, ,
      19. Majewski J,
      20. Nickerson DA,
      21. Shendure J,
      22. Bamshad MJ,
      23. Parboosingh JS
      . Haploinsufficiency of SF3B4, a component of the pre-mRNA spliceosomal complex, causes Nager syndrome. Am J Hum Genet 2012;90:925–33. doi:10.1016/j.ajhg.2012.04.004
      OpenUrlCrossRefPubMed
    30. ↵
      1. Climent F,
      2. Roset F,
      3. Repiso A,
      4. Perez de la Ossa P
      . Red cell glycolytic enzyme disorders caused by mutations: an update. Cardiovasc Hematol Disord Drug Targets 2009;9:95–106. doi:10.2174/187152909788488636
      OpenUrlCrossRefPubMed
    31. ↵
      1. Diez A,
      2. Gilsanz F,
      3. Martinez J,
      4. Perez-Benavente S,
      5. Meza NW,
      6. Bautista JM
      . Life-threatening nonspherocytic hemolytic anemia in a patient with a null mutation in the PKLR gene and no compensatory PKM gene expression. Blood 2005;106:1851–6. doi:10.1182/blood-2005-02-0555
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Makino S,
      2. Kaji R,
      3. Ando S,
      4. Tomizawa M,
      5. Yasuno K,
      6. Goto S,
      7. Matsumoto S,
      8. Tabuena MD,
      9. Maranon E,
      10. Dantes M,
      11. Lee LV,
      12. Ogasawara K,
      13. Tooyama I,
      14. Akatsu H,
      15. Nishimura M,
      16. Tamiya G
      . Reduced neuron-specific expression of the TAF1 gene is associated with X-linked dystonia-parkinsonism. Am J Hum Genet 2007;80:393–406. doi:10.1086/512129
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Kaya N,
      2. Colak D,
      3. Albakheet A,
      4. Al-Owain M,
      5. Abu-Dheim N,
      6. Al-Younes B,
      7. Al-Zahrani J,
      8. Mukaddes NM,
      9. Dervent A,
      10. Al-Dosari N,
      11. Al-Odaib A,
      12. Kayaalp IV,
      13. Al-Sayed M,
      14. Al-Hassnan Z,
      15. Nester MJ,
      16. Al-Dosari M,
      17. Al-Dhalaan H,
      18. Chedrawi A,
      19. Gunoz H,
      20. Karakas B,
      21. Sakati N,
      22. Alkuraya FS,
      23. Gascon GG,
      24. Ozand PT
      . A novel X-linked disorder with developmental delay and autistic features. Ann Neurol 2012;71:498–508. doi:10.1002/ana.22673
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors Conceived and designed the experiments: MH and KW. Conceived and conducted the first study: EWM, SML and ZY. Conceived and conducted the second study: GJL, RJR and KW. Conceived and conducted the third study: JO, GJL and KW. Performed the experiments: MH, TNP, EH, ZY and JO. Analysed the data: MH, TNP, EH, ZY and JO. Wrote the paper: MH, SJH, EH, SJS, PLP, MHB, JO, GJL and KW.

    • Funding This work was supported by the National Institutes of Health (UL1TR000427 to MHB and MH, HG006465 to KW); the Marshfield Clinic Research Foundation to MH; and the Stanley Institute for Cognitive Genomics at Cold Spring Harbor Laboratory to GJL.

    • Competing interests RJR and KW are share holders and board members of Tute Genomics, Inc. G.J.L serves on the advisory board of Omicia, Inc. and GenePeeks, Inc.

    • Ethics approval Marshfield Clinic Research Foundation; University of Utah; Utah Foundation for Biomedical Research.

    • Provenance and peer review Not commissioned; externally peer reviewed.