Data cleaning

Correction exercises

Recover ESTs chromatograms

• Go to the NCBI Traces server (http://www.ncbi.nlm.nih.gov/Traces).
• Select the Blast option.
• Paste your sequence.
• The sequence is a human sequence, therefore choose the human trace database (hstrace) for the blast search.
• Run the blast search and look at the results.
• Choose a few sequences and retrive the traces to look at them. Normally bed quality regions are found at the start and at the end of the sequences.

Translate chromatograms

• The quality of the sequences can be found in the '.qual' files. Values are normally low at the borders of the sequence, while quality increase in the middle region (exept in presence of low complexity regions).
• The meaning of the quality values:
• Quality values are calculated with the formula Quality = -10 * log_10(P_error).
• Ex:
• Q = 10 <=> 10 = -10 * log(P_err) <=> P_err = 0.1 (10% of chances that the base is an error).
• Q = 40 <=> 40 = -10 * log(P_err) <=> P_err = 0.0001 (0.01% of chances that the base is an error).
• The '-trim' option of phred, set to 0 low quality values. To test this option create a directory 'phred_out_trim' and run phred -id traces -sd phred_out_trim -qd phred_out_trim -trim ''.

Filter vector sequences

• The results of cross_match are in the file 'set.fasta.screen'.
• The majority of the sequences are masked in their 5', 3', or both regions by 'X'.

Mask repeats

• The output file from the RepeatMask 'set2.fasta.masked' contains regions masked with 'N'.
• In the report file 'set2.fasta.tbl' you can find information about the type of repeats found:
• 48 simple repeats
• 42 low complexity regions

Assembly

If everything went right you must see 3 contigs:

>dataset.fasta.Contig1
GGCCGCAATGCNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
NNNNNNNNNNNGGGGATTTATATGACTTTTATTTTAATATCATTTTAATA
TCGAATATGTCACAAGATTTAATGACCAAATTAGTCATTTACATTTTTCA
AAGTATCTAAATATAAAATTGCTAAAAATTCAAATACAGATAGCAAGCTA
CAAATATTTCTTTTGTTTTTGGGGGGGGGGTAGGAATGGAAGACATTAAA
AAAGGACGCTCCTGTTGGCTGAACAAAGATCAGAATGAAAGAAGAATTCA
TCAGCTTGACCCCTTGTGCAAAGAAATACACTCAGCTTTAAGATGCTGTT
TTAGACACATCTCTTCCTGTACAACAATTTAAAAAATGTTTCTAATGCAG
GTCCTCAGTGAAACACCCCTCTCCCTGAAACGTGAGAAACAGCAGCTTTC
CGCCTGCTTGAAAAAGGAGGCAGGCGCTGTAAGGAGAAAAGGAATGGGGA
ATGTGGCCCCTTCCAAGGCAGCGAAGCAGAGAACATGCAGGTGGAACTGC
CAGCTGAAACCTACACTTGCTACTTAAGAAACGGAAAGACCATGTCTGGC
TGGGAGTGCAGCGTGCCTGAGGACCAGCCACCGCCCCTCCTCCTGGCACC
CCACACTGTNCACGCAGCTCCTGGGCTCTGAGTCAGAAAGCTGGTGTGCT
GAAGGGGAAGTGACACGGCCTTGACACACCACTAGGTCTTTGTCTAACTT
TAGTAAGACCTACCCTCGAGCAAATGACTCATTTAACAAATGGCATGNTA
AACCTATTGCTAGAC
>dataset.fasta.Contig2
GGCCGCAATGCNNNNNNNNNNNNNNNNNNNNNNNNNNGTATTTATATGAC
TTTTATTTTAATATCATTTTAATATCGAATATGTCACAAGATTTAATGAC
CAAATTAGTCATTTACATTTTTCAAAGTATCTAAATATAAAATTGCTAAA
AATTCAAATACAGATAGCAAGCTACAAATATTTCTTTTGTTTTTGGGTGG
GGGGTAGGAATGGAAGACATTANAAAAGGACGCTCCTGTTGGCTGAACAA
AGATCAGAATGAAAGAAGAATTCATCAGCATGACTTGTAATGGTGGCTGC
TAAGCATATCCTGTACAACAATTTAAAAAATGTTTCTAATGCAGGTCCTC
AGTGAAACACCGGTCCCCGGCCCTGGCTGGGGACAGTAAGGACATCACCG
CAGGAGGGACACTGAAGAGGCTGTCGAGGACTGCAGAGGCATCTGGTGTG
GCCAGAGGCGTGGTGTCAGGGGCATCTGATCCCTTGCTGTTCCACCCAGG
GAGCCGGACGCACGGACACAGGTCTCCCTCCGCTCACCCCTCTCCCTGAA
ACGTGAGAAACAGCAGCTTTTCCCCCTCGTGCCGAATTCTTT
>dataset.fasta.Contig3
GGCACGAGATCATTTTAATATCGAATATGTCACAAGATTTAATGACCAAA
TTAGTCATTTACATTTTTCAAAGTATCTAAATATAAAATTGCTAAAAATT
CAAATACAGATAGCAAGCTACAAATATTTCTTTTGTTTTTGGGTGGGGGG
TAGGAATGGAAGACATTAGAAAAGGACGCTCCTGTTGGCTGAACAAAGAT
CAGAATGAAAGAAGAATTCATCAGCATGACCCCTTGTGCAAAGAAATACA
CTCAGCTTTAAGATGCTGTTTTAGACACATCTCTTCCTGTACAACAATTT
AAAAAATGTTTCTAATGCAGGTCCTCAGTGAAACACCGGTCCCCGGCCCT
GGCTGGGGACAGTAAGGACATCACCGCAGGAGGGACACTGAAGAGGCTGT
CGAGGACTGCAGAGGCATCTGGTGTGGCCAGAGGCGTGGTGTCAGGGGCA
TCTGATCCCTTGCCTGTTCCCACCCAGGGAGCCGGACGGCACGGACACAG
GTCTCCCTCCGCTCACCCCTCTCCCTGAAACGTGAGAAACAGCAGCTTTC
CGCCTGCTTGAAAAAGGAGGCAGGCGCTGTAAGGAGAAGAGGAATGGGGA
ATGTGGGCCCTTCCAAGGCAGCGAAGCAGAGAACATGCAGGTGGAACTGC
CAGCTGAAACCTACACTTGCTACTTAAGAAACGGAAAGACCATGTCTGGC
TGGGAGTGCAGCGTGCCTGAGGACCAGCCACCGCCCCTCCTCCTGGCACC
CCACACTGTCCACGCAGCTCCTGGGCTCTGAGGTCAGAAGCTGGGGTGTG
CTGAAGGGGAAAGTGACACGGCCTTGGACACACCACTAGGTCTCTGTTCC
TAACTTTTAGTAAGACACTACCCTCGAAGCAAATGCACTCATTTAAACAA
ATGGCATAGTTAAAACTATTGACTAAGACCTAAACATTTCTTCTGAGAAA
TCGAACCATAGCTTTTCCAATCTGCCTGTTCAATATGGGAACAGATTTTA
AAGAGAGTAACATAATCAATCCTCCGCCCCAAAGAATGGTCCATCATACC
AACCATGATAAAGAGTACACCTGTTTATTAAAAGGAAAAACAGAAATGTA
CATTTTTGTTGTTTGCTTTAAGAAATTTGATCAAGTTGCAAGGAAATGTG
TGGGCACGGCTCTGTACATCCTCGGGCAGGGTGGCAGGCATTGAAGGTGG
CTGGGCGGCACGTGGCTCCTCTAGGGGGTGGTCTGACCCCCAAGCATCGC
TTATCAAAGCCACTGCCAAGCAGACTTCCGTCCCATGGCAATGTCCCCAG
CGCTCCCTCCTAGGGGGCCCCCGACACCTTCCCCGAGAGCCCACCTGCCC
TGTGCCAGTGAGCCAGGGGCTGGCCTCCCCCCGAGGACTTCAGGACACCG
GGTGGGCTCTAGGGCACTTGGCCCTGGCAGGCAGGCTCTGAGCTACGGAA
CAGATTGAGGCTGGAGGCCAGTGTCACTGCTGCCCGGATTTCTTCTCCTT
CTGGAAGCTGAAGCTCGTACGTCTGTTCAGTTTTCTTTGTTTTTGAGCAA
AGTAGTCAGCCCGGGCAGTGCTAGCTCGCGATTCCAGGATGTAGTTAACC
TTGAGCACAATTTCATTGACGGTAGCAGCGTCCGATTCAAAGTAGAGGTG
TTTATAGTCGTGATTGCTTAGATACGTGAGTTTAAATATTGCGTGACTGG
GGCTTTTCTCTTCAGCAAGGTCACAGGCACAGAGCAGGTCGGAATCGATT
GAGATGGGTTTCTGCTTAATCCAAAACTTAGTGCTGGCTTTCTGATTCGT
AACAGGGTCTATCTCTACTTTGTCTCCAGAGATACCTAGCTGTACGTCGG
TTGTGAATCGCAGTCTGTGGATCATGCTGACTTTGAATGACTTGTAATGG
TGGCTGCTAACAGTATCCTGTACTGTGGCTATGTCAATTTGCGAATCCTC
CTCAAAAACCCCGTCTGCCCTTGAACTGTTCTCGCGGACCAGGCAGAACT
CGCATGCGCTCTGGCTCTCCAAAGTGCTGTCCAGGTCAACGGCGACATTG
GGCTCGCTCTGCTTCTCCAGGCGGTACTGAGGGCTCGGTTCTCTCG

... time for a beer!