all context methylation visualization

[PaulaRomeroLozano]

Hello,

I am using megalodon to identify methylations in the whole genome, not only in the CpG islands.
I was wondering if it is possible to use Methplotlib to visualise also those regions outside the CpG context with the megalodon outputs.
Do you think it is compatible?

Thank you,
Paula

[wdecoster]

Hi Paula,

Yes, that shouldn't be a problem. methplotlib doesn't really care about the type of modification it shows. Which file format did you get from megalodon?
It is possible that I need to add an additional file format as input type, but that shouldn't be too much trouble.

Cheers,
Wouter

[PaulaRomeroLozano]

Hi Wouter,

Thank you very much for the quick response.

That would be incredibly useful!

I am still getting familiar with all the megalodon outputs and which ones are required and "compatible" for downstream analysis.

Please find below the file formats I got when running megalodon:

1. mod_mapping.bam: I was thinking in sort it and convert it to a bedgraph using bedtools?
   
   2.modified_bases.5mc.bed: (0-based)
   
   3.per_read_modified_bases: as a .txt and .bd file (natural log)
   

I was thinking in trying to make those files "nanopolish like" so I could use your scripts, but if you could add an extra input type that would be very helpful.

Thank you so much in advance,
Best,

Paula

[wdecoster]

Please keep me posted how things go for any of these formats. It may not be necessary to make them too similar to nanopolish files.

1. methplotlib can take cram files with nucleotide modifications in the Mm tag, so your bam file should not need too many changes. Converting to cram should be enough. That said, I could also add an option to use bam directly.

2. That looks like a bedmethyl format. You can quite easily make a bedgraph out of that, which is currently one of the compatible input formats. I could in the future also add bedmethyl input directly.

3. Could you share a larger example of that format? I'll add it as a compatible input too. You can find my email address in my profile.

I'm afraid I won't have time to add the code for these formats this week.

[PaulaRomeroLozano]

Thank you very much for your help and feedback.

2. I will go with this option first then.

3. I will send you a small subset of the data by email.

Thank you again and I will keep you posted for sure :)

[JWDebler]

I found an R script in the METEORE repo that takes the megalodon .bed file and computes methylation frequencies like the nanopolish calculate_methylation_frequency.py script does. In its original form the output doesn't work with methplotlib yet, but it shouldn't take much to get the output close enough to nanopolish format. I don't know enough R though to do it.

[marcus1487]

I can help on some of the megalodon options here. Megalodon has a lot of outputs in order to facilitate porting into tools such as methplotlib. Megalodon is designed with two main modified base output types 1) per-read and 2) aggregated.

Per-read outputs include the mod_mappings.bam above. Megalodon can also output cram directly (via --mappings-format cram) and sort the mappings (via --sort-mappings). The mod_mappings output takes the per-read modified base calls and anchors them to the reference sequence (so these reads will artificially contain exactly the reference sequence for each read). mod_basecalls are anchored only to the basecalls and not mapped, so not directly applicable to methplotlib (I think).

The per-read modified base text format is meant as a dump of the per-read scores database mostly for debugging purposes. I would not recommend this as a standard format that should used as input and is subject to change without notice.

Aggregated formats returns results aggregated across reads at each applicable reference position. For the aggregated formats, Megalodon provides 3 options 1) bedmethyl 2) modvcf and 3) wiggle (mostly like a bedgraph but more compact). The modVCF is not really a standard format, but there is no standard format (that I'm aware of) to store multiple modifications in a single file. VCF was the closest, but required adding some fields for strandedness.

@JWDebler that METEORE script does not compute the methylation frequencies, but instead just converts from the format from bedmethyl to a consistent text format for comparison to other algorithms within the METEORE framework. I believe this could essentially be accomplished with a simple awk script to move columns around.

I'm happy to help mediate the porting of megalodon outputs into methplotlib. Let me know if any adjustments or recommendations would be helpful.

[PaulaRomeroLozano]

Hello Marcus,

Thank you so much for your help.

It would be super useful to be able to port megalodon outputs into methplotlib! :)
Since I got mod_mapping.bam as output, could it be possible to simply convert that bam to cram?
I tried:
samtools sort mod_mappings.bam -o mod_mappings.sorted.bam
samtools view -C -T genome.fa mod_mappings.bam > mod_mappings.cram
samtools index -c mod_mappings.sorted.cram

I did so, and then I tried to run methplotlib but I got an error:

methplotlib -m /mnt/fs01/RUNS/minion_run/DNA_meth/megalodon_results/subset_1_5G_norm/mod_mappings.cram -n calls -w chr17:17,722,291-17,725,186 -g /media/data/reference/Homo_sapiens.GRCh38.90.chrplus.gtf --simplify
[W::find_file_url] Failed to open reference "https://www.ebi.ac.uk/ena/cram/md5/a8499ca51d6fb77332c2d242923994eb": Protocol not supported
Traceback (most recent call last):
File "/home/paula/.local/bin/methplotlib", line 8, in
sys.exit(main())
File "/home/paula/.local/lib/python3.8/site-packages/methplotlib/methplotlib.py", line 26, in main
meth_browser(meth_data=meth_data,
File "/home/paula/.local/lib/python3.8/site-packages/methplotlib/methplotlib.py", line 52, in meth_browser
meth_traces = plots.methylation(meth_data, dotsize=dotsize, binary=binary)
File "/home/paula/.local/lib/python3.8/site-packages/methplotlib/plots.py", line 120, in methylation
make_per_read_meth_traces_phred(table=meth.table,
File "/home/paula/.local/lib/python3.8/site-packages/methplotlib/plots.py", line 148, in make_per_read_meth_traces_phred
table = table.join(df_heights, on="read_name")
File "/home/paula/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 7874, in join
return self._join_compat(
File "/home/paula/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 7890, in _join_compat
return merge(
File "/home/paula/.local/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 74, in merge
op = _MergeOperation(
File "/home/paula/.local/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 656, in init
self._maybe_coerce_merge_keys()
File "/home/paula/.local/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 1165, in _maybe_coerce_merge_keys
raise ValueError(msg)
ValueError: You are trying to merge on object and float64 columns. If you wish to proceed you should use pd.concat

The methplotlib logs reported:
2021-04-08 14:51:00,334 methplotlib 0.17.0 started.
2021-04-08 14:51:00,334 Python version is: 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]
2021-04-08 14:51:00,334 Arguments are: Namespace(bed=None, binary=False, dotsize=4, example=False, fasta=None, gtf='/media/data/reference/Homo_sapiens.GRCh38.90.chrplus.gtf', methylation=['/mnt/fs01/RUNS/minion_run/DNA_meth/megalodon_results/subset_1_5G_norm/mod_mappings.cram'], names=['calls'], outfile=None, qcfile=None, simplify=True, smooth=5, split=False, static=None, store=False, window='chr17:17,722,291-17,725,186')
2021-04-08 14:51:00,334 Processing chr17_17722291_17725186
2021-04-08 14:51:00,335 File /mnt/fs01/RUNS/minion_run/DNA_meth/megalodon_results/subset_1_5G_norm/mod_mappings.cram is of type ont-cram
2021-04-08 14:51:00,557 Collected methylation data for 1 datasets
2021-04-08 14:51:00,658 Created QC plots

I am not sure why is not working and how I can fix it...

Another option would be to convert the bedmethyl file into bedgraph but I haven't been able to succeed on this either.

Any feedback or recommendation would be very much appreciated.

Kind regards,
Paula

[marcus1487]

I think the error is the same bug @wdecoster noted in #31. Hopefully a resolution can be found there.

The bedgraph input to methplotlib is slightly different than the mod_mappings. The mod_mappings contain per-read methylation calls, so can provide more information (phasing/linked methylation) while the bedgraph/bedmethyl is aggregated over reads (so links between calls between each read are lost). I'm not sure what outputs methplotlib has in terms of per-read modified base calls, so this may be a moot point here, but thought the distinction was important.

For converting from bedmethyl to bedgraph the following one-liner should work: awk 'BEGIN{OFS="\t"} {$4=$11; NF=4; print}' sample.bedmethyl > sample.bedgraph. You may need to add a header to load this file in some genome browsers.

[Fatihlrcfs]

hi all,

I am working on methylation outputs to visualize that. I have megalodon(files: modified_bases.5mc.bed, per_read_modified_base_calls.db and per_read_modified_base_calls.txt) and ı am wondering to use these file in Methplotlib. Could you say me the steps basically because ı am a freshman in this field? Many thanks. Kind Regards.

[wdecoster]

The megalodon input is in BAM/CRAM format, but you don't have those?

[Fatihlrcfs]

Hi @wdecoster
ı am using comman below to run megalodon and these (fmodified_bases.5mc.bed, per_read_modified_base_calls.db and per_read_modified_base_calls.txt) files were generated. thanks
code: megalodon debarcodedmultifast5/barcode06/ --outputs mods --reference wholegenome.fasta --mod-motif m CG 0 --write-mods-text --processes 40 --guppy-server-path /cluster/lrcfs/2397405/bin/ont-guppy-cpu-6.0.1/bin/guppy_basecall_server --guppy-params "-d /cluster/lrcfs/2397405/bin/rerio/basecall_models/ --num_callers 40" --guppy-timeout 400 --guppy-config res_dna_r941_min_modbases_5mC_CpG_v001.cfg --overwrite --output-directory megalodonresultbarcode06

[wdecoster]

I think --outputs with mod_mappings or mod_basecalls is what you need, but I don't use megalodon myself

[Fatihlrcfs]

okey, @wdecoster ı will configure my code and run with a small data set to generate the required files. thanks

[carolinehey]

Hi @wdecoster

Earlier you wrote this:

1. methplotlib can take cram files with nucleotide modifications in the Mm tag, so your bam file should not need too many changes. Converting to cram should be enough. That said, I could also add an option to use bam directly.
2. That looks like a bedmethyl format. You can quite easily make a bedgraph out of that, which is currently one of the compatible input formats. I could in the future also add bedmethyl input directly.
3. Could you share a larger example of that format? I'll add it as a compatible input too. You can find my email address in my profile.

As for 1. i have a cram file with modifications in the Mm tag and would really like to use methplotlib. However, i get an error regarding the -n argument.

code:
methplotlib -m merged_bam_sup_modbases_sed.cram -w chr15:23000000-25500000
usage: methplotlib [-h] [-v] -m METHYLATION [METHYLATION ...] -n NAMES [NAMES ...] -w WINDOW [-g GTF] [-b BED] [-f FASTA] [--simplify] [--split]
[--static STATIC] [--binary] [--smooth SMOOTH] [--dotsize DOTSIZE] [--minqual MINQUAL] [--example] [-o OUTFILE] [-q QCFILE]
methplotlib: error: the following arguments are required: -n/--names

How to deal with this?