(§Technical contact: kazuki-nakamae@hiroshima-u.ac.jp) 发布: 2026年04月20日第16卷第8期 DOI: 10.21769/BioProtoc.5676 浏览次数: 698
评审: Migla MiskinyteAnonymous reviewer(s)
Abstract
DNA language models, such as DNABERT-2, have recently enabled the accurate prediction of functional sequence elements across species. However, the practical, protocol-style steps needed to transform these resources into training datasets, fine-tune the official DNABERT-2 model, and evaluate classifier performance have not been explicitly described. Herein, we present a step-by-step computational protocol for preparing training data, fine-tuning DNABERT-2, and evaluating sequence-level binary classifiers using readily available command-line tools. The protocol has been demonstrated using RNA off-target sites induced by cytosine base editors, detected by our PiCTURE pipeline from RNA sequencing (RNA-seq) data, and extended to core promoter prediction using the EPDnew database. We describe how to derive positive and negative sequence sets into DNABERT-2 compatible datasets, and fine-tune the official pretrained model of DNABERT-2 using the datasets. We also demonstrate how to compute the standard performance metrics and compare the model outputs with the baselines. This protocol will help researchers adapt DNA foundation models to new genomic tasks, including the safety assessment of genome editing tools and the functional annotation of regulatory sequences.
Key features
• A step-by-step pipeline has been provided for preparing labeled DNA sequence datasets for DNABERT-2 fine-tuning.
• The reproducibility of the official DNABERT-2 implementation and scripts for binary classification is shown.
• Application of PiCTURE-derived C-to-U (T) substitution sites for off-target RNA risk prediction.
• An extension to core promoter prediction using promoter datasets from EPDnew is described.
Keywords: DNABERT-2 (DNABERT-2)Graphical overview
Graphical overview of the fine-tuning pipeline for predicting cytosine base editors (CBE)-specific RNA off-targets and eukaryote promoters using DNABERT-2. Images of the DNA and RNA-seq were obtained from TogoTV (© 2016 DBCLS TogoTV, CC-BY-4.0 https://creativecommons.org/licenses/by/4.0/).
Background
DNA language models trained on large-scale genomic corpora have emerged as powerful tools for the prediction of functional sequence elements [1–6]. DNABERT-2 is a foundation model pretrained on data comprising multi-species genomes using byte-pair encoding tokenization and an efficient transformer architecture [6]. It has demonstrated state-of-the-art performance in diverse genome-understanding tasks.
Cytosine base editors (CBEs) introduce C-to-U (T) substitutions without generating double-strand breaks [7–9]; however, some base editors can also cause unintended C-to-U (T) substitutions on RNA, called RNA off-target [10–12]. RNA off-target sequences are often enriched in canonical ACW motifs (W = A or T/U) [10]; however, non-canonical events are also observed and are more difficult to characterize [13,14]. To systematically profile such off-targets, we previously developed PiCTURE (Pipeline for CRISPR-induced Transcriptome-wide Unintended RNA Editing) (https://github.com/KazukiNakamae/PiCTURE) [15], which processes RNA-seq data from CBE-transfected samples and outputs high-confidence C-to-U (T) substitution sites and their local sequence contexts [13].
In our recent study, PiCTURE outputs were used to fine-tune DNABERT-2 into a binary classifier, the SNL (RNAoffScan v2) model (https://github.com/KazukiNakamae/RNAOffScan), which distinguishes CBE-induced RNA substitutions from background substitution sites and outperforms motif-only baselines in terms of accuracy, precision, recall, and F1 score [13]. Hence, DNA foundation models can capture both canonical and noncanonical sequence signals associated with off-target editing.
In addition to genome editing safety, DNABERT-2 can be adapted to a broad range of regulatory genomics tasks [6]. In this protocol, eukaryotic promoter databases, such as EPDnew [16], provide experimentally validated transcription start sites (TSSs) and flanking promoter sequences for multiple species. These resources enable the systematic construction of a positive dataset for promoter prediction and allow the evaluation of model performance [17,18].
However, the practical steps required to construct training datasets from PiCTURE outputs or the EPDnew database, fine-tune the official DNABERT-2 implementation, and evaluate the classifier performance have not been described in a protocol format. The following protocol provides a reproducible, end-to-end workflow for (i) generating labeled DNA sequence datasets from PiCTURE outputs and the EPDnew database, (ii) formatting them for DNABERT-2, (iii) fine-tuning DNABERT-2 using the official pretrained model, and (iv) evaluating the performance relative to motif-based or other machine learning-based baselines. By following this protocol, researchers can adapt DNABERT-2 to new sequence-level classification tasks, quantify model performance, and leverage DNA language models for genome-editing risk assessment and eukaryotic regulatory genomics.
Equipment
Note: This protocol is entirely computational and does not include wet-lab experiments.
1. MacBook Pro (14 in, 2024), Apple M4 Max 16 cores, LPDDR5 RAM 128 GB Hynix, SSD 2 TB, MacOS version 26.0 (Apple Inc.)
Note: Unless otherwise specified, the analysis in this protocol was performed using a MAC device. Most analysis commands are compatible with a Linux PC.
2. Linux Server, Intel Xeon w7-3445, 20 cores, RAM 1024 GB, SSD (system) 3.8 TB M.2 NVMe × 2, GPU RTX 4500 Ada (VRAM: 24 GB), Ubuntu version 22.04.5 LTS (NABE International, Inc.)
3. Sakura's Cloud Server, Intel(R) Xeon(R) Platinum 8480+, 24 cores, 240 GB RAM, 2 TB SSD, NVIDIA H100 (VRAM: 80 GB), Ubuntu version 22.04.5 LTS (SAKURA Internet Inc.)
Note: The NVIDIA H100 (80 GB VRAM) was primarily used for fine-tuning DNABERT-2. We also confirmed that the workflow can be executed on five NVIDIA Tesla V100 SXM2 GPUs (16 GB VRAM each) using distributed/sharded training. Note that this setup provides 80 GB of aggregate device memory rather than a single 80 GB memory pool. In our environment, the workflow typically completes within one day on an H100, whereas it requires approximately 3–7 days on the multi-V100 setup, depending on the distributed-training configuration and input length.
Software and datasets
All software and dataset information are provided in Table 1.
Table 1. Software tools and datasets used in this pipeline
| Type | Software/dataset/resource | Version (SHA256 hash, Git commit hash, Zenodo DOI, Manifest digest, MD5sum hash) | Date | License | Access (free or paid) |
| Software | DNABERT-2 model and fine-tuning scripts (MAGICS-LAB/DNABERT_2) [6] | Main branch (Git commit hash: d87d29f5570f6d8238de734c932697ef657be59a) | Accessed on 2025-12-09 | Apache-2.0 | Free via GitHub website (https://github.com/MAGICS-LAB/DNABERT_2) |
| Software | PyTorch [19] | Version 2.4.1+cu121-cp38-cp38-linux_x86_64 (SHA256 hash: cb4f502f910b47e1e366ccf7b231dac2967d2efb47d4b8cb33fc63b4bc5eeed8) | Accessed on 2025-12 | BSD 3-Clause License | Free via PyTorch website (https://pytorch.org/) |
| Software | Transformers [20] | Version 4.29.2 (Git commit hash: ba7054533fa455e8b2dd35feb077e0c7aae646b3) | Accessed on 2025-12 | Apache 2.0 License | Free via GitHub website (https://github.com/huggingface/transformers) |
| Software | An unofficial collection of helper scripts for DNABERT-2 (KazukiNakamae/DNABERT_2_helper) | Version 1.0.0 (DOI: 10.5281/zenodo.18954363) | Accessed on 2026-01-11 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/DNABERT_2_helper) |
| Software | PiCTURE pipeline for RNA off-target detection (KazukiNakamae/PiCTURE) [13] | Version 1.0.0 (DOI: 10.5281/zenodo.18954750) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/PiCTURE) |
| Software | A collection of scripts for data processing of plants’ promoters (KazukiNakamae/plant_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963418) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/plant_promoter_classifier) |
| Software | A collection of scripts for data processing of mammals’ promoters (KazukiNakamae/mammal_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963493) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/mammal_promoter_classifier) |
| Software | A collection of scripts for data processing of mammals’ ncRNA promoters (KazukiNakamae/mammal_ncRNA_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963677) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/mammal_ncRNA_promoter_classifier) |
| Software | A collection of scripts for data processing of birds’ promoters (KazukiNakamae/bird_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963776) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/bird_promoter_classifier) |
| Software | A collection of scripts for data processing of insects’ promoters (KazukiNakamae/insect_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963825) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/insect_promoter_classifier) |
| Software | A collection of scripts for data processing of fish’s promoters (KazukiNakamae/fish_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963878) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/fish_promoter_classifier) |
| Software | A collection of scripts for data processing of nematoda’s promoters (KazukiNakamae/nematoda_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18963993) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/nematoda_promoter_classifier) |
| Software | A collection of scripts for data processing of yeasts’ promoters (KazukiNakamae/yeast_promoter_classifier) | Version 1.0.0 (DOI: 10.5281/zenodo.18964048) | Accessed on 2025-12-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/yeast_promoter_classifier) |
| Software | A script for di-nucleotide shuffling (KazukiNakamae/shuffle_fasta_di) | Version 1.0.0 (DOI: 10.5281/zenodo.18964299) | Accessed on 2026-03-10 | MIT License | Free via GitHub website (https://github.com/KazukiNakamae/shuffle_fasta_di) |
| Software | Trim Galore (Docker container used in PiCTURE) [21] | Version 0.6.7 (SHA256 hash: ffc30fbdb4d28bd385465f661ed717168d10f77640371d237167d521f57b62d2) | Accessed on 2025-12-10 | GNU General Public License v3.0 | Free via Docker Hub website (https://hub.docker.com/r/clinicalgenomics/trim_galore) |
| Software | GATK (Docker container used in PiCTURE) [22,23] | Version 4.4.0.0 (SHA256 hash: e7996ba655225c1cde0a1faec6a113e217758310af2cf99b00d61dae8ec6e9f2) | Accessed on 2025-12-10 | BSD 3-Clause License | Free via Docker Hub website (https://hub.docker.com/r/broadinstitute/gatk) |
| Software | BCFtools (Docker container used in PiCTURE) [24] | Version 1.16 (SHA256 hash: 8b358095bc71232c407d72e5b8d88d727dbff39aedf928a76aca6d6bc8e35e60) | Accessed on 2025-12-10 | MIT/Expat License | Free via Docker Hub website (https://hub.docker.com/r/staphb/bcftools) |
| Software | Picard (Docker container used in PiCTURE) [25] | Version 2.18.1 (SHA256 hash: 7a7cc0d02da5f1e0e487d47559a510f2ae318a8727e39e0f95a46a59656d0c25) | Accessed on 2025-12-10 | MIT License | Free via Docker Hub website (https://hub.docker.com/r/mgibio/picard-cwl) |
| Software | MultiQC (Docker container used in PiCTURE) [26] | Version 1.14 (SHA256 hash: ca2e4b4bed61e484a301b3ee585efb73abc68c46a6944ec3735942c9e3017193) | Accessed on 2025-12-10 | GNU General Public License v3.0 | Free via Docker Hub website (https://hub.docker.com/r/ewels/multiqc) |
| Software | pysam helper (Docker container used in PiCTURE) [27] | pysam version 0.19.1 (linux/amd64: SHA256 hash: 027eab43717917325172e1d71384883d4e17552823d28afdea92ffec4df68beb, linux/arm64: SHA256 hash: 85c7774051b295124cb8e3dcbd332bf02b80a7efd9b988b279482b9ec09892a1) | Accessed on 2025-12-10 | BSD 3-Clause "New" or "Revised" License | Free via Docker Hub website (https://hub.docker.com/r/kazukinakamae/pysam) |
| Software | motif_extraction (Docker container used in PiCTURE) | Version 1.0 (linux/amd64: SHA256 hash: f530f9e5d0cb2a2918b2dc071eb2c49db296dac1be67bf02447170c3619d514a, linux/arm64: SHA256 hash: 6f06b3938db074f1797719cc436b50774ec27f308039ec5b412687006c108ad5) | Accessed on 2025-12-10 | MIT License | Free via Docker Hub website (https://hub.docker.com/r/kazukinakamae/motif_extraction) |
| Software | STAR index container (Docker container used in PiCTURE) | STAR version 2.7.4a [28], Docker image version 1.0 (SHA256 hash: ba4617f902a7d900c912ad84208944cb36dd57ae3cca125ecb534756e537e12b) | Accessed on 2025-12-10 | MIT License | Built locally by preparation.sh in PiCTURE |
| Software | MEME suites (Conda package used in promoter_dataset_generator) [29] | Version 5.5.9 (SHA256 hash: cb4fa670c3cbd498241e6e9e476754b898f8705a03e023f2aca0d122198a64d5) | Accessed on 2025-12-10 | Custom license (https://github.com/cinquin/MEME?tab=License-1-ov-file#readme) | Free for non-profit Purposes (https://conda.anaconda.org/bioconda/linux-64/meme-5.5.9-pl5321h1ca524f_0.conda) |
| Software | fastq-dump (SRA Toolkit [15]) | Version 3.0.1 (Git commit hash: 674e7b317eec5e47f9677d7e5b8f2e4ca2808514) | Accessed on 2025-12-10 | Public Domain | Free via GitHub website (https://github.com/ncbi/sra-tools/releases/tag/3.0.1) |
| Dataset | GATK resource bundle for hg38 (GRCh38.p14 + dbSNP138 [30]) | Homo_sapiens_assembly38.fasta (SHA256 hash: 93157a161863464c9435062fd67c173fdaf99cb8b32f1455018361387ffa5564) resources-broad-hg38-v0-Homo_sapiens_assembly38.dict (SHA256 hash: 07523b4a0afd1127ae909f92f8b3911aff096461aed7eff3f1acf5bdfc1dc103) resources-broad-hg38-v0-Homo_sapiens_assembly38.fasta.fai (SHA256 hash: edefd93c489dc1baefad312f40388089f8db5cf6dcc3ba0955669ead274e8b6b) resources-broad-hg38-v0-Homo_sapiens_assembly38.dbsnp138.vcf (SHA256 hash: 0ff368a3a4e16fc539be19ddbc58996d7e9a959a50e8597a7de07e217cce8527) resources-broad-hg38-v0-Homo_sapiens_assembly38.dbsnp138.vcf.idx (SHA256 hash: d0c185e0b5b43f3c2587e01c93f0c0dc81754649c0e0de69af64defa74cb33f8) resources_broad_hg38_v0_wgs_calling_regions.hg38.interval_list (SHA256 hash: 6b240928c5b367a0fcfceb4fe04a0ca8847026e6cdfbea44ae9342bed24bc7cc) | Accessed on 2025-12-10 | No license (https://gatk.broadinstitute.org/hc/en-us/community/posts/16865825134747-License-use-terms-for-GATK-resource-bundle) | Free via Broad Institute website (https://gatk.broadinstitute.org/hc/en-us/articles/360035890811-Resource-bundle), or our figshare (https://doi.org/10.6084/m9.figshare.31742074) |
| Dataset | NCBI Sequence Read Archive (SRA) FASTQ files [11] | SRR11561273_1.fastq (MD5sum hash: d8b44bd60b4854febdc6893c7b259c5d) SRR11561273_2.fastq (MD5sum hash: f7685993088b8908e67e8f5d01063f81) SRR11561289_1.fastq (MD5sum hash: 2e8039102e8a6fdace188ec00945aae3) SRR11561289_2.fastq (MD5sum hash: 045b98ecec8d4a3b1f258634f5011384) SRR11561292_1.fastq (MD5sum hash: 4e64b30c2ea84790ade1afe9c54c4bba) SRR11561292_2.fastq (MD5sum hash: f8f6e21b2b296b085187852facab484c) SRR11561298_1.fastq (MD5sum hash: ddfa3a569bc3a34faad4dfc5400d0e1b) SRR11561298_2.fastq (MD5sum hash: 42d4dd9aba8dfb4fbd1d421b68371624) SRR11561314_1.fastq (MD5sum hash: c64cc259347666c474d5b7ae14a24437) SRR11561314_2.fastq (MD5sum hash: abfe5940d0fbf79178d81b9dd4cab8db) SRR11561324_1.fastq (MD5sum hash: 19a2455a4e94a91f3b3fb4aa360f922d) SRR11561324_2.fastq (MD5sum hash: ac1c31c0dbdd8d7c914d11ec4c94ea3a) SRR11561326_1.fastq (MD5sum hash: e3acaaf6eb448b623bbab58566bc8aef) SRR11561326_2.fastq (MD5sum hash: 7db849ec149f4944de0f37ae3fec4aca) | Custom license (https://www.ncbi.nlm.nih.gov/home/about/policies/) | Free via NCBI website (https://www.ncbi.nlm.nih.gov/sra) | |
| Dataset | EPDnew promoter dataset for eukaryotes [16] | Arabidopsis thaliana: Version 004 Zea mays: Version 001 Hordeum vulgare: Version 000 Homo sapiens: Version 000 Macaca mulatta: Version 000 Mus musculus: Version 000 Rattus norvegicus: Version 000 Gallus gallus: Version 000 Canis familiaris: Version 000 Drosophila melanogaster: Version 000 Apis mellifera: Version 000 Danio rerio: Version 000 Caenorhabditis elegans: Version 000 Saccharomyces cerevisiae: Version 000 Schizosaccharomyces pombe: Version 000 | Accessed on 2025-12-03 | Creative Commons Attribution 4.0 International Public License (CC BY 4.0) | Free via EPD website (https://epd.expasy.org/epd/EPDnew_database.php) |
To improve computational reproducibility, we recorded the operating system and GPU driver versions used for the protocol. PiCTURE was executed on Ubuntu 22.04.5 LTS (Linux kernel: 5.15.0-118-generic, NVIDIA driver: version 550.54.14, CUDA runtime: version 12.4, No CUDA Toolkit [31], Docker: version 27.5.1). DNABERT-2 fine-tuning was executed on Ubuntu 22.04.5 LTS with an NVIDIA H100 (80 GB) (Linux kernel: Linux 5.15.0-163-generic, NVIDIA driver: version 580.95.05, CUDA runtime: version 13.0, CUDA Toolkit: version 12.2). The other data processing tasks were executed using only CPU resources on macOS 26.0 (Darwin kernel: version 25.2.0).
DNABERT-2 was fine-tuned using Transformers 4.29.2 and PyTorch 2.4.1 (CUDA 12.1 build). The NVIDIA driver reported CUDA 13.0 support via nvidia-smi, and CUDA Toolkit 12.2 was installed for development and optional compilation tasks.
RNA-seq dataset
RNA-seq data were used in the PiCTURE pipeline. RNA-seq datasets were downloaded from the NCBI Sequence Read Archive (SRA). In this protocol, data for BE4-rAPOBEC1–transfected samples (SRR11561324, SRR11561298, and SRR11561273), BE4-RrA3F–transfected sample (SRR11561292), and nCas9-transfected samples (SRR11561289, SRR11561314, and SRR11561326) [11] were downloaded in FASTQ format using the fastq-dump utility from the SRA Toolkit.
Procedure
文章信息
稿件历史记录
提交日期: Jan 22, 2026
接收日期: Mar 22, 2026
在线发布日期: Apr 3, 2026
出版日期: Apr 20, 2026
版权信息
© 2026 The Author(s); This is an open access article under the CC BY license (https://creativecommons.org/licenses/by/4.0/).
如何引用
Nakamae, K. and Bono, H. (2026). Workflow for Fine-Tuning and Evaluating DNA Language Models for Specific Genomics Issues. Bio-protocol 16(8): e5676. DOI: 10.21769/BioProtoc.5676.
分类
生物信息学与计算生物学
系统生物学 > 基因组学 > 功能基因组学
您对这篇实验方法有问题吗?
在此处发布您的问题,我们将邀请本文作者来回答。同时,我们会将您的问题发布到Bio-protocol Exchange,以便寻求社区成员的帮助。
Share
Bluesky
X
Copy link




