English | 简体中文

Authors: Shaolei Zhang, Ju Fan*, Meihao Fan, Guoliang Li, Xiaoyong Du

Renmin University of China, Tsinghua University

DeepAnalyze is the first agentic LLM for autonomous data science. It can autonomously complete a wide range of data-centric tasks without human intervention, supporting:

• 🛠 Entire data science pipeline: Automatically perform any data science tasks such as data preparation, analysis, modeling, visualization, and report generation.
• 🔍 Open-ended data research: Conduct deep research on diverse data sources, including structured data (Databases, CSV, Excel), semi-structured data (JSON, XML, YAML), and unstructured data (TXT, Markdown), and finally produce analyst-grade research reports.
• 📊 Fully open-source: The model, code, training data, and demo of DeepAnalyze are all open-sourced, allowing you to deploy or extend your own data analysis assistant.

• [2025.10.28]: We welcome all contributions, including improving the DeepAnalyze and sharing use cases (see CONTRIBUTION.md). All merged PRs will be listed as contributors.
• [2025.10.27]: DeepAnalyze has attracted widespread attention, gaining 1K+ GitHub stars and 200K+ Twitter views within a week.
• [2025.10.21]: DeepAnalyze's paper, code, model, training data are released!

Upload the data, DeepAnalyze can perform data-oriented deep research 🔍 and any data-centric tasks 🛠

• Clone this repo and download DeepAnalyze-8B.
• Deploy DeepAnalyze-8B via vllm: vllm serve DeepAnalyze-8B
• Run these scripts to launch the API and interface, and then interact through the browser (http://localhost:4000):

  cd demo/chat
  npm install
  cd ..
  bash start.sh
  
  # stop the api and interface
  bash stop.sh

• If you want to deploy under a specific IP, please replace localhost with your IP address in ./demo/backend.py and ./demo/chat/lib/config.ts

• Install packages: torch, transformers, vllm>=0.8.5

  conda create -n deepanalyze python=3.12 -y
  conda activate deepanalyze
  pip install -r requirements.txt
  
  # For training
  (cd ./deepanalyze/ms-swift/ && pip install -e .)
  (cd ./deepanalyze/SkyRL/ && pip install -e .)

• requirements.txt lists the minimal dependencies required for DeepAnalyze inference. For training, please refer to ./deepanalyze/ms-swift/requirements.txt and ./deepanalyze/SkyRL/pyproject.toml
• We recommend separating the inference and training environments to avoid dependency conflicts.

• Deploy DeepAnalyze-8B via vllm: vllm serve DeepAnalyze-8B

• Run these scripts for any data science tasks:

  • You can specify any data science tasks, including specific data tasks and open-ended data research.
  • You can specify any number of data sources, and DeepAnalyze will automatically explore them.
  • You can specify any type of data sources, e.g., structured data (Databases, CSV, Excel), semi-structured data (JSON, XML, YAML), and unstructured data (TXT, Markdown)

  from deepanalyze import DeepAnalyzeVLLM
  
  prompt = """# Instruction
  Generate a data science report.
  
  # Data
  File 1: {"name": "bool.xlsx", "size": "4.8KB"}
  File 2: {"name": "person.csv", "size": "10.6KB"}
  File 3: {"name": "disabled.xlsx", "size": "5.6KB"}
  File 4: {"name": "enlist.csv", "size": "6.7KB"}
  File 5: {"name": "filed_for_bankrupcy.csv", "size": "1.0KB"}
  File 6: {"name": "longest_absense_from_school.xlsx", "size": "16.0KB"}
  File 7: {"name": "male.xlsx", "size": "8.8KB"}
  File 8: {"name": "no_payment_due.xlsx", "size": "15.6KB"}
  File 9: {"name": "unemployed.xlsx", "size": "5.6KB"}
  File 10: {"name": "enrolled.csv", "size": "20.4KB"}"""
  
  workspace = "/home/u2023000922/zhangshaolei/deepanalyze_public/DeepAnalyze/example/analysis_on_student_loan/"
  
  deepanalyze = DeepAnalyzeVLLM(
      "/fs/fast/u2023000922/zhangshaolei/checkpoints/deepanalyze-8b/"
  )
  answer = deepanalyze.generate(prompt, workspace=workspace)
  print(answer["reasoning"])

  You shoud get a deep research report, which can be rendered as a PDF.:

  # Comprehensive Analysis of Student Enrollment Patterns and Institutional Transfers
  
  ## Introduction and Research Context
  
  The analysis of student enrollment patterns represents a critical area of educational research with significant implications for institutional planning, resource allocation, and student support services. This comprehensive study examines a comprehensive dataset encompassing 1,194 enrollment records across six educational institutions, merged with supplementary demographic, financial, and employment status data. The research employs advanced analytical techniques including network analysis, predictive modeling, and temporal pattern recognition to uncover both macro-level institutional trends and micro-level student mobility patterns. The dataset's longitudinal nature, spanning fifteen months of enrollment records, provides unique insights into the complex dynamics of student pathways through higher education systems.
  
  Our methodological approach combines quantitative analysis of enrollment durations, transfer probabilities, and financial indicators with qualitative ...
  
  The research contributes to the growing body of literature on student mobility by providing empirical evidence of institutional transfer networks and their relationship to student outcomes...
  .....
  

  

  For more examples and task completion details, please refer to DeepAnalyze's homepage.

• You can build an OpenAI-Style API, using this script (note to change MODEL_PATH = "DeepAnalyze-8B" in demo/backend.py to your vllm model name):

  python demo/backend.py
  

• API usage (streaming response):

  curl -X POST http://localhost:8200/chat/completions \
       -H "Content-Type: application/json" \
       -d '{
             "messages": [
               {
                 "role": "user",
                 "content": "Generate a data science report."
               }
             ],
             "workspace": "example/student_loan/"
           }'
  

• Download DeepSeek-R1-0528-Qwen3-8B. Or you can directly finetune based on DeepAnalyze-8B.

  • If you use DeepSeek-R1-0528-Qwen3-8B as the base model, you should add the special tokens, using:

    MODEL_PATH=path_to_DeepSeek-R1-0528-Qwen3-8B
    SAVE_PATH=path_to_save_DeepSeek-R1-0528-Qwen3-8B-addvocab
    
    python deepanalyze/add_vocab.py \
      --model_path "$MODEL_PATH" \
      --save_path "$SAVE_PATH" \
      --add_tags

• Download training data DataScience-Instruct-500K.

  • unzip DataScience-Instruct-500K/RL/data.zip

• Single-ability Fine-tuning: ./scripts/single.sh
• Multi-ability Agentic Training (cold start): ./scripts/multi_coldstart.sh
• Multi-ability Agentic Training (RL): ./scripts/multi_rl.sh

• We have unified the evaluation of most existing data science benchmarks using vLLM (with more being continuously added...). You can directly follow the introduction in ./playground to quickly evaluate DeepAnalyze or your own agent.

We welcome all forms of contributions, and merged PRs will be listed as contributors.

• We welcome all forms of contributions on DeepAnalyze's code, model and UI, such as Docker packaging, DeepAnalyze model conversion and quantization, and submitting DeepAnalyze workflows based on closed-source LLMs.
• You can submit a pull request directly.

• We also especially encourage you to share your use cases and feedback when using DeepAnalyze; these are extremely valuable for helping us improve DeepAnalyze.
• You can place your use cases in a new folder under .example/. We recommend following the folder structure of .example/analysis_on_student_loan/, which includes three parts:
  • data/: stores the uploaded files
  • prompt.txt: input instructions
  • README.md: documentation. We suggest including the input, DeepAnalyze’s output, outputs from other closed-source LLMs (optional), and your evaluation/comments of the case.
• DeepAnalyze only has 8B parameters, so we also welcome examples where DeepAnalyze performs slightly worse than the closed-source LLMs — this will help us improve DeepAnalyze.

• Training framework: ms-swift, SkyRL
• Source of Training Data: Reasoning-Table, Spider, BIRD, DABStep

If this repository is useful for you, please cite as:

@misc{deepanalyze,
      title={DeepAnalyze: Agentic Large Language Models for Autonomous Data Science}, 
      author={Shaolei Zhang and Ju Fan and Meihao Fan and Guoliang Li and Xiaoyong Du},
      year={2025},
      eprint={2510.16872},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2510.16872}, 
}

If you have any questions, please feel free to submit an issue or contact zhangshaolei98@ruc.edu.cn.

Welcome to join the DeepAnalyze WeChat group, chat and share ideas with others!

If you like DeepAnalyze, give it a GitHub Star ⭐.

A Linux/CUDA/NVIDIA GPU environment compatible with vLLM and PyTorch. Model weights are obtained separately. See requirements.txt; memory requirements depend on model and inference settings.

git clone https://github.com/RoxyXu151/DeepAnalyze.git
cd DeepAnalyze

Private repositories require GitHub authentication. Relative paths and steps below start from this repository.

python -m pip install -r requirements.txt

run.py contains upstream machine-specific model and workspace paths; replace them with local model and input directories before running. Read docker/README.md for containers. Compose defaults to an interactive shell rather than automatically launching the model API.

Prepare a trusted data workspace and follow the DeepAnalyzeVLLM example in the English README. run.py demonstrates prompt, workspace, and generate calls, producing reasoning and workspace artifacts. The web demo is under demo/ and needs its own script configuration.

• requirements.txt
• run.py
• demo
• docker
• example
• CONTRIBUTION.md
• docker/README.md
• example/README.md

Check model loading, file access, and artifacts with a small dataset and simple question. Review generated code and conclusions manually. GPU inference and paper metrics were not reproduced by this documentation update. See CONTRIBUTION.md and the English home page for contribution and citation details.

Keep both READMEs in sync when functionality or usage changes. Report reproducible steps, environment versions, and sanitized logs. Describe the verification actually performed with each change.

See LICENSE for repository terms. Third-party code, data, models, papers, and artwork retain their own licenses and attribution.

This is a fork; upstream authorship, citations, and copyright notices remain in place.