Skip to content
BranchesTags

Repository files navigation

ADSA (Autonomous Data Scientist Agent)

Atenea Labs Logo

By Atenea Labs Research Team

馃嚞馃嚙 English | 馃嚜馃嚫 Espa帽ol

馃嚞馃嚙 English

Overview

ADSA is an autonomous multi-agent system developed by Atenea Labs that automates the entire machine learning process. The system has demonstrated the capability to outperform 98% of data scientists in global competitions.

馃搫 Read our detailed whitepaper: English Version

About Atenea Labs

Atenea Labs is a leading AI consultancy firm that specializes in transforming businesses through cutting-edge artificial intelligence solutions. Our team of experts helps companies leverage the power of AI to:

  • 馃殌 Accelerate Digital Transformation
  • 馃挕 Develop Custom AI Solutions
  • 馃搳 Automate Data Analysis
  • 馃攧 Optimize Business Processes
  • 馃幆 Drive Innovation

Visit atenealabs.com to learn how we can help transform your business with AI.

Key Features

  • Autonomous Operation: Fully automated machine learning pipeline
  • Multi-Agent Architecture: Utilizes multiple AI agents for different tasks
  • Iterative Optimization: Continuously improves models through multiple iterations
  • GPU Acceleration: Automatic GPU detection and utilization when available
  • Error Recovery: Self-healing capabilities with automatic error detection and correction
  • Performance Monitoring: Detailed progress tracking and reporting

System Components

  1. API Clients Setup

    • Azure OpenAI integration
    • Anthropic Claude integration
    • Environment configuration management

  2. Data Processing

    • Automated data loading and preparation
    • Dynamic feature engineering
    • Training/test set management

  3. ML Solution Generation

    • Automated code generation
    • Model selection and optimization
    • GPU utilization when available
    • Cross-validation implementation

  4. Error Handling & Optimization

    • Automatic error detection
    • Code optimization for performance
    • Timeout management
    • Self-healing capabilities

Requirements

  • Python 3.8+
  • Required Python packages (see requirements.txt)
  • API Keys:
    • Azure OpenAI API
    • Anthropic Claude API

API Setup

Azure OpenAI Setup

  1. Visit Azure OpenAI Service
  2. Sign up for an Azure account if you don't have one
  3. Create a new Azure OpenAI resource
  4. Deploy o1 model in your resource
  5. Get your API credentials:
    • Endpoint URL
    • API Key
    • Deployment Name
    • API Version

Anthropic Claude Setup

  1. Visit Anthropic Claude
  2. Sign up for an account
  3. Navigate to API settings
  4. Generate an API key

Installation

  1. Clone the repository:
git clone https://github.com/atenealabs/adsa.git
cd adsa
  1. Install dependencies:
pip install -r requirements.txt
  1. Configure environment variables:
cp .env.example .env
# Edit .env with your API keys:
# AZURE_OPENAI_KEY=your_azure_key
# AZURE_OPENAI_ENDPOINT=your_azure_endpoint
# AZURE_OPENAI_DEPLOYMENT=your_deployment_name
# AZURE_OPENAI_VERSION=your_api_version
# ANTHROPIC_API_KEY=your_claude_key

Problem Specification

Before running ADSA, you need to create a problem_info.md file that describes your machine learning task. This file is crucial as it guides the AI agents in understanding and solving your problem.

problem_info.md Structure

# Problem Description
[Brief description of the problem]

## Objective
[Clear statement of what needs to be predicted/classified/etc.]

## Data Description
- Input file: train.csv
- Test file: test.csv
- Target variable: [name of the target column]
- Features: [list of important features]

## Evaluation Metric
[Specify the metric used to evaluate the model (e.g., accuracy, RMSE, etc.)]

## Constraints
- [Any time constraints]
- [Memory constraints]
- [Other technical constraints]

## Expected Output Format
[Description of the required output format]

Example problem_info.md

# Spaceship Titanic Classification

## Objective
Predict which passengers were transported to an alternate dimension during the Spaceship Titanic's collision with a spacetime anomaly.

## Data Description
- Input file: train.csv (8700 passengers)
- Test file: test.csv (4300 passengers)
- Target variable: Transported (boolean)
- Features: PassengerId, HomePlanet, CryoSleep, Cabin, Destination, Age, VIP, RoomService, FoodCourt, ShoppingMall, Spa, VRDeck

## Evaluation Metric
Binary classification accuracy

## Constraints
- Maximum runtime: 30 minutes per iteration
- Memory usage: <32GB RAM
- GPU acceleration when available

## Expected Output Format
CSV file with columns:
- PassengerId
- Transported (boolean: True/False)

Usage

  1. Place your training data in train.csv
  2. Place your test data in test.csv
  3. Create and configure problem_info.md as described above
  4. Run ADSA:
python adsa.py

Output Files

  • solution.py: Current iteration's ML solution
  • progress_report.json: Performance metrics and progress tracking
  • results.csv: Final predictions and results
  • execution_logs.txt: Detailed execution logs
  • older_solutions/: Archive of previous iterations

Architecture

ADSA operates through an iterative process:

  1. Initial Setup

    • Environment configuration
    • API client initialization
    • Data preparation

  2. Solution Generation

    • ML code generation via Azure OpenAI
    • Code optimization and error checking
    • GPU utilization verification

  3. Execution & Monitoring

    • Solution implementation
    • Performance tracking
    • Error detection and recovery

  4. Optimization Loop

    • Performance analysis
    • Code improvement suggestions
    • Iterative refinement

Performance

  • Maximum runtime: 30 minutes per iteration
  • Maximum iterations: 50
  • Automatic performance optimization
  • GPU acceleration when available

License

See LICENSE.md for details.

Contact

Transform your business with AI:

About the Team

ADSA is developed by the Atenea Labs Research Team, led by Carlos Navarro Cabrera. Our team combines expertise in artificial intelligence, machine learning, and software engineering to create cutting-edge AI solutions that drive business transformation.

馃嚜馃嚫 Espa帽ol

Descripci贸n General

ADSA es un sistema multi-agente aut贸nomo desarrollado por Atenea Labs que automatiza todo el proceso de machine learning. El sistema ha demostrado la capacidad de superar al 98% de los cient铆ficos de datos en competiciones globales.

馃搫 Lee nuestro whitepaper detallado: Versi贸n en Espa帽ol

Sobre Atenea Labs

Atenea Labs es una consultora l铆der en Inteligencia Artificial que se especializa en transformar empresas mediante soluciones de IA de vanguardia. Nuestro equipo de expertos ayuda a las empresas a aprovechar el poder de la IA para:

  • 馃殌 Acelerar la Transformaci贸n Digital
  • 馃挕 Desarrollar Soluciones de IA Personalizadas
  • 馃搳 Automatizar el An谩lisis de Datos
  • 馃攧 Optimizar Procesos de Negocio
  • 馃幆 Impulsar la Innovaci贸n

Visita atenealabs.com para descubrir c贸mo podemos ayudar a transformar tu empresa con IA.

Caracter铆sticas Principales

  • Operaci贸n Aut贸noma: Pipeline de machine learning completamente automatizado
  • Arquitectura Multi-Agente: Utiliza m煤ltiples agentes de IA para diferentes tareas
  • Optimizaci贸n Iterativa: Mejora continua de modelos a trav茅s de m煤ltiples iteraciones
  • Aceleraci贸n GPU: Detecci贸n autom谩tica y utilizaci贸n de GPU cuando est谩 disponible
  • Recuperaci贸n de Errores: Capacidades de auto-reparaci贸n con detecci贸n y correcci贸n autom谩tica de errores
  • Monitoreo de Rendimiento: Seguimiento y reportes detallados del progreso

Componentes del Sistema

  1. Configuraci贸n de Clientes API

    • Integraci贸n con Azure OpenAI
    • Integraci贸n con Anthropic Claude
    • Gesti贸n de configuraci贸n del entorno

  2. Procesamiento de Datos

    • Carga y preparaci贸n automatizada de datos
    • Ingenier铆a de caracter铆sticas din谩mica
    • Gesti贸n de conjuntos de entrenamiento/prueba

  3. Generaci贸n de Soluciones ML

    • Generaci贸n autom谩tica de c贸digo
    • Selecci贸n y optimizaci贸n de modelos
    • Utilizaci贸n de GPU cuando est谩 disponible
    • Implementaci贸n de validaci贸n cruzada

  4. Manejo de Errores y Optimizaci贸n

    • Detecci贸n autom谩tica de errores
    • Optimizaci贸n de c贸digo para rendimiento
    • Gesti贸n de tiempos de ejecuci贸n
    • Capacidades de auto-reparaci贸n

Requisitos

  • Python 3.8+
  • Paquetes Python requeridos (ver requirements.txt)
  • Claves API:
    • API de Azure OpenAI
    • API de Anthropic Claude

Instalaci贸n

  1. Clonar el repositorio:
git clone https://github.com/atenealabs/adsa.git
cd adsa
  1. Instalar dependencias:
pip install -r requirements.txt
  1. Configurar variables de entorno:
cp .env.example .env
# Editar .env con tus claves API

Uso

  1. Coloca tus datos de entrenamiento en train.csv
  2. Coloca tus datos de prueba en test.csv
  3. Configura las especificaciones del problema en problem_info.md
  4. Ejecuta ADSA:
python adsa.py

Archivos de Salida

  • solution.py: Soluci贸n ML de la iteraci贸n actual
  • progress_report.json: M茅tricas de rendimiento y seguimiento del progreso
  • results.csv: Predicciones y resultados finales
  • execution_logs.txt: Logs detallados de ejecuci贸n
  • older_solutions/: Archivo de iteraciones anteriores

Arquitectura

ADSA opera a trav茅s de un proceso iterativo:

  1. Configuraci贸n Inicial

    • Configuraci贸n del entorno
    • Inicializaci贸n de clientes API
    • Preparaci贸n de datos

  2. Generaci贸n de Soluciones

    • Generaci贸n de c贸digo ML v铆a Azure OpenAI
    • Optimizaci贸n y verificaci贸n de c贸digo
    • Verificaci贸n de utilizaci贸n de GPU

  3. Ejecuci贸n y Monitoreo

    • Implementaci贸n de soluciones
    • Seguimiento del rendimiento
    • Detecci贸n y recuperaci贸n de errores

  4. Ciclo de Optimizaci贸n

    • An谩lisis de rendimiento
    • Sugerencias de mejora de c贸digo
    • Refinamiento iterativo

Rendimiento

  • Tiempo m谩ximo de ejecuci贸n: 30 minutos por iteraci贸n
  • Iteraciones m谩ximas: 50
  • Optimizaci贸n autom谩tica del rendimiento
  • Aceleraci贸n GPU cuando est谩 disponible

Licencia

Ver LICENSE.md para m谩s detalles.

Contacto

Transforma tu empresa con IA:

Sobre el Equipo

ADSA es desarrollado por el Equipo de Investigaci贸n de Atenea Labs, liderado por Carlos Navarro Cabrera. Nuestro equipo combina experiencia en inteligencia artificial, machine learning e ingenier铆a de software para crear soluciones de IA de vanguardia que impulsan la transformaci贸n empresarial.

About

ADSA is an Autonomous Data Scientist Agent who reach top 1% in Global ML competitions

Resources

Readme
License
Activity
Custom properties

Stars

4 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages