How LinkedIn built a comprehensive agent platform serving 20+ teams and 30+ production services

Introduction

As AI agents move from experimental proof-of-concepts to production-scale systems, organizations face a critical question: how do you scale not just the technical infrastructure, but the adoption and development of agents across an entire organization?

LinkedIn recently shared their journey of building agents at scale, culminating in their first production agent - the LinkedIn Hiring Assistant. This isn't just a story about building one successful agent, but about creating an entire ecosystem that has enabled over 20 teams to deploy 30+ production AI services.

The Two Dimensions of Scale

When we think about scaling AI agents, LinkedIn identified two critical dimensions:

Performance Scale: The traditional engineering concern of handling massive data volumes and maintaining system performance under load.

Adoption Scale: The more subtle but equally important challenge of scaling agent development across an organization, ensuring teams can innovate quickly and generate the best ideas.

While most discussions focus on the first dimension, LinkedIn's success came from solving the second.

[[NEWSLETTER]]

graph LR
    subgraph "Scale Dimensions"
        subgraph "Performance Scale"
            PS1[High Data Volume] --> PS2[System Performance]
            PS2 --> PS3[Load Handling]
        end
        
        subgraph "Adoption Scale"
            AS1[20+ Teams] --> AS2[30+ Services]
            AS2 --> AS3[Days vs Weeks<br/>Development]
        end
        
        subgraph "Key Results"
            R1[Standardized Patterns]
            R2[Developer Productivity]
            R3[Production Quality]
        end
        
        PS3 --> R1
        AS3 --> R2
        R1 --> R3
        R2 --> R3
    end
    
    style PS1 fill:#ffeb3b,color:#000
    style PS2 fill:#ffeb3b,color:#000
    style PS3 fill:#ffeb3b,color:#000
    style AS1 fill:#4caf50,color:#fff
    style AS2 fill:#4caf50,color:#fff
    style AS3 fill:#4caf50,color:#fff
    style R1 fill:#ff9800,color:#fff
    style R2 fill:#ff9800,color:#fff
    style R3 fill:#ff9800,color:#fff

LinkedIn Hiring Assistant: The Production Blueprint

LinkedIn's first production agent, the Hiring Assistant, demonstrates what thoughtful agent design looks like in practice. The agent follows an "ambient agent" pattern:

Initial Input: Recruiters describe the role they want to fill and attach relevant documents
Automatic Processing: The agent generates qualifications based on input and supplementary documents
Background Execution: The system works asynchronously, notifying the recruiter of progress
Results Delivery: Candidates are sourced and presented in a detailed review interface

Under the hood, this follows a supervisor multi-agent architecture where a main coordinator manages specialized sub-agents, each capable of interacting with LinkedIn's existing services through tool calling (or as LinkedIn calls them, "skills").

graph TB
    subgraph "LinkedIn Hiring Assistant Flow"
        A[Recruiter Input<br/>Job Description + Documents] --> B[Agent Processing<br/>Generate Qualifications]
        B --> C[Background Execution<br/>Candidate Sourcing]
        C --> D[Results Notification<br/>Candidates Ready]
        D --> E[Detailed Review Interface<br/>Candidate List]
    end
    
    style A fill:#e1f5fe
    style B fill:#f3e5f5
    style C fill:#fff3e0
    style D fill:#e8f5e8
    style E fill:#fce4ec

graph TB
    subgraph "Multi-Agent Architecture"
        SA[Supervisor Agent<br/>Coordinator] --> SB[Sourcing Agent<br/>Candidate Search]
        SA --> SC[Qualification Agent<br/>Requirements Processing]
        SA --> SD[Communication Agent<br/>User Interface]
        
        SB --> T1[LinkedIn Search API]
        SB --> T2[Profile Analysis]
        SC --> T3[Document Processing]
        SC --> T4[Skills Extraction]
        SD --> T5[Messaging System]
        SD --> T6[Notification Service]
    end
    
    style SA fill:#ff9800,color:#fff
    style SB fill:#2196f3,color:#fff
    style SC fill:#4caf50,color:#fff
    style SD fill:#9c27b0,color:#fff
    style T1 fill:#e3f2fd
    style T2 fill:#e3f2fd
    style T3 fill:#e8f5e8
    style T4 fill:#e8f5e8
    style T5 fill:#f3e5f5
    style T6 fill:#f3e5f5

The Great Python Migration

Perhaps the most surprising aspect of LinkedIn's agent strategy was their bold decision to abandon Java for AI applications and standardize on Python - despite Java being their primary language for business logic.

Why the Switch Was Necessary

Initially, LinkedIn tried to build GenAI applications in Java, their traditional stack. This worked for simple prompt-based applications but quickly revealed fundamental problems:

Experimentation Friction: Teams wanted to prototype in Python but were forced to production-ize in Java
Innovation Barriers: The language gap made it nearly impossible to leverage the rapidly evolving Python-based AI ecosystem
Development Speed: Every new technique, library, or model required custom Java implementations

The Decision Point

LinkedIn made several key observations that led to their Python adoption:

Undeniable Demand: Teams across all verticals wanted to use generative AI
Java-Python Gap: The existing stack wasn't working for AI-specific needs
Ecosystem Reality: Staying current with AI developments required Python integration

Their solution was characteristically bold: use Python for everything - business logic, engineering, evaluations, and production services.

graph LR
    subgraph "Technology Stack Evolution"
        subgraph "Before (Java-based)"
            J1[Java Business Logic] --> J2[Limited AI Libraries]
            J2 --> J3[Custom Implementations]
            J3 --> J4[Slow Development]
        end
        
        subgraph "After (Python-based)"
            P1[Python gRPC Services] --> P2[LangChain/LangGraph]
            P2 --> P3[Rich AI Ecosystem]
            P3 --> P4[Rapid Development]
        end
        
        J4 -.->|Migration| P1
    end
    
    style J1 fill:#ffcdd2
    style J2 fill:#ffcdd2
    style J3 fill:#ffcdd2
    style J4 fill:#ffcdd2
    style P1 fill:#c8e6c9
    style P2 fill:#c8e6c9
    style P3 fill:#c8e6c9
    style P4 fill:#c8e6c9

Building the Agent Framework

LinkedIn's agent framework centers around several key architectural decisions:

Core Technology Stack

Python gRPC: For service communication with built-in streaming, binary serialization, and cross-language support
LangChain & LangGraph: For modeling core business logic and agent workflows
Standard Utilities: Centralized tools for LLM inference, conversational memory, and checkpointing

Why LangChain and LangGraph?

The choice wasn't arbitrary. LinkedIn evaluated the ecosystem and selected these tools for specific reasons:

Ease of Use: Even Java engineers could quickly understand and implement agent logic. The syntax is intuitive enough that teams could build non-trivial applications in days rather than weeks.

Sensible Interfaces: The abstractions provided clean ways to model LinkedIn's internal infrastructure. For example, teams could switch between Azure OpenAI and on-premise models with just a few lines of code.

Community Integration: Pre-built implementations like ReAct agents and community integrations dramatically accelerated development.

The Agent Platform: Solving Distribution Challenges

Building individual agents is one challenge; orchestrating them at scale is another entirely. LinkedIn identified two critical problems with agentic systems:

Long-Running Processes: Agents can take significant time to process data, requiring asynchronous execution patterns
Complex Dependencies: Agents need to execute in parallel while respecting dependencies between their outputs

Messaging as the Foundation

LinkedIn solved the first problem by modeling agent communication as a messaging challenge, extending their existing robust messaging infrastructure to support:

Agent-to-agent communication
User-to-agent messaging
Automatic retry mechanisms through queuing systems

Layered Memory Architecture

For the dependency problem, LinkedIn built a sophisticated memory system with different scopes:

Working Memory: For immediate interaction context
Long-term Memory: For persistent agent-user relationships
Collective Memory: For shared knowledge across agent instances

This layered approach allows agents to maintain context appropriately while scaling across interactions.

Skills: Beyond Simple Tool Calling

LinkedIn's "skills" concept extends traditional function calling in several important ways:

Scope: Skills aren't limited to local functions - they can be RPC calls, database queries, prompts, or even other agents

Execution Model: Skills support both synchronous and asynchronous invocation, critical for the ambient agent pattern

Centralization: A skill registry allows teams to expose capabilities that other agents can discover and use

This design remarkably anticipated the Model Context Protocol (MCP) concept well before it was formally introduced.

graph TB
    subgraph "Agent Platform Architecture"
        subgraph "Messaging Layer"
            M1[Agent-to-Agent Messages]
            M2[User-to-Agent Messages]
            M3[Retry Queue System]
        end
        
        subgraph "Memory Layer"
            MEM1[Working Memory<br/>Current Context]
            MEM2[Long-term Memory<br/>User History]
            MEM3[Collective Memory<br/>Shared Knowledge]
        end
        
        subgraph "Skills Registry"
            S1[RPC Calls]
            S2[Database Queries]
            S3[Other Agents]
            S4[External APIs]
        end
        
        subgraph "Application Framework"
            AF1[Python gRPC]
            AF2[LangChain/LangGraph]
            AF3[Standard Utilities]
        end
        
        M1 --> MEM1
        M2 --> MEM2
        M3 --> MEM3
        S1 --> AF1
        S2 --> AF2
        S3 --> AF3
        S4 --> AF1
    end
    
    style M1 fill:#e1f5fe
    style M2 fill:#e1f5fe
    style M3 fill:#e1f5fe
    style MEM1 fill:#fff3e0
    style MEM2 fill:#fff3e0
    style MEM3 fill:#fff3e0
    style S1 fill:#f3e5f5
    style S2 fill:#f3e5f5
    style S3 fill:#f3e5f5
    style S4 fill:#f3e5f5
    style AF1 fill:#e8f5e8
    style AF2 fill:#e8f5e8
    style AF3 fill:#e8f5e8

sequenceDiagram
    participant R as Recruiter
    participant SA as Supervisor Agent
    participant SRC as Sourcing Agent
    participant SR as Skills Registry
    participant API as LinkedIn API
    
    Note over R,API: Skills Execution Flow
    
    R->>SA: "Find mid-level engineer"
    SA->>SRC: Task: Search candidates
    SRC->>SR: Request suitable skill
    SR-->>SRC: Return search skill
    SRC->>API: Execute skill (search)
    API-->>SRC: Return candidates
    SRC-->>SA: Processed results
    SA-->>R: Candidate recommendations
    
    Note over SRC,SR: Async execution supported
    Note over API: External service integration

Production Lessons and Key Insights

LinkedIn's journey offers several critical insights for organizations scaling AI agents:

Invest in Developer Productivity

The AI space moves incredibly fast. Success requires making it as easy as possible for developers to:

Adopt best practices automatically
Adapt quickly to new techniques and models
Contribute without extensive specialized knowledge

Standardizing patterns and providing opinionated frameworks dramatically reduces the barrier to entry.

Don't Forget Production Fundamentals

Despite the novelty of AI agents, they're still production software systems. This means:

Reliability: Traditional availability and performance concerns still apply
Observability: Custom observability solutions are essential for agentic execution patterns
Evaluation: Robust testing frameworks must account for non-deterministic behavior

Results That Speak

LinkedIn's approach has proven successful at scale:

20+ teams actively using the framework
30+ production services supporting GenAI product experiences
Days instead of weeks for building non-trivial agent applications

The Path Forward

LinkedIn's experience demonstrates that scaling AI agents requires more than just technical infrastructure - it demands a comprehensive approach to developer experience, organizational adoption, and production operations.

Their success came from recognizing that the bottleneck isn't usually computational resources, but rather the ability of teams across an organization to innovate quickly and reliably with AI agent technology.

As AI agents become increasingly central to product experiences, LinkedIn's blueprint offers a proven path for organizations looking to scale beyond single-agent experiments to comprehensive agent-powered ecosystems.

The key insight? Scale isn't just about handling more data or users - it's about enabling more teams to build better agents, faster. LinkedIn's platform approach shows how to achieve both dimensions of scale simultaneously.

References

📺 Watch the Full Presentation: Agents at Scale using LangGraph - LinkedIn

This blog post is based on LinkedIn's presentation about their agent scaling journey. Their approach demonstrates how thoughtful platform engineering can accelerate AI adoption across large organizations while maintaining production quality and reliability.

Tag: Multi-Agent Systems

Filter by Tags

Posts tagged with "Multi-Agent Systems"

Scaling AI Agents at LinkedIn: From Framework to Production