*How LinkedIn built a comprehensive agent platform serving 20+ teams and 30+ production services* ## Introduction As AI agents move from experimental proof-of-concepts to production-scale systems, organizations face a critical question: how do you scale not just the technical infrastructure, but the adoption and development of agents across an entire organization? LinkedIn recently shared their journey of building agents at scale, culminating in their first production agent - the LinkedIn Hiring Assistant. This isn't just a story about building one successful agent, but about creating an entire ecosystem that has enabled over 20 teams to deploy 30+ production AI services. ## The Two Dimensions of Scale When we think about scaling AI agents, LinkedIn identified two critical dimensions: **Performance Scale**: The traditional engineering concern of handling massive data volumes and maintaining system performance under load. **Adoption Scale**: The more subtle but equally important challenge of scaling agent development across an organization, ensuring teams can innovate quickly and generate the best ideas. While most discussions focus on the first dimension, LinkedIn's success came from solving the second. [[NEWSLETTER]] ```mermaid graph LR subgraph "Scale Dimensions" subgraph "Performance Scale" PS1[High Data Volume] --> PS2[System Performance] PS2 --> PS3[Load Handling] end subgraph "Adoption Scale" AS1[20+ Teams] --> AS2[30+ Services] AS2 --> AS3[Days vs Weeks
Development] end subgraph "Key Results" R1[Standardized Patterns] R2[Developer Productivity] R3[Production Quality] end PS3 --> R1 AS3 --> R2 R1 --> R3 R2 --> R3 end style PS1 fill:#ffeb3b,color:#000 style PS2 fill:#ffeb3b,color:#000 style PS3 fill:#ffeb3b,color:#000 style AS1 fill:#4caf50,color:#fff style AS2 fill:#4caf50,color:#fff style AS3 fill:#4caf50,color:#fff style R1 fill:#ff9800,color:#fff style R2 fill:#ff9800,color:#fff style R3 fill:#ff9800,color:#fff ``` ## LinkedIn Hiring Assistant: The Production Blueprint LinkedIn's first production agent, the Hiring Assistant, demonstrates what thoughtful agent design looks like in practice. The agent follows an "ambient agent" pattern: 1. **Initial Input**: Recruiters describe the role they want to fill and attach relevant documents 2. **Automatic Processing**: The agent generates qualifications based on input and supplementary documents 3. **Background Execution**: The system works asynchronously, notifying the recruiter of progress 4. **Results Delivery**: Candidates are sourced and presented in a detailed review interface Under the hood, this follows a supervisor multi-agent architecture where a main coordinator manages specialized sub-agents, each capable of interacting with LinkedIn's existing services through tool calling (or as LinkedIn calls them, "skills"). ```mermaid graph TB subgraph "LinkedIn Hiring Assistant Flow" A[Recruiter Input
Job Description + Documents] --> B[Agent Processing
Generate Qualifications] B --> C[Background Execution
Candidate Sourcing] C --> D[Results Notification
Candidates Ready] D --> E[Detailed Review Interface
Candidate List] end style A fill:#e1f5fe style B fill:#f3e5f5 style C fill:#fff3e0 style D fill:#e8f5e8 style E fill:#fce4ec ``` ```mermaid graph TB subgraph "Multi-Agent Architecture" SA[Supervisor Agent
Coordinator] --> SB[Sourcing Agent
Candidate Search] SA --> SC[Qualification Agent
Requirements Processing] SA --> SD[Communication Agent
User Interface] SB --> T1[LinkedIn Search API] SB --> T2[Profile Analysis] SC --> T3[Document Processing] SC --> T4[Skills Extraction] SD --> T5[Messaging System] SD --> T6[Notification Service] end style SA fill:#ff9800,color:#fff style SB fill:#2196f3,color:#fff style SC fill:#4caf50,color:#fff style SD fill:#9c27b0,color:#fff style T1 fill:#e3f2fd style T2 fill:#e3f2fd style T3 fill:#e8f5e8 style T4 fill:#e8f5e8 style T5 fill:#f3e5f5 style T6 fill:#f3e5f5 ``` ## The Great Python Migration Perhaps the most surprising aspect of LinkedIn's agent strategy was their bold decision to abandon Java for AI applications and standardize on Python - despite Java being their primary language for business logic. ### Why the Switch Was Necessary Initially, LinkedIn tried to build GenAI applications in Java, their traditional stack. This worked for simple prompt-based applications but quickly revealed fundamental problems: - **Experimentation Friction**: Teams wanted to prototype in Python but were forced to production-ize in Java - **Innovation Barriers**: The language gap made it nearly impossible to leverage the rapidly evolving Python-based AI ecosystem - **Development Speed**: Every new technique, library, or model required custom Java implementations ### The Decision Point LinkedIn made several key observations that led to their Python adoption: 1. **Undeniable Demand**: Teams across all verticals wanted to use generative AI 2. **Java-Python Gap**: The existing stack wasn't working for AI-specific needs 3. **Ecosystem Reality**: Staying current with AI developments required Python integration Their solution was characteristically bold: use Python for everything - business logic, engineering, evaluations, and production services. ```mermaid graph LR subgraph "Technology Stack Evolution" subgraph "Before (Java-based)" J1[Java Business Logic] --> J2[Limited AI Libraries] J2 --> J3[Custom Implementations] J3 --> J4[Slow Development] end subgraph "After (Python-based)" P1[Python gRPC Services] --> P2[LangChain/LangGraph] P2 --> P3[Rich AI Ecosystem] P3 --> P4[Rapid Development] end J4 -.->|Migration| P1 end style J1 fill:#ffcdd2 style J2 fill:#ffcdd2 style J3 fill:#ffcdd2 style J4 fill:#ffcdd2 style P1 fill:#c8e6c9 style P2 fill:#c8e6c9 style P3 fill:#c8e6c9 style P4 fill:#c8e6c9 ``` ## Building the Agent Framework LinkedIn's agent framework centers around several key architectural decisions: ### Core Technology Stack - **Python gRPC**: For service communication with built-in streaming, binary serialization, and cross-language support - **LangChain & LangGraph**: For modeling core business logic and agent workflows - **Standard Utilities**: Centralized tools for LLM inference, conversational memory, and checkpointing ### Why LangChain and LangGraph? The choice wasn't arbitrary. LinkedIn evaluated the ecosystem and selected these tools for specific reasons: **Ease of Use**: Even Java engineers could quickly understand and implement agent logic. The syntax is intuitive enough that teams could build non-trivial applications in days rather than weeks. **Sensible Interfaces**: The abstractions provided clean ways to model LinkedIn's internal infrastructure. For example, teams could switch between Azure OpenAI and on-premise models with just a few lines of code. **Community Integration**: Pre-built implementations like ReAct agents and community integrations dramatically accelerated development. ## The Agent Platform: Solving Distribution Challenges Building individual agents is one challenge; orchestrating them at scale is another entirely. LinkedIn identified two critical problems with agentic systems: 1. **Long-Running Processes**: Agents can take significant time to process data, requiring asynchronous execution patterns 2. **Complex Dependencies**: Agents need to execute in parallel while respecting dependencies between their outputs ### Messaging as the Foundation LinkedIn solved the first problem by modeling agent communication as a messaging challenge, extending their existing robust messaging infrastructure to support: - Agent-to-agent communication - User-to-agent messaging - Automatic retry mechanisms through queuing systems ### Layered Memory Architecture For the dependency problem, LinkedIn built a sophisticated memory system with different scopes: - **Working Memory**: For immediate interaction context - **Long-term Memory**: For persistent agent-user relationships - **Collective Memory**: For shared knowledge across agent instances This layered approach allows agents to maintain context appropriately while scaling across interactions. ### Skills: Beyond Simple Tool Calling LinkedIn's "skills" concept extends traditional function calling in several important ways: **Scope**: Skills aren't limited to local functions - they can be RPC calls, database queries, prompts, or even other agents **Execution Model**: Skills support both synchronous and asynchronous invocation, critical for the ambient agent pattern **Centralization**: A skill registry allows teams to expose capabilities that other agents can discover and use This design remarkably anticipated the Model Context Protocol (MCP) concept well before it was formally introduced. ```mermaid graph TB subgraph "Agent Platform Architecture" subgraph "Messaging Layer" M1[Agent-to-Agent Messages] M2[User-to-Agent Messages] M3[Retry Queue System] end subgraph "Memory Layer" MEM1[Working Memory
Current Context] MEM2[Long-term Memory
User History] MEM3[Collective Memory
Shared Knowledge] end subgraph "Skills Registry" S1[RPC Calls] S2[Database Queries] S3[Other Agents] S4[External APIs] end subgraph "Application Framework" AF1[Python gRPC] AF2[LangChain/LangGraph] AF3[Standard Utilities] end M1 --> MEM1 M2 --> MEM2 M3 --> MEM3 S1 --> AF1 S2 --> AF2 S3 --> AF3 S4 --> AF1 end style M1 fill:#e1f5fe style M2 fill:#e1f5fe style M3 fill:#e1f5fe style MEM1 fill:#fff3e0 style MEM2 fill:#fff3e0 style MEM3 fill:#fff3e0 style S1 fill:#f3e5f5 style S2 fill:#f3e5f5 style S3 fill:#f3e5f5 style S4 fill:#f3e5f5 style AF1 fill:#e8f5e8 style AF2 fill:#e8f5e8 style AF3 fill:#e8f5e8 ``` ```mermaid sequenceDiagram participant R as Recruiter participant SA as Supervisor Agent participant SRC as Sourcing Agent participant SR as Skills Registry participant API as LinkedIn API Note over R,API: Skills Execution Flow R->>SA: "Find mid-level engineer" SA->>SRC: Task: Search candidates SRC->>SR: Request suitable skill SR-->>SRC: Return search skill SRC->>API: Execute skill (search) API-->>SRC: Return candidates SRC-->>SA: Processed results SA-->>R: Candidate recommendations Note over SRC,SR: Async execution supported Note over API: External service integration ``` ## Production Lessons and Key Insights LinkedIn's journey offers several critical insights for organizations scaling AI agents: ### Invest in Developer Productivity The AI space moves incredibly fast. Success requires making it as easy as possible for developers to: - Adopt best practices automatically - Adapt quickly to new techniques and models - Contribute without extensive specialized knowledge Standardizing patterns and providing opinionated frameworks dramatically reduces the barrier to entry. ### Don't Forget Production Fundamentals Despite the novelty of AI agents, they're still production software systems. This means: - **Reliability**: Traditional availability and performance concerns still apply - **Observability**: Custom observability solutions are essential for agentic execution patterns - **Evaluation**: Robust testing frameworks must account for non-deterministic behavior ### Results That Speak LinkedIn's approach has proven successful at scale: - **20+ teams** actively using the framework - **30+ production services** supporting GenAI product experiences - **Days instead of weeks** for building non-trivial agent applications ## The Path Forward LinkedIn's experience demonstrates that scaling AI agents requires more than just technical infrastructure - it demands a comprehensive approach to developer experience, organizational adoption, and production operations. Their success came from recognizing that the bottleneck isn't usually computational resources, but rather the ability of teams across an organization to innovate quickly and reliably with AI agent technology. As AI agents become increasingly central to product experiences, LinkedIn's blueprint offers a proven path for organizations looking to scale beyond single-agent experiments to comprehensive agent-powered ecosystems. The key insight? Scale isn't just about handling more data or users - it's about enabling more teams to build better agents, faster. LinkedIn's platform approach shows how to achieve both dimensions of scale simultaneously. --- ## References 📺 **Watch the Full Presentation**: [Agents at Scale using LangGraph - LinkedIn](https://www.youtube.com/watch?v=NmblVxyBhi8) *This blog post is based on LinkedIn's presentation about their agent scaling journey. Their approach demonstrates how thoughtful platform engineering can accelerate AI adoption across large organizations while maintaining production quality and reliability.*

Frequently Asked Questions