Title: Andrew Ng: State of AI Agents Resource URL: https://www.youtube.com/watch?v=4pYzYmSdSH4 Publication Date: 2025-05-28 Format Type: Video Reading Time: 27 minutes Contributors: Andrew Ng;Harrison Chase; Source: LangChain (YouTube) Keywords: [Artificial Intelligence, AI Startups, Agentic System Design, Voice AI Applications, AI-Assisted Coding] Job Profiles: Academic/Researcher;Machine Learning Engineer;Artificial Intelligence Engineer;Data Analyst;Chief Technology Officer (CTO); Synopsis: In this video, LangChain CEO Harrison Chanse and DeepLearning.AI founder Andrew Ng shares insights on building agentic systems, voice-based AI applications, and startup execution speed as key success drivers. Takeaways: [The debate over what qualifies as an “agent” is less productive than discussing how agentic a system is across a spectrum of autonomy., Most business processes involve simple, mostly linear workflows that can be partially agentic, offering low-hanging opportunities for automation., Voice-based interfaces are underutilized but valuable, reducing user friction and improving interaction by encouraging spontaneous input., Evaluation systems (evals) are crucial for improving agent performance but are often delayed due to perceived complexity. Start with fast, simple ones., Speed and technical depth are the strongest predictors of startup success. Companies that understand tooling and iterate quickly can gain disproportionate advantage.] Summary: Andrew Ng advocates for framing AI applications not in binary terms of “agent” or “not an agent” but as lying on a continuum of “agenticness,” allowing for varying levels of autonomy based on application needs. This shift reduces unproductive debates and encourages flexible, task-appropriate design. Many high-value business applications remain simple and linear—comprising step-by-step workflows with occasional branches—suggesting rich ground for partial automation with agentic systems. Ng emphasizes the need for practical skills among developers: breaking down workflows, applying appropriate levels of autonomy, and using systematic evaluation (evals) to identify and refine weak components. He notes that many teams underutilize evals due to misconceptions about their complexity. Instead, quick, imperfect evals can be iterated on like any other code to great effect. On voice interfaces, Ng explains that speaking lowers user friction compared to typing and better supports natural, iterative thinking. He identifies latency as the biggest technical challenge in voice-based applications and shares strategies such as “pre-response” prompts and ambient noise to manage user expectations. Despite strong enterprise demand, developer attention to voice remains disproportionately low. Ng also discusses Multi-Agent Communication Protocol (MCP) as a key development in reducing plumbing work and simplifying data and tool integration for AI systems. He acknowledges that Agent-to-Agent protocols are still in a very early stage and not yet widely viable. In terms of coding, Ng critiques the term “vibe coding” for trivializing what he views as a deeply intellectual process enabled by AI tools. He encourages everyone, even non-engineers, to learn basic coding to improve their ability to instruct AI systems effectively. AI-assisted coding doesn’t reduce the need for programming skills—it raises expectations for precise communication with machines. Ng concludes by offering advice to prospective startup founders: execution speed and technical insight are the two greatest determinants of success. While business and marketing skills are important, technical fluency is rarer and more decisive in the fast-evolving AI landscape. Content: ## Introduction ### Fireside Chat Format We are about to embark on a fireside chat with our guest instructor, whose courses on an online learning platform have guided many participants in deep learning. Although the audience may already be familiar with this speaker, it is worth acknowledging their pivotal role in the creation of an early educational offering on agentic systems, which helped spark widespread interest. ## The Agentic Paradigm ### From Binary to Spectrum Around a year and a half ago, debates about whether an application qualified as a true “agent” dominated discussions in our field. We argued then that this binary framing was unhelpful and that it would be more productive to view systems in terms of degrees of *agenticness*—the extent to which they exhibit autonomous behavior. By treating every system as an agentic workflow with varying levels of autonomy, the community avoided endless semantic disputes and focused instead on building useful solutions. ## Business Applications of Agentic Systems ### Linear versus Complex Workflows In many corporate settings, opportunities exist for automating relatively linear processes—such as form review, web searches for compliance checks, and data entry—or workflows that involve only occasional decision branches. These tasks often consist of a fixed sequence of steps punctuated by simple conditionals (for instance, rejecting an application if certain criteria are not met). More sophisticated use cases involve intricate loops and multi-stage decision trees, but the sheer volume of straight-through processes still presents significant business value. ### Identifying Granularity and Evaluation Points One major challenge for organizations is determining the appropriate task granularity: how to break down a process into micro-tasks, which modules to refine first when performance lags, and how to implement robust evaluation mechanisms. Many teams postpone systematic, automated evaluations—relying instead on manual review—so they lack clarity about which module or prompt is the primary obstacle. Building the instinct to interleave human judgment with targeted automated tests remains a rare and valuable skill. ## Core Competencies for Agent Builders ### Task Decomposition and Product Framing Effective agent developers must learn to analyze everyday workflows—often performed by compliance, legal, or human resources personnel—and translate them into a series of discrete, programmable steps. This involves integrating data from multiple sources, crafting prompts for each stage, and constructing a feedback loop that flags regressions and tracks improvements over time. ### The ‘Lego Bricks’ Analogy for AI Tooling Contemporary AI tooling can be likened to a set of multi-shaped, multi-colored bricks. While having only one kind of brick limits what can be assembled, a diverse toolkit—comprising workflow orchestrators, memory modules, evaluators, and guardrails—enables rapid prototyping and complex system design. Teams that master a wide range of these components achieve faster assembly of robust solutions and make better-informed decisions about which module to draw on next. ### Evaluation Strategies: Start Small, Iterate Quickly Many practitioners view evaluation frameworks as large, daunting undertakings. In practice, it is more effective to create quick, lightweight tests targeting specific failure modes. For example, if a particular prompt suddenly begins yielding incorrect outputs, a simple automated test suite of a few examples can detect the regression and free developers from manually checking every response. Over successive refinements, these tests become more sophisticated and reliable, mirroring how the underlying application evolves. ## Underrated AI Technologies ### Voice-Enabled Agents Voice interfaces are gaining traction, especially in large enterprises seeking more natural, conversational modes of interaction. By allowing users to speak freely rather than compose text prompts, voice agents reduce friction and leverage the forward momentum of speech. However, they also introduce stringent latency requirements—sub-one-second response times—necessitating techniques such as pre-responses and ambient background noise to mask processing delays. ### AI-Assisted Coding AI-driven coding assistants can dramatically accelerate development, yet some organizations still restrict their use. Empirical metrics show that teams employing these tools write and debug code far more quickly. Encouraging all employees to adopt AI-assisted coding—regardless of their formal role—can unlock productivity gains, from front-office functions to finance and legal. ### Universal Literacy in Coding As AI makes programming more accessible, we are witnessing a resurgence in the number of individuals learning to code. This trend parallels past transitions—from punch cards to interactive terminals, and from assembly language to higher-level languages—where ease of use led to broader adoption. Familiarity with at least one programming language remains crucial for articulating precise instructions to AI systems and troubleshooting generated code. ## Voice Interface Design Considerations ### Reducing User Friction While text prompts can intimidate some users, voice input feels more natural and less prone to hesitation. Users can speak, revise, or retract statements on the fly, and modern models manage such conversational dynamics effectively. By collecting insights through speech, developers can guide the system toward more relevant responses without demanding perfect, written prompts. ### Latency Mitigation Techniques To overcome the perception of long processing times, developers employ strategies like simulated “thinking” utterances (e.g., “Let me consider that…”) and subtle background audio reminiscent of a customer-service environment. Such techniques maintain conversational flow and improve the overall user experience, even when back-end operations take several seconds. ## Metadata Context Protocol (MCP) ### Standardizing Data Connectivity The Metadata Context Protocol addresses a fundamental need: simplifying how AI agents access diverse data sources and external services. Instead of custom plumbing for each model–data combination, MCP offers a unified interface that supports scalable integration of *n* models with *m* resources using an *n + m* rather than *n × m* approach. ### Early Adoption and Growing Pains Although the protocol itself is a solid foundation, existing implementations often exhibit discoverability and authentication challenges. Future iterations will likely introduce hierarchical discovery mechanisms—so agents can navigate large sets of available connectors more efficiently—while refining security and token-management workflows. ## Multi-Agent Architectures ### Current Status Although multi-agent systems hold promise for complex problem-solving, real-world examples of independently developed agent suites interacting seamlessly remain scarce. Most successes to date involve agents orchestrated by a single team under shared design assumptions. Enabling cross-team collaboration between heterogeneous agent collections will require further standardization and maturity. ## The Reality of AI-Assisted Development ### Rethinking “Vibe Coding” The term *vibe coding* suggests an effortless, intuition-led style of programming, but the reality is that AI-assisted development still demands deep cognitive effort and domain knowledge. Effective use of coding assistants requires constant critical evaluation of generated code, debugging skill, and an understanding of architectural best practices. While the name may be misleading, the practice itself represents a transformative advancement in software engineering. ### The Indispensability of Computational Literacy Even when AI generates code, human developers must interpret error messages, assess edge cases, and refine logic. Therefore, learning the fundamentals of at least one programming language—such as Python—remains a strategic imperative, enabling more precise communication with AI tools and stronger trust in the resulting systems. ## Building AI-Driven Startups ### Velocity and Technical Depth Speed is the single most important predictor of success in AI ventures. Skilled teams can iterate and deliver working prototypes at a pace unmatched by traditional enterprises. Equally critical is deep technical expertise: while go-to-market tactics are well documented and widespread, true mastery of rapidly evolving AI technologies remains rare. ### Balancing Technical and Business Acumen Founding teams must pair their engineering prowess with robust marketing, sales, and pricing strategies. Although these business functions are essential for scaling, they build on established frameworks. In contrast, technical innovation continues to push the frontier, making it vital to secure talent that can navigate both the art and science of AI development. --- Thank you for joining this discussion on agentic systems, AI tooling, and the practice of building tomorrow’s intelligent workflows.