Why Organizations Build AI That Looks Like Themselves—and the Toll It Takes on the Value of AI

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.” — Melvin Conway (1967)

When programmer and consultant Melvin Conway published this observation, he was discussing software architecture. He noticed that a four-person team building a compiler inevitably produced a four-pass compiler—not because four passes were optimal, but because each person owned one pass. The interfaces among the passes reflected the lines of communication among the people building them.

The architecture was being driven by the organizational chart, not for what was optimal.

When optimally designed, AI solutions should reflect the highest-value design for the problem at hand and outcome desired. But in practice, solutions often replicate the human organization and process they were meant to transcend, shedding value as a result. This tension extends into the core of how AI systems are scoped, trained, prompted, integrated, and deployed.

Failure to Thrive

This dilemma inhibits AI’s ability to achieve its promised potential. In pediatric medicine, “failure to thrive” describes a child who is given adequate nutrition but does not grow as they should because the child’s system does not absorb nutrients properly.

Enterprise AI finds itself in precisely this condition. Capability is abundant. Tools are available. Talent exists. But the operating environment required to convert capability into compounding business performance is missing. Individuals get faster and dashboards look busier, but the business itself barely moves. Speed without redesign converts old ambiguity into risks that come at you more quickly.

Recent studies capture both sides of the failure-to-thrive problem. Stanford’s Digital Economy Lab found that 77 percent of the most significant challenges in successful AI deployments arose from less-visible operating conditions (e.g., change management, data quality, governance, process redesign) rather than the technology itself. And while MIT Project NANDA documented the corresponding value gap, it estimated as many as 95 percent of AI pilots fail to deliver return on investment (ROI). The findings support the failure-to-thrive diagnosis: access to AI is not translating into durable economic value because most organizations have not built the operating environment needed to absorb it.

Organizations will achieve success only when they can perform workflows better under real conditions, explain what changed, preserve the learning, and decide what should scale or stop. Absorption—not just adoption—is the executive frame that turns scattered capability into operating knowledge.

Optimal Design vs. Organizational Design

Imagine what an AI solution could be if designed purely for value. A large enterprise wants an AI assistant to help its sales team. The highest-value design could be a system that draws on customer history, product catalogs, pricing models, competitive intelligence, support tickets, and market data—all unified into a single reasoning layer that can answer any question a salesperson might have, generate tailored proposals, and flag risks before they become problems.

From the user’s perspective, this is a single intelligent system. From a design perspective, it doesn’t care which department owns which data. It cares about the work to be done.

Now consider what actually is getting built:

• The sales data is owned by the sales technology team, which has access to customer relationship management (CRM) data.
• The finance team—which is not part of the project—owns the system that houses pricing data.
• Support tickets reside in a system owned by customer service, and integrating them requires a cross-functional agreement that hasn’t been signed off.
• The competitive intelligence is scattered across marketing’s wiki, which the sales technology team doesn’t even know exists.

So the AI that ships is a CRM-grounded chatbot that can summarize deal notes and draft follow-up emails. But, by not incorporating insights from a broader set of relevant data, the value potential of the system is substantively reduced.

This is Conway’s law in action. What an AI system’s architecture can see, reason about, and do copies the structure of the organization that built it. The silos in the org chart become the silos in the AI’s world model.

Instead, start with the problem to solve, design the technology to solve that problem without regard to org design, and then work to redesign the organization and processes around that solution. This paper will explore how to make that a reality.

Why AI Is Especially Vulnerable

1) AI systems model the world they are given access to

Traditional software encodes a process: step A, step B, step C. If the process is siloed, the software is siloed, but the damage is contained. You get a clunky workflow, not a fundamentally limited intelligence.

AI systems, by contrast, build a model of the world. A language model grounded in retrieval doesn’t just execute steps; it creates a representation of what is knowable and reasonable about a domain. If that representation is built from only one department’s documents, the AI’s entire understanding of the business reflects that department’s understanding. This is a widespread constraint: a 2025 IBM study found that 83 percent of chief data officers cited data silos as a barrier to innovation. For AI, those silos define the boundaries of what the system can know, further perpetuating data fragmentation. Without access to pricing data, the model doesn’t even know that pricing data exists as a relevant input or understand what questions are worth asking.

2) AI projects are being scoped by organizational boundaries

Most enterprise AI initiatives are funded and governed within a single business unit. The VP of Sales sponsors a sales AI. The VP of Support sponsors a support AI. The CIO sponsors an internal knowledge AI. Each project has its own budget, success metrics, data access agreements, and vendor relationships.

Before a single line of code is written, the scope of the AI has been narrowed to the boundaries of the sponsoring organization. The question “What would the highest-value AI solution for our sales team look like?” is rarely asked in a way that permits an answer crossing organizational lines. What gets asked instead is “What can we build with the data and authority we already have?”

3) Process replication disguises itself as “domain expertise”

When subject-matter experts are asked to define what an AI should do, they naturally describe what they do—their workflows, decision points, and heuristics. This approach carries an implicit assumption: that the current human process is the thing to be replicated, rather than the outcome the process was meant to achieve.

An underwriter describes the steps of reviewing a policy application. A claims adjuster describes the triage process. A financial analyst describes the monthly reporting cycle. The AI is then designed to automate steps that exist because of human constraints—limited attention, limited memory, limited ability to process parallel information. An AI doesn’t share those constraints. The highest-value design might collapse ten sequential steps into one parallel reasoning pass or eliminate a step entirely because the AI can access information the human had to request manually.

When the AI is designed to mirror the current process rather than the outcome, it inherits all the inefficiencies of the human workflow while adding the cost and risk of an AI layer on top.

4) Judgement needs to be relocated

AI does not remove the need for judgment; it relocates it. As execution becomes cheaper, human judgement becomes more valuable. But that judgment must move upstream into intent, framing, and system design and outward into exception handling, escalation, and ownership of outcomes.

How Organizations Break the Cycle

Conway’s law is a pre-existing design constraint. The organizations building the best AI solutions are the ones that recognize and actively work against it. Several strategies have emerged:

Design based on desired outcome

Start every AI initiative by defining the desired outcome in the user’s terms, such as “a salesperson can answer any question a prospect asks and generate a tailored proposal in minutes.” Then work backward to identify what data, reasoning, and actions the AI needs. This forces the conversation to confront organizational boundaries early, rather than unintentionally inheriting them.

Build cross-functional AI teams

The most effective AI projects are staffed across organizational lines from day one. If the sales AI needs pricing data, include a finance representative as part of the core team, not a stakeholder to be consulted. This doesn’t eliminate Conway’s law—the team still has a communication structure—but it ensures that the communication structure is designed for the AI’s needs, not inherited from the existing org chart. Include privacy and data compliance teams early so that data sharing blockers can be surfaced early and solved.

Treat data access as a critical design decision

Before architecture begins, map every data source that the highest-value design would require. Explicitly assign an owner to secure data access; this is a design constraint to surface, not a problem to discover six months into implementation. It may require contract amendments or new identity management structures that require lead time. Organizations that treat data access as a technical afterthought always build Conway-shaped AI. Organizations that treat it as a governance and design priority can build something better.

Separate “what the AI knows” from “who built it”

One of the most powerful architectural choices is to build a shared knowledge layer—a unified retrieval and grounding infrastructure—that multiple AI applications draw from. This requires building a technical substrate that is explicitly not shaped by the org chart. Each department can still own its AI applications, but they all reason over the same world model. This is the technical countermeasure to Conway’s law: you can’t always change the organization, but you can change what the AI sees.

Design for the AI’s capabilities

When defining what an AI should do, explicitly ask: “What can the AI do that the human process doesn’t exploit?” The highest-value designs are the ones that leverage capabilities humans don’t have—instant access to all documents, parallel reasoning across thousands of records, pattern detection across datasets no human could hold in working memory. If your AI design doesn’t use at least one of these, Conway’s law has likely already prevailed.

Manage to value

Transformation succeeds based on value realized. Establish OKR and KPI discipline with clarity on what “success” means for each initiative. Track at both the workflow-element and program levels. Ensure that outcomes focus on financial value that impacts the P&L. Costs must include both enterprise costs (e.g., supervision, review, exception handling, integration, audit) and technology spend. Realign KPIs and incentives to outcomes and cross-team reuse before scaling.

Address the human vs. machine asymmetry directly

There is an asymmetry between the speed at which AI moves—and continues to move ever faster, and the dramatically slower speed at which humans move and change. This asymmetry will continue to widen. This triggers core human identity issues: loss aversion around control, competence, status, and job security. The resulting perception is that value lives at the enterprise level, but risk at the individual level.

Change management must address this asymmetry as an iterative process, not a one-time rollout. AI initiatives must clarify authority before adding speed, redesign work before automating it, and involve the people who will use the new workflows in shaping them. Recent research from MIT Sloan reinforces this. Its 2024 study, conducted with participation from AI developers, business leaders, government officials, and academic experts, found that AI-enabled change is more likely to take hold when employees help shape it rather than having it imposed on them. From defining the opportunity and designing new work processes to building skills and managing the transition, employees’ knowledge of how work actually gets done is critical to effective implementation (Bringing Worker Voice Into Generative AI, 2024).

The Deeper Lesson

Conway’s law is ultimately about the relationship between structure and output. Organizations are very good at producing systems that look like themselves. This was always true for software. The stakes for AI are higher because AI systems encode understanding. An AI that mirrors your org chart will retain its blind spots. The AI doesn’t know what it doesn’t know, which is everything outside the silo that built it.

The organizations that will derive the most value from AI will recognize Conway’s law as a force acting on their AI initiatives and choose, deliberately, to design around it. Build AI that reflects the highest-value design for the problem, not the highest-fidelity replica of the organization that happened to build it.

The ultimate goal is to become an adaptive enterprise: a behavioral system that constantly adapts to its environment through feedback loops, incentives, and selection pressure, much as a biological system adapts to stay alive. The central mechanism is institutional learning, not heroic individuals. Work continuously produces insight, and insight continuously reshapes how work gets done.

Successful organizations will clarify authority before adding speed, redesign work before automating it, codify learning so the next wave starts ahead of the last, and align incentives to outcomes—not activity.