I have spent most of my career watching technology change how people work. Some changes were incremental. Others eliminated entire categories of work almost overnight. In nearly every case, the interesting part wasn’t the technology itself. It was what happened to the people, processes, decisions, and assumptions around it.
Artificial intelligence is no different, although the speed and reach certainly are.
I use AI regularly. I experiment with it, build workflows around it, test different models, and look for places where it can remove friction or help me approach a problem differently. I think the technology is extraordinary. Used well, it can compress hours of work into minutes, expose relationships buried in information, challenge assumptions, and give an individual access to capabilities that once required several different specialists.
That kind of leverage is difficult to ignore. It is also exactly why I think we need to understand what happens when the person using the tool begins depending on it for more than leverage.
There is a point where AI stops helping someone think and starts thinking in place of them. The transition can be surprisingly difficult to see because productivity may actually increase. More documents get produced. Analysis appears faster. Emails become more polished. Presentations acquire summaries, recommendations, and carefully organized bullet points. From the outside, it can look like capability increased because output increased.
I’m not convinced those are always the same thing.
The Problem With Output That Looks Finished
Traditional technology often tells you when something has failed. Bad code throws an error. A network path stops passing traffic. An authentication system denies access. A mechanical component starts making a noise that strongly suggests your afternoon plans have changed.
Generative AI introduced something different into everyday knowledge work: failure that can look remarkably polished.
An AI system can produce an answer that is grammatically clean, logically structured, professionally written, and functionally useless. It can misunderstand the problem while describing its misunderstanding beautifully. It can provide technically accurate information that has almost nothing to do with the decision that actually needs to be made. In some ways, that is harder to detect than an obvious failure because nothing looks broken.
Fluency and competence are not the same thing, but modern AI is good enough at the first that it can create the appearance of the second.
When I use AI in an area I understand well, I have a reference point. I can recognize when an answer doesn’t fit the environment. I can see an assumption that shouldn’t be there, notice an important dependency that has been ignored, or recognize that the answer is technically correct but operationally useless. I know enough about the subject to push back.
Sometimes that means the AI was wrong. Other times it means I was wrong, which is considerably more valuable. I can ask it to challenge my reasoning, argue another position, identify missing evidence, find weaknesses in an approach, or explain why a conclusion might fail. Then I can take those results back through my own experience and judgment.
That’s an extremely powerful working relationship because neither side of the interaction has to be treated as infallible.
The trouble starts when I don’t know enough to recognize the difference between an impressive answer and a useful one. If I lack the underlying knowledge, I have fewer ways to evaluate what comes back. The terminology may sound right, the recommendations may seem reasonable, and the structure may create confidence that the substance hasn’t earned.
That doesn’t mean people should only use AI in fields where they are already experts. That would throw away one of its greatest benefits. AI can be a remarkable learning tool precisely because it allows us to explore things outside our existing knowledge. But using AI to learn about something you don’t understand is very different from using AI to make decisions you don’t understand.
That distinction matters.
Assistance and Substitution Are Not the Same Thing
I keep coming back to two very different relationships with these tools. In one, I am asking AI to help me think. In the other, I am asking it to think for me. Both can end with a document on the screen, so the difference isn’t always obvious if we measure only the output. It becomes much clearer when we look at what happens to the person using the technology over time.
When AI helps me think, I still own the problem. I bring context into the conversation, challenge what comes back, verify important claims, and reject recommendations that don’t fit the environment. I can ask why a conclusion was reached or what assumptions would have to be true for it to work. Sometimes the process reveals that my original question was poorly framed and I need to start over.
The tool accelerates the process, but I remain responsible for understanding it.
When AI starts thinking for me, the objective changes. Understanding the problem becomes less important than obtaining an answer. That is an attractive trade because answers have suddenly become incredibly cheap, while understanding remains expensive. Understanding still requires context, experience, curiosity, skepticism, and enough humility to recognize that the first answer might not be the right one.
In a real organization, that understanding includes things that may never appear in a prompt. There are dependencies between systems, historical decisions, regulatory requirements, budget constraints, personalities, competing priorities, technical debt, institutional knowledge, and dozens of other factors that affect whether an otherwise reasonable recommendation will actually work. Anyone who has spent enough time solving problems in complex environments eventually learns that the documented system and the system that actually exists are rarely identical.
AI can help us reason through that complexity, but only if we give it enough context and retain enough judgment to evaluate what comes back. It cannot relieve us of responsibility for understanding the environment in which the decision will be made.
A Human in the Loop Isn’t Enough
We hear a lot about keeping a human in the loop when AI is involved in important decisions. That’s sensible, but I think the phrase can give us more comfort than it should. Having a human somewhere in the workflow does not guarantee that human judgment is actually being applied.
If an AI system generates a recommendation and a person accepts it without understanding the reasoning, the organization technically had a human in the loop. If someone copies an AI-generated analysis into a presentation, changes a few words, and presents it to leadership, there was human involvement. If a decision moves forward because the output sounded reasonable and nobody in the room knew enough to challenge it, there were probably several humans in the loop.
The better question is what those humans added.
The value of keeping a person involved isn’t the biological presence of a person somewhere between input and output. The value comes from context, experience, domain knowledge, skepticism, institutional memory, accountability, and judgment. Without those things, the human can become little more than an approval mechanism. We preserved human involvement while quietly removing most of the reasons we wanted it in the first place.
This becomes particularly important in leadership because decisions rarely exist in isolation. A technically reasonable recommendation can still be the wrong decision because of timing, resources, dependencies, organizational capacity, regulatory requirements, or risks that were never represented in the information given to the AI. A model can only reason from the problem and context available to it. Leadership is still responsible for understanding the problem the organization actually has.
That responsibility can’t be delegated simply because the resulting document looks convincing.
The Feedback Loop Can Run Both Ways
This is where AI becomes especially interesting to me from a systems perspective. Useful systems depend on feedback. We make a decision, observe what happens, learn from the result, and carry that information into the next decision. Over time, those individual cycles can produce something much larger than any single improvement.
Jim Collins’ flywheel concept has influenced the way I think about this. I believe strongly in building momentum through accumulated progress, both professionally and personally. Make the best decision you can with the information available, pay attention to what happens, keep what worked, correct what didn’t, and begin the next turn with more knowledge than you had before. It isn’t dramatic, but neither is most meaningful progress.
AI can make that flywheel spin much faster. I can start with something I know, use AI to examine it from several directions, discover something I missed, investigate that discovery, test the result, and finish the process knowing more than I did when I began. The next time I approach the problem, I bring that additional knowledge with me. My capability increases along with the capability of the tool.
That’s leverage.
But feedback loops don’t have a preference for improvement. They reinforce whatever the system allows them to reinforce, and the same basic mechanism can operate in the opposite direction.
Someone begins with limited knowledge of a subject and asks AI for an answer. The answer is plausible, but the person doesn’t have enough domain knowledge to evaluate it deeply, so the result is accepted. The next time, relying on AI feels natural and a little less independent analysis takes place. Fewer assumptions get challenged, less knowledge gets developed, and the tool becomes a larger part of the decision process. As dependence grows, the person’s ability to recognize weak output can decline with it.
The uncomfortable part is that the person may produce more work during this process. Reports get completed faster. Responses appear sooner. More material moves through the organization. If we are measuring output alone, the system may look like a success even while the relationship between the human and the technology is becoming less healthy.
Output can increase while independent capability decreases.
Eventually the human may be providing weaker questions, weaker context, and weaker evaluation to a system they increasingly depend upon for answers. The resulting output becomes less useful, which encourages even more reliance on the tool to explain, rewrite, expand, and repair it. Nothing in that loop guarantees that either the human or the decision is getting better.
The technology didn’t necessarily fail. The feedback loop did.
Competence Debt
Technology people understand technical debt. Sometimes we take a shortcut because delivering something today is more valuable than building the perfect solution. There is nothing inherently wrong with that decision. The problem comes when temporary compromises accumulate until yesterday’s shortcuts begin determining tomorrow’s options.
I think AI creates the possibility of something similar: competence debt.
Every useful tool allows us to exchange effort for leverage. That’s why we build tools in the first place. I have no interest in manually doing something a machine can reliably do better, faster, and cheaper simply to prove that I still can. Independence doesn’t require deliberately making life difficult, and efficiency isn’t evidence of intellectual laziness.
Outsourcing work and outsourcing understanding, however, are different transactions.
If AI creates a first draft that I then evaluate and develop, I gained leverage. If it summarizes a large amount of information so I can identify where deeper analysis is needed, I gained leverage. If it gives me alternatives I hadn’t considered or forces me to defend an assumption I had been treating as fact, I gained leverage. In each case, the technology reduces friction while I remain engaged with the substance of the work.
The trade changes when I stop understanding the work because the tool can produce something that looks like the work. At that point I am no longer simply saving effort. I am borrowing capability from the tool.
Like technical debt, competence debt may be a completely rational trade in the short term. Nobody needs to maintain every skill at expert level, and organizations have always depended on specialized tools and outside expertise. The bill arrives when we need to evaluate something the tool produced and discover that we no longer have enough independent knowledge to do it.
There is an even more difficult version of that problem. Someone who knows they don’t understand something can compensate for the gap. They can research it, find expertise, ask better questions, slow the decision down, or acknowledge the uncertainty. Someone who mistakes polished output for understanding may not realize there is a gap to compensate for.
That is a much harder risk to manage because confidence can increase at the same time competence is declining.
The Test I Care About
I don’t think the most useful measure of AI adoption is how many employees are using it or how many hours an organization believes it has saved. Those numbers can tell us something about adoption and efficiency, but they tell us very little about what is happening to the capability of the people using the technology. An organization could produce more work in less time and still be losing something important in the process.
What I want to know is whether people become better at their work after using AI for a year. I would expect someone using a powerful knowledge tool every day to develop a better understanding of the problems they solve, ask more sophisticated questions, recognize weak assumptions sooner, and become more effective at evaluating possible solutions. Their experience with AI should add to their own experience, not replace it. If that is happening, then the organization isn’t merely getting more output from the same people. It is developing people who are more capable than they were before.
That distinction matters because productivity can hide declining competence for a surprisingly long time. Someone who once spent two hours developing an analysis might produce one in twenty minutes with AI. On a dashboard, that looks fantastic. If the person can still explain the analysis, defend its assumptions, identify its limitations, and recognize when it doesn’t apply, then the organization has probably gained real leverage. The tool removed effort without removing understanding.
The picture looks very different if the person becomes increasingly unable to produce or evaluate the work without the tool. The same productivity numbers might still look impressive, but the organization has exchanged internal capability for external dependency. That may even be a reasonable trade in some circumstances. We make that trade with technology constantly. What concerns me is making it without realizing that we made it.
There is also an organizational risk hiding inside that dependency. When enough people stop understanding how an answer was developed, accountability begins to blur. A recommendation moves through a meeting because the analysis looked convincing. A strategy makes its way into a presentation because the supporting language sounded reasonable. Eventually someone asks why a particular decision was made, and the explanation becomes some variation of, “That’s what the analysis showed.”
But analysis doesn’t make decisions. People do.
The standard I would rather use is whether AI leaves the organization with more accumulated knowledge and better judgment. When the technology is unavailable, when the situation falls outside the normal pattern, or when the AI produces an answer that conflicts with experience, there still needs to be someone in the room who understands the problem well enough to recognize that something doesn’t fit.
That is where I see the real value of the flywheel. Each interaction with the technology should leave something behind in the person using it. Maybe that’s new knowledge, a better question, a corrected assumption, or a faster way to recognize a familiar pattern. Whatever it is, some of the value needs to remain with the human after the window closes.
If all of the accumulated capability remains inside the tool, we haven’t built the flywheel I want. We’ve built dependence.
It Learned From You
None of this makes me less interested in AI. It makes me more interested in using it well.
I don’t want AI to preserve the way I already think. I want it to challenge me. I want it to find the assumption I missed, expose the weakness in an argument, automate work that doesn’t require my judgment, and give me more time for the work that does. If I am wrong, I want the interaction to help me discover that. If the AI is wrong, I want to understand the subject well enough to recognize it.
Most of all, I want to leave the interaction more capable than when I entered it. That seems like a reasonable standard for any tool, particularly one powerful enough to become part of how we think and make decisions.
So when someone tells me, “We’re using AI to make better decisions,” I’m interested in much more than whether AI was involved. I want to understand the system around it. I want to know whether the people using it understand the problem, whether they can challenge the output, whether important conclusions are being verified, and whether the organization is learning from what happens after the decision is made. I also want to know what happens when experience and the machine disagree, because that moment tells us a great deal about whether we built a decision-support system or simply found a more sophisticated way to outsource judgment.
Putting a human in the loop isn’t enough. The human has to add something to the loop.
Someone in the meeting says:
“We’re using AI to make better decisions.”
The presentation promises more efficiency, better insights, faster decisions, and a stronger future. Somewhere at the table, /null is holding a coffee mug and listening.
“It learned from you?”
That’s either a compliment or a warning.
The system will tell us which.
