The Investigation That Changed SMAK
How a question about artificial intelligence became a lesson in human judgement.
One of the questions I am asked most often is whether artificial intelligence can be trusted.
I think it is the wrong question.
A better question is: How do we decide when an answer has earned the right to become knowledge?
The answer to that question led to one of the most valuable lessons I have learned while developing the SMAK framework.
The Investigation
It began with a simple question about the discoverability of a newly launched website. I asked an AI assistant what it knew about the site. The response sounded convincing. It suggested that the website had little public presence and implied that it was either inactive or largely undiscoverable.
Nothing about the language raised immediate suspicion. It was fluent, logical and confidently presented. Many people—including myself in the past—would probably have accepted the answer and immediately started trying to fix a problem.
Instead, I paused. Not because I believed the AI was wrong, but because I had not yet established whether the conclusion had earned the right to be believed.
Applying SMAK
Source
Where had this information actually come from? The AI explained that it was relying on information available through its own retrieval methods rather than directly experiencing the website as a human visitor would. The issue was no longer whether the answer sounded convincing; it was whether the source was sufficient to support the conclusion.
Method
How had the answer been produced? Had the AI opened the website in a browser? Had it inspected the page itself? Or was it relying on search indexes, cached information and publicly available signals? A search index is not the same as a web browser. A cached record is not the same as a live inspection. Without understanding the method, it is impossible to judge the strength of the conclusion.
Assessment
Rather than asking the AI to defend its answer, I sought independent evidence. Google’s diagnostic tools confirmed that the website was publicly accessible and technically structured as intended. The evidence suggested that the AI and Google’s diagnostic tools were working from different sources and methods, leading to different conclusions.
Knowledge
Only then could a conclusion be reached. The original concern—that the website had a serious technical discoverability problem—had not earned the right to become knowledge. It was replaced by a better-supported conclusion: the technical foundations were sound. The remaining challenge was time, authority and continued publication.
What SMAK Actually Prevented
At first glance, it might appear that the investigation simply exposed an AI mistake. I do not think that is the most important lesson.
SMAK did not change the AI’s answer. It changed my response to it.
Without a structured process, it would have been easy to redesign pages, rewrite metadata or spend days trying to solve a problem that independent evidence later showed did not exist. The framework prevented an unsupported conclusion from becoming an accepted fact.
The Framework Grew
Until this investigation, I viewed SMAK primarily as a framework for evaluating AI-generated answers. I now think its purpose is broader. It is a framework for evaluating important conclusions, regardless of where they originate: an AI response, a news article, a social media post, advice from a colleague, or even our own first impressions. Every source deserves to be questioned before it becomes knowledge.
The Principle
Technology will continue to improve. Artificial intelligence will become more capable. Search and retrieval systems will become more sophisticated. The need for human judgement will not disappear. If anything, it becomes more important.
SMAK is not designed to tell us what to think. It is designed to slow us down long enough to ask whether a conclusion has earned the right to become knowledge.
Final Reflection
When I first began developing SMAK, I believed I was creating a framework for evaluating artificial intelligence.
I now think I was creating something different.
Artificial intelligence happened to be the catalyst. The real subject was judgement.
Every generation develops tools that make answers easier to obtain. Very few develop habits that make those answers harder to accept without evidence.
SMAK is one such habit.
It does not slow technology down.
It slows us down.
Long enough to ask one final question before acting:
Has this conclusion earned the right to become knowledge?