Ask an AI assistant for hotel rates in Lisbon next weekend and you'll receive a confident number. Press further on where that figure came from, and the answer grows murky. The agent may have checked a booking platform moments before. Or it may have pulled a price from its training data—possibly from two summers ago—and delivered it in the same assured tone it uses when correct.

Industry observers often label this a hallucination. Yet the distinction matters: in many cases, the model retrieved a price that was genuinely accurate at some point. No mechanism in the system required it to verify whether that rate still applied.

The cost of checking creates a hidden choice

Before responding to any query, an agent faces a decision: draw from stored knowledge or venture onto the web to find and parse a live page. In developer and AI circles, this is known as the fetch decision. Users never witness it. At Decodo, nearly all the teams we partner with leave it unrecorded, despite dashboards that track query costs down to fractions of a cent.

Most agent frameworks don't require engineers to make this call explicitly. Instead, web search appears to the model as an optional tool. The model itself decides whether a question warrants the effort. This judgment proves weaker than it appears. Models lack awareness of how recent any particular fact in their weights might be. A hotel rate from 2024 carries the same undated certainty as the boiling point of water—nothing signals the model to treat one differently.

The lookup process involves more steps than most assume. An agent needs a search call to locate the relevant page, then a fetch to retrieve it. The fetch is where failures cluster: sites reject automated traffic, prices load only after JavaScript execution, or the agent encounters a cookie consent banner instead of the data it needs. Whatever content does return must be stripped down before the model processes it. Each paragraph consumes tokens and processing time. Recall costs almost nothing by comparison.

Speed and smoothness push skipping into default behavior

Product teams evaluate agents on velocity and user experience. In a review meeting, an agent that says "I couldn't load that page" appears broken, while one that cites an outdated price looks functional. Through successive release cycles, every prompt adjustment and every deadline-driven default makes the lookup slightly easier to skip. Eventually, answering from memory becomes the agent's standard mode.

Spotting what came from where

The fetch decision remains invisible to users, but its output usually leaves traces. Answers built on live pages typically include a source—a link to the page and often a timestamp of when it was checked. Answers from memory read just as smoothly, minus the link. For anything volatile, such as prices or flight times, treat an unsourced answer as a guess. Request that the assistant look it up and show its source. That single follow-up often shifts the response.

Organizations acquiring AI tools can pose the same questions before signing contracts. Ask vendors how frequently their agent checks the web before answering questions about price or availability. Also ask what users see when that check fails. Web browsing is becoming standard, and most agents can already do it—though not all can reach the web independently. For those that can't, verify whether a third-party web data service can be integrated via plugin or API, allowing the agent to search and read live pages instead.

Making verification the cheaper path

Upgrading to a newer model is typically the first remedy developers attempt, yet it delivers less than hoped. The latest models and AI tools support web browsing, yet they deliberately choose to skip it. When a model deems a search unnecessary, it answers from training. For prices, this can mean information months or years out of date.

Phrasing often determines whether a search occurs. Questions containing "today" or "latest" usually trigger a web lookup. But "What does our main competitor charge for its entry-level plan?" can still draw an answer from memory, even though that price may have shifted twice since training ended.

Four things to ask for

The solution lies in how the product is constructed around the model. When evaluating an agent product, I would request these four capabilities:

  • Track lookup frequency: measure the proportion of answers based on pages the agent actually read, segmented by topic. If prices are verified only half the time, start there.
  • Mandate checking for certain queries: prices, inventory levels, and anything legal or financial should always require a lookup. This should be hardcoded into the product so the model cannot bypass it.
  • Signal when checks fail: if the agent cannot load a page, it should inform the user it could not verify the answer. The team should monitor how often this occurs.
  • Simplify the lookup process: teams frequently combine separate services for web search and page reading. Each handoff between them introduces delay and another failure point.

Where Decodo fits

That final point is where Decodo concentrates its efforts, which is why we created the Web Data API. With a single API key, an agent can search the web, understand how a site is structured, gather pages across a domain, or extract content from a single page. It operates on an ethically-sourced IP pool, enabling it to read sites that block automated visitors. The page returns as clean text the agent can use immediately, rather than raw HTML code.

Return to the competitor pricing question. With live web access enabled, the agent navigates to the competitor's current pricing page and reads today's rate directly. The answer originates from the page itself, not from what the model learned during training. Most AI tools already support web search, and many display sources when they choose to. What remains elusive is consistency. An agent that verifies the latest price on one question yet answers from memory on the next, with no way to distinguish between them, cannot be trusted with anything involving cost.

Source: The Next Web