LLMOcheki launches Attribute Fidelity, scoring attribute by attribute whether AI describes your company correctly

Itera, Inc. (head office: Shinjuku-ku, Tokyo; Representative Director: Takayuki Muto) has begun offering Attribute Fidelity analysis in LLMOcheki, its tool for measuring and improving LLMO. The analysis quantifies, attribute by attribute, how accurately generative AI describes a company or its products. Consideration Set analysis, which observes the brands AI presents alongside your own, is offered at the same time.

For each item — pricing, cancellation terms, features and so on — what AI says is checked against the correct data the company has registered, and an accuracy rate is calculated. The yardstick for AI search work widens from “were you mentioned?” to “were you described correctly?”.

Background 1: being cited means nothing if you are described wrongly

As generative AI has spread, a company’s information increasingly reaches consumers as an AI answer without passing through its own site. AI search work so far has centred on measuring the volume of exposure: whether AI mentioned you, whether your site was used as a source.

But more exposure produces nothing if the content is wrong. Quote an old price and the story falls apart in the sales meeting; describe a feature you do not offer and you lose the deal or the customer later cancels. A metric for exposure volume alone cannot tell you this is happening.

Background 2: why AI keeps repeating out-of-date information

Most of the time, generative AI describes something incorrectly not because the AI is faulty but because of the information it is referring to. An article written before your site was updated is still out there; a third-party comparison article still shows the old specification; a page from before the price change remains in cache.

In other words, knowing which attribute is described wrongly, on which AI, and how, lets you identify the source that needs fixing. Attribute Fidelity is the metric for that.

Feature 1: Attribute Fidelity

Statements inside AI search engines’ answers are checked against the correct data a company registers — pricing, how to cancel, features offered, product specifications, opening hours and so on — and an accuracy rate is calculated for each attribute.

The judgement has three levels. Matching the correct data is “accurate”; correct at some point in the past but different now is “out of date”; contrary to fact is “incorrect”. Separating “out of date” from “incorrect” lets you distinguish between two different jobs: dealing with something you failed to update, and correcting misinformation.

Results are shown by engine, ordered from the attributes with the lowest accuracy. Which of your communications to fix first comes with its priority attached.

Feature 2: Consideration Set analysis

This observes the brands listed alongside yours within the same answer when AI presents your company. AI answers often take the form of several options side by side, and the brands appearing together there are, for the consumer, the real comparison set.

It is not unusual for a company you do not regard as a competitor to appear alongside you repeatedly. You can check whether your existing competitor settings match the reality of AI’s answers.

How it is measured, and what to note

The basis for judging accuracy is the correct data the company registers. It is not an objective determination of fact by a third party, but a measure of agreement with the information you hold to be correct.

Measurement runs the same question multiple times and uses statistically processed values. We publish an LLMO measurement standard v1.0, and this metric is designed in line with it.

This feature is not intended to evaluate or compare the quality of any particular generative AI service. It is a metric for understanding whether information about your own company is being conveyed correctly, and for improving how you communicate it.

Availability

Available on every LLMOcheki plan at no additional charge.

Comment from our Representative Director

“The discussion around AI search work has concentrated on how to get mentioned. In practice, though, what happens first is something like AI quoting an old price and the sales conversation falling apart. Being able to measure whether you are described correctly comes first, in that order.”

LLMOcheki https://llmocheki.com/