Appraising the value of an item based on photos

I’m working on a project to appraise the dollar value of antiques. I have a FAISS database with prices and text descriptions, and just need to find the most accurate description of an image so I can run a search in the database.

Simple use case : generic brass candle holder → take image → recognizes it’s a brass candle holder → $10

For the simple use case, I just used Gemini API, and it correctly recognized what the image was.

Advanced use case : branded brass candle holder

In this case, Gemini API failed to recognize the brand, and classified it as a generic brass candle holder.

I’ve added a tool which calls Google Lens. It was able to recognize the brand, and from the database I got its real price → $100

Pro use case: Japanese vase

Google Lens (incorrectly) recognized the vase as being from the Meiji period, potentially worth up to $500. I then uploaded 5 photos to Gemini Chat (Pro), showing all sides including the bottom. Gemini discovered that the creators used “tube-lined technique” to mimic the look of traditional Meiji-period vases, and it’s a fake, probably worth $50.

So this is where I am right now. I tried both Gemini API and Google Lens, but I’m still not able to properly distinguish a Japanese antique vase from a fake. I also wonder how come Gemini Chat managed to recognize it, but Gemini API didn’t, despite using the same model.

I’m thinking that maybe I should create a loop:

1. Gemini API appraises the images
2. Google Lens appraises the “best” image (since it can only do one)
3. The two are combined as input to a 3rd agent that does quality control

The problem is, the QC agent cannot just blindly pick the “cheaper” evaluation, because then I’ll miss out on actual rare finds.

Anyone has an idea how best to create a system which gives an expert advice on the value of antiques? Is there maybe a dataset which explains what exactly to look for - eg. “if it’s a Japanese vase, check the image to see if it’s clay or metal, etc” and then I could feed that to the LLM?