Use of SpotlightSearchTool() returns "Model Catalog error: Error Domain=com.apple.UnifiedAssetFramework Code=5000" , although model is available

Whelp, that's totally a bug. 🐛

You're doing everything correctly! That's not an error you should ever see normally.

Thanks for reporting! I'm filing a bug report for this, although it would definitely help if you can tell me:

• Did you update your Mac right before this error or within the past few hours before the error?

Rebooting your Mac should resolve the issue...

Thx for filing this bug!

This error already occurred during developer beta 3. After upgrading to developer beta 4 nothing changed. On beta 3 i already tried rebooting, changing general and Siri language settings, but nothing helped.

Sorry for the delay, and sorry you're still seeing this issue! We haven't been able to replicate the bug on our side unfortunately.

If you're still experiencing this issue, could you possibly file a Feedback Assistant report with a Sysdiagnose attachment? Also kindly include if you are located in the European Union or China, because that might affect this bug. That would give us the full diagnostics from your machine to figure out which lower-level system is causing the error. Then if possible, past the FB report # back in this thread so we can fast-track the report, thanks!

One thing to try as well:

1. Verify that your Siri language and general device language match exactly.

After that, there's a few more things I'd suggest to try and essentially "trick" the OS into reloading some of these model assets:

1. If you already have Siri turned OFF, turn it ON. Make sure the language matches your system language.

2. If you already have Siri turned ON, change it to a mismatched language (e.g. if your system is in US English, set Siri to Irish English) then turn Siri OFF, and finally reboot your device. This should trick the device into uninstalling some model assets. Next, change Siri to a language that matches your system and turn Siri back ON.

Once you turn Siri back on, you may need to wait for a couple minutes to an hour for all the model assets to re-install. While the Foundation Models framework and Siri are obviously different, they both use Apple Foundation Models in the OS, which is why this debugging method sometimes works.

Thx for your reply!

Unfortunately your suggested procedure did not solve the issue.

I filed a FB report under the number FB24079629.

Thank you so much for the report details! We're able to replicate your bug now and pinpoint the underlying issue. On it! 🙂

Like you said, given the issue, unfortunately my blunt "reinstall all model assets" workarounds won't work for this one. Will let you know if we're able to find a different workaround.

Update! There are actually 2 separate issues going on in your report.

First, running this on Mac produces a context size exceeded error:

import CoreSpotlight
import FoundationModels
import Playgrounds
  
#Playground {
    let tool = SpotlightSearchTool()
    let session = LanguageModelSession(tools: [tool])
    
    let response = try await session.respond(to: "What hikes have I gone on?")
    print(response.content)
}

The workaround:

Give SpotlightSearchTool a configuration that limits the scope of its search. This makes the tool results small enough to fit in the on-device model's context window:

let config = SpotlightSearchTool.Configuration(sources: [.coreSpotlight], guide: .focused())
let tool = SpotlightSearchTool(
        configuration: config
    )    

No update yet for the 2nd issue with the missing model asset on simulator, investigating.

Based on my initial testing, adding a configuration to SpotlightSearchTool is an effective workaround for the missing model asset issue on simulator as well.

Kindly let me know if the workaround doesn't work for you, thanks!

Thanks a lot for Your help! The code you suggested works so far, that the SpotlightSearchTool() is invoked with returning a message in the console like:

"📍 NativeSpotlightSearchTool: Completed with 82 total results, returning 15 items"

However in some instances for reasons yet unclear to me, the execution stops here, not finishing the response still returning the error:

"Playground has encountered an error

Provided X.XXX tokens, but the maximum allowed is 4.096."

, with the number X depending on the prompt.

Also the workaround yet works only with the "guide: .focused())" option, not with ".complete" or ".none" .

Yay progress!

So I think it makes sense to keep using .focused() in this case, since that will help the model stay in context.

Since you're getting 15 items back then running out of context, the next thing I'd suggest trying is add the maximumResponseSize argument to your SpotlightSearchTool.Configuration.

Try dialing down maximumResponseSize to 10, then 5, then lower, to figure experimentally out how many responses your session can process without running out of context.

Alternatively, if you want all 15 responses, try the process shown in this article https://developer.apple.com/documentation/corespotlight/making-your-indexed-content-available-to-foundation-models to directly consume the output of the SpotlightSearchTool. Save the results somewhere, and then if your session runs out of context, simply start a new session that gives the model the results.