Hi everyone, I ran into a very similar problem
In my case, the failure was in Spotlight's background embedding pipeline on macOS 27.2
Literal search worked, but semantic search could not find newly indexed notes
I traced it to a saved reading position that was ahead of the journal entries it needed to process
The original post reports a timeout generating the query's embedding
My confirmed failure was skipped generation of the documents' embeddings
These are separate stages, so this may be a different cause behind similar symptoms
What happens after we index a note
My app passes a note's ID, title, text and dates to CSSearchableIndex.indexAppEntities
Spotlight accepts the data and the note becomes searchable by its actual words
Generating the note's embedding happens separately
An embedding is a numerical representation of the text's meaning that allows a query to find related content without matching the same words
Successful indexing does not mean this background work has finished
The queue is files plus a bookmark
On the Mac I investigated, spotlightknowledged.updater processes a file-based journal
Three kinds of files matter
.journal files contain numbered pages of changes to indexed records- A matching
.toc file is the table of contents that identifies the entries inside a journal page bookmarks.json stores where the worker should resume, including the journal family, page and entry
The instant, priority and backlog processing jobs have separate bookmarks within this same queue
Think of someone reading a book
They read a page, move their bookmark and resume there next time
The affected embedding queue was under
~/Library/Metadata/CoreSpotlight/DocumentProcessing/PipelineStorage/Embedding/Journals/NSFileProtectionCompleteUntilFirstUserAuthentication
What was broken
The captured state looked like this
New notes were on pages: 13878, 13879
Saved continuation page: 14558
Journal family: unchanged
The worker resumed at page 14558
Within that same journal family, the smaller page numbers were behind its bookmark, so it skipped the entries containing the new notes
The text was already in Spotlight, but those records never reached embedding generation
That is why literal lookup could succeed while semantic lookup failed
The worker could wake up normally and still find no eligible new work
What was fixed
I rebuilt the embedding worker's reading state so it could generate embeddings and save valid bookmarks
This procedure resets the one embedding queue shown above, including all its saved reading positions
Other processing queues and the rest of the Spotlight index are outside its scope
!!! Be careful, this procedure will delete the old embedding queue without a backup !!!
1 Find and stop the embedding worker
updater='^/usr/libexec/spotlightknowledged[.]updater( |$)'
pgrep -u "$EUID" -fl "$updater"
pkill -TERM -u "$EUID" -f "$updater"
pgrep -u "$EUID" -fl "$updater"
2 Find the current journal and its queue filenames
base="$HOME/Library/Metadata/CoreSpotlight"
protection=NSFileProtectionCompleteUntilFirstUserAuthentication
queue_dir="$base/DocumentProcessing/PipelineStorage/Embedding/Journals/$protection"
journal_files=("$base/$protection/index.spotlightV3"/journalAttr.<->(om[1]))
journal_file=${journal_files[1]}
cookie=$(plutil -extract progress.1.page.cookie raw -o - "$queue_dir/bookmarks.json")
native_name="evt_v12_journalAttr_${cookie}_$(stat -f %i "$journal_file")_${journal_file##*.}"
3 Replace the old queue with clean reading state
(
set -eu
(( EUID != 0 ))
[[ -d "$queue_dir" && ! -L "$queue_dir" && $(stat -f %u "$queue_dir") == $EUID ]]
[[ "$queue_dir/$native_name.journal" -ef "$journal_file" && "$queue_dir/$native_name.toc" -ef "${journal_file}_toc" ]]
fresh_dir=$(mktemp -d "$HOME/Library/Spotlight-embedding-reset.XXXXXX")
chmod "$(stat -f %Lp "$queue_dir")" "$fresh_dir"
printf '%s\n' '{"files":[],"progress":[]}' > "$fresh_dir/bookmarks.json"
ln "$journal_file" "$fresh_dir/$native_name.journal"
ln "${journal_file}_toc" "$fresh_dir/$native_name.toc"
if pgrep -u "$EUID" -f "$updater"; then exit 1; fi
rm -rf -- "$queue_dir"
mv "$fresh_dir" "$queue_dir"
printf '%s\n' 'Queue replaced'
)
4 Restart the worker and verify semantic search
Let the native worker generate embeddings and write its own correct bookmarks
launchctl kickstart "gui/$EUID/com.apple.spotlightknowledged.updater"
Embedding generation is asynchronous, so allow time for processing
Republish existing items from their source apps if you need complete library coverage
