Recognizing Handwritten Mathematics with the On-Device Language Model

Question

Created 12h

Replies 2

Boosts 0

Participants 2

Hello,

I'm currently working on an iPad app that combines the flexibility of handwritten notes with advanced mathematical computation, including linear algebra, calculus, and solving systems of equations.

One of its core features is to recognize handwritten mathematical notation and converting it into a structured representation that can subsequently be parsed and evaluated.

For this, I want to use the SystemLanguageModel() together with Guided Generation.

My approach

I created a custom set of "MathToken" enums using @Generable. The idea is to let the model recognize mathematical handwriting and directly return an array of these tokens.

The tokens represent both basic mathematical elements (integers, identifiers, operators, etc.) and more complex structures such as matrices, vectors, function applications, derivatives, integrals, and systems of equations.

The resulting tokens are then passed to a parser, which constructs a structured statement that can eventually be evaluated.

For handwriting input, I exported the relevant content from PaperKit/PencilKit as a UIImage and attached it to the LanguageModelSession.

I've attached both the MathToken definition and my current prompting approach.

The problem

Unfortunately, the recognition accuracy is not yet sufficient for a reliable user experience.

I've experimented with different prompting strategies:

Very short and general instructions.
More detailed descriptions what to watch for
Explicit rules explaining when each token should be used

However, none of these approaches has significantly improved the results. The model frequently selects tokens that contradict the instructions it was given.

Some examples:

The model sometimes generates an identifier containing a number instead of using integer or real.
Instead of using structured tokens like functionApplication, it uses identifiers and parentheses instead
For ordinary multiplication, such as 3 × 5, it sometimes generates the product token, which is specifically intended for mathematical product notation (capital Pi).

These are not ambiguous cases. I've explicitly described the distinctions in my instructions, yet the model still makes these mistakes.

I'm aware that prompting and model context size have their limitations, so I'm trying to understand whether my overall approach is appropriate.

Questions

Prompting: Is there a better way to guide the model towards consistently selecting the correct enum cases?
Guided Generation: Is an enum with many associated values and recursively nested [MathToken] arrays a suitable structure for Guided Generation? Could the complexity or structure of my MathToken definition itself be contributing to the problem? How could a more suitable structure look like (which I could transform into the MathTokens)?
Model capabilities: Is recognizing handwritten mathematics and mapping it directly to such a structured representation simply too complex a task for the on-device model in its current form?
Architecture: Could this problem benefit form splitting it into multiple stages? What stages would you recommend?

I'd be particularly interested in understanding whether there are any known limitations or recommended patterns when using Guided Generation for tasks involving complex, recursively structured output.

I've included my current implementation and the MathToken definition below/attached for context.

MathToken.txt(1.7 KB)
MathHandwritingRecognizer.txt(1.9 KB)
Thank you for your time and any suggestions!

Answer 1 · 4h

Frameworks Engineer OP

Apple

4h

While SystemLanguageModel's vision capabilities can recognize text, where domain specific APIs exist, you're generally better off relying on them. In this case, PencilKit has optimized handwriting recognition models that are faster and more accurate than you can expect an LLM to be.

Similarly, while a smart enough LLM might be able to turn an equation string into a structured representation, classical computer science and NLP techniques are ultimately the best way to get the job done. They are many orders of magnitude faster, and are determinstic. Look into creating an Abstract Syntax Tree (AST) from your equation string -- that's effectively what your @Generable enum is.

The places where an LLM would help you the most would be with cleaning up the recognized text before attempting to parse it as an equation, and explaining issues like unbalanced parenthesis or unsupported functions to the end user.

Answer 2 · 3h

luca_bmrs OP

3h

Thank you for taking the time to share your thoughts!

I've actually experimented with PencilKit's handwriting recognition before exploring Foundation Models. Unfortunately, I found that it doesn't work particularly well for my use case, especially when dealing with more complex mathematical notation such as matrices or integrals.

Another issue is that PKStrokeRecognizer can return the same textual output for handwritten expressions with different mathematical meanings, making the result ambiguous.

For example:

A fraction with 2 as the numerator and 5 as the denominator, followed by multiplication with x.
- Intended meaning: (2/5) * x
A fraction with 2 as the numerator and 5x as the denominator.
- Intended meaning: 2/(5x)

Both are recognized as 2/5x, even though they represent two mathematically different expressions.

Regarding the AST, perhaps I should clarify my architecture: My MathToken enum is intended as an intermediate representation rather than the AST itself. I've implemented my own parser based on a formal grammar, which processes these tokens and constructs the actual AST.

So I agree that parsing and constructing the AST should be handled deterministically rather than by an LLM. The idea is to use the LLM for the recognition and generation of the structured tokens that serve as input to my parser.

I'd be curious how you would approach mathematical handwriting recognition in general, particularly for complex structures such as matrices and integrals, where preserving mathematical meaning is essential. Do you think Foundation Models is a suitable tool for this, perhaps with a different approach or multi-stage architecture, or would you consider it fundamentally the wrong tool for the job?

Thanks again for your input!