With iPadOS 18, Apple added some useful iPad-specific features in the Notes app, including Smart Script and Math Notes. Both of these additions are designed to work with the Apple Pencil, and they're some of the neater new features on Apple's tablets. Apple product manager Ty Jordan and engineering manager Jenny Chen recently did an interview with Chris Lawley to discuss the work that went into the iPadOS 18 updates.
Smart Script is one of the ways that Apple is using machine learning in iPadOS 18. It refines handwritten text, making your handwriting look neater without losing the personal flare that makes it your handwriting rather than generic writing. Smart Script learns your handwriting style, which Jordan said makes it as useful as typed text.
You'll notice that as soon as you start writing words, it refines your handwriting. You'll notice that it spell checks, so if you make a spelling mistake, it'll be underlined. It can also do some really cool things with reflowing your text, so if you tap and hold your Pencil, you can move words around, and you'll see that it reflows in your paragraph so you can get some more space.
You can even paste typed text as your own handwritten text right in the middle of a paragraph, so a lot of things that we take for granted with typed text, you can now do with handwritten text.
Smart Script learns continually and while it is designed to work quickly, it will get better over time as you write more, especially for features such as paste as handwriting. Recognition does not persist from note to note, however. Apple wanted the feature to feel personal and not like a font, which is why it refines dynamically with each note.
It's very contextual. So, for example, if you were to go to a previous paragraph where you had some band handwriting, we're going to match the handwriting that was refined there and not sort of try to replicate what you've had in other notes.
As for Math Notes, Jordan said that Apple took the time to "really reimagine" the calculator feature could be designed for the iPad's interface. If you're unfamiliar with the feature, Math Notes on iPad lets you write out equations with the Apple Pencil, with the Notes app solving the equation automatically when an equals sign is added.
It sort of works like magic. It combines the natural input of a pencil with the amazing on-device machine learning models that Jenny and her teams work on. You just write math like on a piece of paper and like magic, it just gives you the answer. It's so simple and intuitive.
Math Notes is able to recognize variables and the numbers associated with them. So if you define variables like price and area in one part of the note, and then write "price x area," Math Notes is able to provide an answer using the numbers specified earlier. Jordan explains:
I'll do my best shot and go back to high school. So a variable is just a named value, right? So you probably know a lot of them from school, like X and Y, and you could declare a variable with that [...] but you can do this with any word that you want.
I've personally loved using it for natural experiences, where I might declare like drinks equal something and food equals something else, and then I can just say that plus that and divided by the number of people, and I get an answer. So it just creates this very natural way to work with your math.
Math Notes is intelligent enough to understand the difference between separate variables in a note, so if you have multiple math problems with the same variable like X, it won't get confused. You can have graphs added automatically in Math Notes if an equation can be grafted, which Chen says uses the work that Apple has previously done for the Swift Charts API.
If you're interested in more context on the new Notes features in iPadOS 18, the full video is well worth watching. Both Math Notes and Smart Script are available in the iPadOS 18 beta, which developers and public beta testers can download. The new features will launch in the fall.
