With the release of iPadOS 18, Apple has made sure users get what they’ve wanted for years – a Calculator app for iPad. However, Apple simply didn’t want to offer a standard standalone calculator app because they wanted it to be the best app in its category on every device, and from what we’ve seen, the company has definitely achieved its goal. Apple executives sat down for an interview where the company delved into the math notes and smart scripts in iPadOS 18, describing what went behind the scenes to create the technology.
Apple executives talk about the Calculator app, Math Notes and more in a new interview, detailing the behind-the-scenes processes
In an interview with Chris Lawley, Apple executives detail how the new Math Notes and Smart Script features work behind the scenes. If you’re not familiar, the new Math Notes feature allows users to write math notations that the app can solve in an instant as it recognizes your handwriting. Apple’s system experience product manager, Ty Jordan, explained why the company decided to bring the Calculator app to the iPad and how the Math Notes feature works.
We think very deeply when we bring these types of experiences to the iPad about how these experiences can be designed for the unique capabilities of the iPad and its unique accessories like the Apple pencil.
We could have just brought a basic or scientific calculator to the iPad, but instead we took the time to really rethink how you can even do math with math notes and a calculator. Somehow it works like magic.
It combines natural pencil input with the device’s amazing machine learning models that [Jenny Chen, Apple’s engineering manager for input experiences] and its teams work on the strength of Apple Silicon. So you just write math like on a piece of paper and like magic, it just gives you the answer.
Apple’s engineering manager, Jenny Chen, also shared details about input experiences, suggesting how you can insert a graphic with one tap and how Math Notes recognizes it.
One of the great things in the first place is, how do we even recognize that this equation can even be graphed? So if we have two variables that are undefined, we first use the amazing recognition mechanism under the hood to say, “Hey, can we graph this?”
And then for the charts, it actually builds on all the years of work that we’ve done on the Swift charts API and what it provides. It’s great that we’re both improving it, but also using it in our own products. So some of the fun things that came out of that are the fun animations that make it feel nice and smooth and fantastic.
The ability to sort of scrub the graph so you can see the value as the value changes along the X-axis.
Smart Script, on the other hand, is a different feature compared to Math Notes as it aims to improve your handwriting as you take notes. The feature does this by straightening and smoothing your handwriting when you write with your Apple Pencil and has the ability to edit the text. Here’s how Jordan explains it:
Smart Script is an on-device machine learning model that learns continuously but also works very quickly. So you’ll actually notice that even if you start with a brand new iPad, as soon as you start typing words, we’ll refine them. We know how to make it neater for you.
We’re also going to make the spell check work right away because we can see what you’ve written, like Jenny said, understand it, and we can give you those suggestions.
There are some features like “paste as manuscript” that get better as you write more. So really liking the dial in your handwriting style takes one to two paragraphs of that style. But it’s going to be constantly learning and it’s very contextual.
So, for example, if you go to a previous paragraph where you had bad handwriting, we’ll match the handwriting that was refined there, and not try to copy what you had in other notes.
iPadOS 18 is a major update that brings a host of new features and changes to all iPad models compatible with the platform, and the update will be available in the fall, potentially alongside the new iPhone 16 lineup. However, the company has reserved some features just for ” Pro” iPad models that include Apple Intelligence features, but they won’t be available at launch as the company will release them at a later date. We’ll be sharing more details and guides on how to use the feature soon, so be sure to stay tuned.
