With iOS 15, Apple introduces new welcome features including Live Text, its character recognition system. No need for a third-party app to extract text from an image, the camera takes care of everything.
Live Text is undoubtedly one of the most useful functions introduced in iOS 15, the new version of Apple’s system for its mobile devices (read our article iOS 15 is available: all you need to know about the update). It is an OCR system (optical character recognition, in French). The principle is not new and it has existed on smartphones for years with the Google Lens app for iOS and Android and even directly in the system with Samsung mobiles for example. It allows, in a few seconds of analysis, to extract text contained in an image. Practical for, for example, copying passages from a text, a cooking recipe or contact details such as a phone number, a postal address, an email address, etc. Especially since it does not work only from a photo of a paper medium: it can be used perfectly to extract text from a sign, a table, a screen or anything other support.
In practice, it suffices to point the lens of the camera of the iPhone (or of the iPad, because the function also exists in iPadOS 15) at the text to be recovered. Artificial intelligence does the rest. It analyzes on the fly the image displayed on the screen and identifies the presence of textual elements. All that remains is to activate the character recognition function to extract the desired content. Well seen too, Live Text works a posteriori on images saved in the Photos app and therefore allows text to be extracted from old photos.
Once the character recognition has been performed, several possibilities are available to you. You can simply copy the text in order to paste it – and of course modify it – in a document, an email or a message for example, but also request a translation. Apple offers this service with nine other possible languages. In addition, if the recognized text contains a phone number, it will be interpreted as such and you can directly initiate a call. Same thing with the email addresses with which the Mail application will take over to generate a new missive. And if it’s a mailing address, the Maps app can show you its exact location on a map. Practice ! Finally, note that Live Text also works with handwriting, however, with varying degrees of success depending on its readability. She will not do miracles with the feet of flies.
No need to install an additional app to retrieve text contained in an image: with iOS 15, the iPhone Camera app takes care of everything in a rather intuitive way.
- Launch the iPhone Camera app. Then point the objective on the text you want to recover. Almost immediately, the area containing text is identified and decorated with a yellow frame. A new icon Text is also displayed at the bottom right of the image.
- Tap the icon Text. Instantly the framed text is extracted. Note that no photos were taken.
- Now press on the text. Cursors appear. Move them to the start and end of the text you want to extract.
- You can now copy the text to the iOS clipboard by pressing To copy in the menu bar displayed just above.
- You just need to paste it into any document allowing you to edit text, such as the Notes app for example.
- You can also share this content directly from the function Live Text. Press the small arrow to the right of the menu bar to scroll through the other options and choose To share.
- The usual iOS sharing options are displayed to send this content in an SMS, an Mail or any other messaging application such as WhatsApp or Messenger for example.
iOS 15 allows you to translate text from an image quite simply into nine different languages (German, English, Arabic, Chinese, Korean, Spanish, Italian, Japanese, Portuguese and Russian). The translation is not instantaneous as with Google Translate (which superimposes the translated text in real time on the image) but it is rather efficient in use. Please note, this function requires an Internet connection (Wi-Fi, 4G or 5G) to exchange data with Apple’s servers.
- Repeat the operations described above to extract the text from the camera. Once the text has been selected, Press the small arrow to the right of the menu bar to scroll through the other options and choose Translate. You must agree to the selected items being shipped to Apple’s servers for the operation.
- Immediately after, you get the translation of the text. By default it is done in French for a text written in English and vice versa.
- If you want to get a translation into another language, scroll down the contents of the window Translate down and choose Change the language.
- All the languages available for translation are displayed. Tap the one you want, then close the window by pressing the cross at the top right of the screen.
- The text is instantly translated.
- You can of course copy this translation to the iOS clipboard to paste it into a document or even listen to it by pressing the play key among the possible options at the bottom of the window.
Live Text works, as the name suggests, live from the Camera app, but also a posteriori on the images already saved in the iOS photo gallery.
- Open the Photos app, then tap an image with text that you want to recover.
- Wait a few seconds during which the image analysis will be performed. When artificial intelligence has detected text, a Text icon is displayed at the bottom right of the image.
- Tap the icon Text. All the textual elements located on the snapshot are highlighted.
- You can now proceed with the text selection as explained above and use the same functions to copy, share or translate it.
With Live Text, you can point the iPhone camera lens at a phone number, email address, website, or postal address and take action.
Make a call
- Point the camera at a phone number. Then press the Text then on the image.
- Immediately a menu unfolds. If the number matches one of the contacts in your directory, it is immediately identified. iOS 15 will offer you to dial the number, send a message, initiate a FaceTime call (audio or video) or simply copy it.
Write an email
- Point the camera at an email address. Then press the Text then on the image.
- The Mail application immediately takes over and offers to compose a new message with, for recipient, the email address previously scanned and automatically indicated in the field. TO :.
Go to a website
- Point the camera at a website address. Then press the Text then on the image.
- Safari opens instantly and displays the previously scanned website.
Display an address on a map
- Point the camera at a street address. Then press the Text then on the image.
- This time, the Maps application opens and refocuses the map on the scanned address.
Function Live Text adapts very well to typed texts. But it can also extract text from handwriting. Convenient for quickly transcribing notes … provided that the writing is legible.
- Point the camera at hand written text. Then press the Text then on the image.
- Then choose Select all from the menu bar above the text extract then the option To copy.
- Now open a text editing application like Notes for example. Create a new note then do a long press on the document. In the menu that appears, choose To stick on.
- The handwritten text scanned and copied to the clipboard takes place in the document but this time in block letters. It is totally editable. But character recognition needs to be improved a bit further.
