On-device AI is the next big differentiator between flagship smartphones.
Cameras that leave top-of-the-range DSLRs redundant? Check. Refresh rates that rival Playstation and Xbox? Check. Screen brightness that makes them usable from the surface of the sun? Check (ish). Zoom capabilities that are verging on creepy? Well, you get the idea.
These days, flagship smartphones are a far cry from the laggy, cumbersome handsets we’d use to play Snake and squeeze messages into the 140-character count. Whether you’re team Android, Apple or Pixel, you’ll find your argument for one side or the other becoming more and more tenuous.
So, as the differentiating factors between modern-day smartphones get increasingly trickier to define, what’s the next big thing for these powerhouse devices?
Pixel 8 Pro is taking the lead with on-device AI.
With ChatGPT and Bard becoming widely adopted, the Pixel 8 Pro segwayed masterfully from the camera and customisation battle between the latest iPhone and Galaxy models into the relatively unchartered realm of smartphone AI.
Harnessing Google’s inimitable AI research and development, the Pixel 8 Pro’s on-device AI functionality is currently unmatched by its competitors. Some of these features include:
- Magic Eraser: Magic Eraser uses AI to remove unwanted objects and people from your photos. It’s a lot like the Content-Aware Fill tool in Photoshop but even more powerful and easy to use.
- Magic Editor: Beyond removing typical photo-ruiners like red-eye and blemishes and adjusting the usual aesthetics like colour and contrast – Magic Editor can make mindblowing changes to your pictures. From altering the appearance of the sky (goodbye grey, hello golden hour) to repositioning and resizing the subject matter, Magic Editor is the driving force behind the ‘what-is-a-photo’ apocalypse.
- Pro Camera controls: The Pixel 8 Pro’s camera app now has a new Pro Camera mode that gives you more control over your manual settings, which includes the ability to adjust the shutter speed, ISO, white balance and focus peaking. There’s never been a better reason to leave your bulky DSLR at home.
- Audio Magic Eraser: Audio Magic Eraser uses AI to remove unwanted sounds from your videos – sounds like traffic, wind, and even your own voice. This one’s got us thinking about the movie Click and how long it’ll be before we can mute people in real time!
- Summarisation capabilities: The Pixel 8 Pro’s Record app now has summarisation capabilities that use AI to generate a summary of your recordings, which can be helpful when trying to find a specific part of a recording or when you want to share a recording with someone else.
- Bard integration: The Pixel 8 Pro’s Google Assistant now uses Bard (the large language model (LLM) from Google AI) to provide more natural and engaging conversations. Bard can answer questions comprehensively and informally, even if they’re open-ended, challenging, or somewhat bizarre. Our favourite strange question asked to Bard: “What is the best way to cook a pet rock?” The mind boggles…
So what’s next for Pixel?
Well, some sources suggest that Pixie – an AI-assistant powered by Gemini – could be set to launch with the Pixel 9 and Pixel 9 Pro. Harnessing data from Google products like Gmail and Google Maps, Pixie aims to perform “complex and multimodal tasks, such as suggesting directions to the closest store where someone can buy a product they have photographed”, which could see it becoming a far more personalised version of the current Google Assistant.
The Pixel team has said that the “broader family of Gemini models will unlock new capabilities for the Assistant with Bard experience early next year on Pixel.”
The new era of Galaxy AI.
Perhaps controversially and contrary to Apple fanatics’ belief, Samsung takes the crown this year when it comes to flagship victors. The S23 Ultra managed to confidently hold its own since its release in February 2023 against an onslaught of powerhouse releases from the likes of Huawei, Apple, OnePlus and the new Nothing Phone – with many reviewers still claiming it the conqueror compared to the latest iPhone 15 Pro Max.
But with the photography, performance and personalisation margins becoming dramatically – well, marginal – Samsung have recently announced its foray into the on-device AI arena. Promising an experience powered by Samsung, as well as open collaboration with industry leaders, Samsung has revealed its first Galaxy AI feature to be AI Live Translate Call, which gives users a personal translator whenever they need it.
Integrated into the native call feature, AI Live Translate Call scraps the need for third-party apps. Audio and text translations will appear in real-time during the conversation, turning multi-lingual calls into the equivalent of closed captions for show streaming.
The full force of Galaxy’s on-device AI functionality is yet to be determined. But, as we know from Samsung, the brand will continue to leverage its reputation for security and privacy in a subtle yet unavoidable snub against Google. But let’s face it: as the uptake of useful AI increases, people’s fear of hyper-personalisation and ‘they’re listening’ is becoming more of an afterthought.
Wonjoon Choi, EVP and Head of R&D, Mobile eXperience Business, claims, “Galaxy AI is our most comprehensive intelligence offering to date, and it will change how we think about our phones forever.”
How is Apple competing in the on-device race?
According to recent rumours, Apple is currently using LLM to completely revamp Siri into the ultimate virtual assistant and preparing to develop it into Apple’s most powerful AI app.
Speculation has it that ‘Siri 2.0’ (as it’s being dubbed) could be unveiled as early as the Apple Worldwide Developers Conference next year (WWDC 2024), with plans for it to be made standard on the iPhone 16 models and beyond.
Despite the lack of any official statement from Apple about its exciting new projects, its notable large investments in generative AI and machine learning and eye-watering spends on building new learning models make the smartphone trailblazer’s AI developments feel like something we’ll get our twitchy fingers on in the not-too-distant future.
Like Samsung, Apple is focusing on on-device AI development while also considering combining AI services through the cloud. The company is also said to be investing research and development resources into AI safety and ethics to make sure that its AI products and services are used responsibly and ethically.
Some rumours of Apple’s under-wrap generative AI, machine learning, and LLM model developments include:
- Natural language processing: Apple is using NLP to improve Siri (queue the collective sigh of relief) and to develop new features for its iMessage app. The company is also reportedly working on a new NLP-powered tool that would allow users to generate different creative text formats of text content, like poems, code, scripts, musical pieces, emails and letters.
- Computer vision: Apple is said to be using computer vision to improve its camera performance, develop new features for its Photos app, and power its augmented reality experiences. The company is also reportedly working on a new computer vision-powered tool that would allow users to remove or add objects in photos and videos, similar to that of Pixel’s Magic Editor.
- Machine learning: Apple is using ML to personalise its products and services, improve its apps’ performance, and develop new features. We can expect personalised recommendations in the App Store and improved accuracy of its voice recognition technology.
- Large language models: Apple is reportedly working on several LLMs, including one that is said to be more powerful than OpenAI’s GPT-3. These LLMs could be used to power new features for Siri, develop new creative tools, and improve the performance of Apple’s translation services.
The flagships of the future.
As smartphone features have reached peak evolution, plateauing hardware specifications combined with a decline in customer spending have contributed to a global plummet in smartphone sales.
In China, smartphone shipments fell 11% year-over-year in the first half of 2023 – the worst performance for the Chinese market in a decade – according to IDC. Meanwhile, in North America, smartphone shipments fell 24% year-over-year in the second quarter of 2023, driven by a decrease in both Apple and Android smartphone sales, according to Counterpoint Research.
It’s no surprise, then, that the tech titans are turning to on-device AI as their next key differentiator. And unlike ChatGPT, on-device AI should offer significant privacy benefits compared to their cloud-based counterparts. In theory, that means that we should be able to fuse the sensitive data stored on our smartphones with the key benefits of LLMs, such as enhanced creativity and productivity, improved personalisation and customisation, and automated data analysis and insights.
2024 should give us a closer look behind the shrouded curtain of smartphone on-device AI developments. Until then, let the Pro camera comparison videos continue.
Interested in how AI can help your company differentiate? Contact our team today.
