Skip to main content

The iPhone X is among the new generation of mobiles using machine learning to produce better quality pictures.

Elijah Nouvelage/Reuters

First there was the size of the camera lens. Then there were megapixel counts. Now, the one factor that is most likely to determine the quality of the photo you just took is the software and the artificial intelligence applied to it.

This third wave of imaging – also known as computational photography – is quickly eroding the conventional thinking of only a few years ago, that cellphones and their tiny lenses would never be able to produce results that rival full-size single lens reflex (SLR) cameras.

Newer smartphones, such as the Google Pixel 2 and the iPhone X, are proving the opposite.

Story continues below advertisement

"When I take a trip, I'll take a big camera with me and I'll also take my phone and I'll assiduously take out my large camera to take photos," says Marc Levoy, a principal engineer at Google. "But I'm always disappointed by the dynamic range, which is better on cellphones."

Google's first Pixel smartphone, released last year, introduced a high-dynamic range (HDR) mode that could instantaneously snap several photos at different lighting exposures. The device then stitches the multiple shots together into single photos that accurately convey both brightly lit and darker areas.

Many SLR cameras have similar HDR modes, but they can be slow and make errors when matching up multiple images.

"They really are far behind and need to catch up," says Dr. Levoy, who is also a professor emeritus of computer science and electrical engineering at Stanford University in California.

Reviewers gushed over the Pixel camera as a result. It garnered an 89 score on influential camera site DxOMark, the highest rating to date for a smartphone at the time.

The recently released Pixel 2 takes imaging a step further by utilizing machine learning, a term often used interchangeably with artificial intelligence because both involve algorithms that become smarter after crunching large amounts of data.

The Pixel 2 relies on algorithms that have studied millions of images, which gives its camera the ability to detect common features and techniques in photographs. It can automatically differentiate between foreground subjects and backgrounds, for example.

Story continues below advertisement

As with several competing phones, the Pixel 2 has a "portrait mode" that produces a sharp and crisp foreground subject with a slightly blurred background. A photographer using an SLR can achieve the same effect manually by focusing the camera's lens on the subject.

The Pixel 2 also differs from its competitors, including the iPhone X, in how it achieves this effect. Most current high-end phones have dual lenses that, in portrait mode, capture two separate images – background and foreground – that are then combined through software processing.

Google's device has just a single lens and relies instead on AI processing, which is yielding better results overall, according to DxOMark. The Pixel 2 has scored a 98, again the highest smartphone rating yet and one point better than the iPhone X.

Apple's device also uses machine learning, but the company says it doesn't draw processing power from the internet as many of its competitors do. Techniques such as AI-assisted facial recognition are done entirely on the iPhone in accordance with Apple's focus on privacy, according to a company blog post.

Rylo Inc., a San Francisco-based company started by former Apple and Instagram employees, is also relying on software processing to create better images. The company says its product, a $500 (U.S.) GoPro-sized camera, lets amateurs create professional-quality videos through fast and easy postproduction.

The Rylo camera shoots in 360 degrees and then effectively crops out the most desired portion of the entire image into a regular two-dimensional video. That means users can simultaneously eliminate camera shake and keep the focus on the subject, both of which are difficult for amateurs to achieve when spontaneously shooting.

Story continues below advertisement

"It's really hard to do because when you're capturing something, things are just happening and you don't know what's coming," says Alex Karpenko, Rylo co-founder and chief executive officer. "That never happens in [the film industry] because you have this entire crew that has planned everything ahead of time."

Full SLR cameras do still hold an advantage in some areas – especially when it comes to zoom, which only proper lenses are capable of doing properly so far.

But industry observers agree that traditional camera makers will need to focus less on hardware and speed up their adoption of computational photography, or risk ceding more of the market to software-oriented companies such as Google, Apple and Rylo.

"It's a major revolution," says Peter van Beek, professor in the Cheriton School of Computer Science at the University of Waterloo. "If the lens doesn't do it, you do it computationally."

Report an error Editorial code of conduct
Tickers mentioned in this story
Unchecking box will stop auto data updates
Comments

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • All comments will be reviewed by one or more moderators before being posted to the site. This should only take a few moments.
  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed. Commenters who repeatedly violate community guidelines may be suspended, causing them to temporarily lose their ability to engage with comments.

Read our community guidelines here

Discussion loading ...

Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.
Cannabis pro newsletter