Reading mode: Light Dark
80

LG patent describes smartphone camera with 16 sensors and lenses

With a total of five, LG's current flagship smartphone V40 ThinQ already offers more camera modules than most of its competitors. However, it appears the current dual and triple-camera setups are only the beginning of a multi-camera arms race in the mobile industry.

According to a recent patent filing discovered by Dutch technology publication LetsGoDigital, the Korean manufacturer could be working on a smartphone with 16 sensors and lenses in the main camera array. The individual modules are arranged in a 4 x 4 matrix and designed to record the same scene from slightly offset angles, enabling the capture of 3D video or interesting image manipulation effects.

In an example it is shown how it would be possible to rotate the head of a portrait subject (or a teddy bear in the illustration below) to the angle you like best after capture.

Another example shows how the multi-cam technology could be combined with artificial intelligence features. Users could select a subject by drawing around it. The system then searches for other photos of the same person and then offers other, potentially more flattering, "heads" to replace the original with. This could be a useful function when one person in a group shot has their eyes closed, for example.

The patent also describes how the main camera array could be used to take selfies using a simple mirror, a concept that users of older flip-phones or compact cameras might still be familiar with.

As usual there is no way of knowing if or when the technology will make it into a final product but it's great to see manufacturers are working on new ways of making the cameras in our pockets even more powerful. For more detail, you can have a look at the patent documents on the USPTO website.

View Comments (80)

Comments

All (80)
Most popular (5)
Editors' picks (0)
DPR staff (0)
Oldest first
Jesse_Just_Him
Jesse_Just_Him

When in doubt, add more camera~!!!

1 week ago
peterwr
peterwr

*Cough* Light L16 *cough*...

1 week ago
808_freedive

V20 I had was horrible in every way. V30 means 30 additional faults.

Its beyond me why LG, Samsung always get it wrong and spend a fortune on marketing to cover that sad little fact up. 16 cams and still a turd.

1 week ago
Mike FL

By using Winnie the Pooh, LG is asking Chinese communist leader XI to ban the LG. Take a look:

https://www.theguardian.com/world/2018/aug/07/china-bans-winnie-the-pooh-film-to-stop-comparisons-to-president-xi

1 week ago
Jake2046

well...at least LG has a reason to jack up their price.

2 weeks ago
Eskilsson
Eskilsson

Someone should invent a monocular that I could use as an evf för my phone. No matter how good your screen is, you always will have problem with extrem sunlight.

2 weeks ago
SandySandy

I would have preferred a camera with 4 sensors, 2 with 48MP 1/1.7" and 2 with 12MP and 1/1.55". And some superb computational photography for 5x Zoom at 12MP and usable ISO to ISO6400 equivalent. And 14 stops in HDR photo in HEIF format.

Also if it could have a 5k video mode, for high speed photography, that would be the cherry on the cake.

2 weeks ago
Franz Weber
Franz Weber

The main benefit of this 4x4 array is that you can have lenses combined from different manufacturers and all these flavors are blendet to one super-jpg photo. Zeiss, Nikon, Fuji... you name it. Think of the possibilties: A Zuiko Batis L lense.. with anti vibration, stadyshot AND image stabilizer.

It will give you more flexibility with assesories as well: Attach a lens hood on some lenses for image quality and leave it off on other lenses for compactness and convenience. And for savety reasons allways apply a lens cap on one lense which stays there while shooting

2 weeks ago*
ozturert

With a tiny sensor, I wonder why you'd want to pay for a Zuiko or Zeiss lens.
Plus, these phones keep getting more and more expensive every day.

2 weeks ago
Mike FL

"wonder why you'd want to pay for a Zuiko or Zeiss lens." while Zeiss can not wait to sell you some phone lenses. Take a look:

https://exolens.com/products.html

1 week ago
barrym1966

be cool if they 1st increased the battery life to more than 1 day and also got them to run android properly instead of being lagtastic

2 weeks ago
Crypt Keeper
Crypt Keeper

LG has always been one of the most underrated companies, despite being very innovative.
As I recall, they created the first 3d smartphone (LG Optimus 3D), they were the first to produce higher screen to body ratio smartphones, optical image stabilization in an smartphone camera, high resolution, high pixel density smarphone screens, modular smartphones, etc, etc, etc....

2 weeks ago*
SandySandy

That's true. LG innovates extremely well. Their only downside has been implementation which is not refined or perfected and quality control which could improve. If they perfected those two, they could easily be the No1 or No2 smartphone company.

2 weeks ago
Critical Thinker

And here comes the barrage of comments from old dinosaurs complaining about having this many lenses because they think its literally 16 different focal lengths and didnt read the article.

2 weeks ago
madeinlisboa
madeinlisboa

No. It's just ridiculous...

2 weeks ago
ozturert

I have read the article and I think it's stupid if the price is over 1000+. I want one big and good sensor in my phone. I don't need to change heads of my friends in photos.

2 weeks ago
beavertown
beavertown

Only Apple knows how to do photography with smartphones.

2 weeks ago
HowaboutRAW

Panasonic is still leading the smartphone camera category, with the CM1--discontinued 2015 (or perhaps early 2016).

2 weeks ago
Bobthearch
Bobthearch

Really? According to these sample comparison shots the Apple phone camera is even worse than the Google and Huawei models.

https://www.dxomark.com/huawei-p20-pro-camera-review-innovative-technologies-outstanding-results/

2 weeks ago
HowaboutRAW

Bobthearch:

I assume beavertown is joking.

No, in all seriousness, DXO sensor scores are next to useless.

Separately, yes, the raws I've extracted (not shot myself) with the Huawei are impressive for a current (2018 available) smartphone.

2 weeks ago
Bobthearch
Bobthearch

I give no credence to their sensor scores, as the scores are only relative to other camera phones; I was looking only at the side-by-side photo samples.

The Huawei photos are pretty bad, but the Apple phone's are even worse.

2 weeks ago
HowaboutRAW

Bobthearch:

Right, Apple has some catching up to do even in the limited market that is smartphone cameras.

Samsung could introduce a smartphone based on the already developed NX-Mini ILC system and have the best smartphone camera out. Or Panasonic could introduce the CM2 with the same lens/lens and a bigger screen, more RAM and a faster CPU and best every other smartphone camera for a while.

2 weeks ago*
Charlie Jin

Apple knows how to hypnotize people.

2 weeks ago
ozturert

Only Apple knows how to turn your face to plastic and sell their "effect" for USD1000+.

2 weeks ago
Mariano Pacifico

I agree ... Apple knows how to do photography with $50.00 lens ... I ABSOLUTELY AGREE !!! And they have beauty effect that makes ugly faces beautiful and appealing ...

2 weeks ago*
EXkurogane
EXkurogane

Meanwhile, Google pixel 3 with only 1 camera: Am I a joke to you?

2 weeks ago
Mariano Pacifico

1 camera in smartphone is sooooo 6 months ago.

2 weeks ago
falconeyes
falconeyes

I browsed through the patent application which dates back 5 years.

It seems it is looking desparately for applications of a 4x4 camera array. And it fails at that, it’d say.

The most important and innovative use case of a 4x4 array would be to multiply an otherwise huge crop factor by 0.25, bringing smartphones pretty close to system cameras in terms of image quality. And that seems to not be even mentioned in the patent.

2 weeks ago*
Mariano Pacifico

Ohhh Mmmm Geeee !!! How am I going to hold the smartphone without smudging 16 lenses?

Will there be a Nikon & Canon smartcamera? Right at this moment these soon-to-be-irrelevant behemoths do not have smartcameras while cellphones have already evolved into smartphones.

2 weeks ago
ikan154
ikan154

use 16-legged pod for best stability

2 weeks ago
HowaboutRAW

Sorry, which smartphone competes with say a half decent Canon lens mounted on a Canon 6D Mark 1 for plain image quality?

Which 2018/17 smartphone can you easily shoot at ISO 2500 again?

2 weeks ago*
mxx
mxx

Or at 600mm?

2 weeks ago
brn

With a 16 way zoom lens attachment, of course.

2 weeks ago
Mariano Pacifico

16-lens smartphones will have ultra-wide, normal wide, 35mm, normal lense 50mm, 80 & 105 portrait lenses ... how many is that now ... 200mm ... 300mm ... 600mm ... that is 9 lens total 7 more lenses to go ... wheeew !!! That is more than a bagful of lenses ...

Of course it cannot compete with 6D Mark 1 & ISO 2500 but the convenience when we need it is there at an instant in our pocket. If we have to fiddle and tweedledee we lost that "moment".

2 weeks ago
HowaboutRAW

MP:

But then a Sony RX100 IV or a Panasonic ZS200 would be better pocket cameras.
And both of those one can readily shoot at ISO 2500.

No bag full of lenses needed with the Panasonic and Sony.

2 weeks ago
RUcrAZ
RUcrAZ

Great news! It means that there will be a war among cellphone makers, to improve their camera function. Cellphone-cams are still at their infancy, give them time. (I remember the early digicams were like the Sony that took 1 single digital photo on a whole floppy disk; and look how far we've come!) If it wasn't for WWI and WWII, airplanes may still be biplane stringbags! The (new to me) software-massaging photo approach by Google Pixel, coupled with multiple lenses & better sensors is obviously the start (just the start) of something significant.

2 weeks ago
photophile
photophile

My Gillette razor now has 17 blades - but I think I'd get a closer shave if I had 18.

2 weeks ago
webber15

No...fewer but larger blades are the way to go 😂

2 weeks ago
Lars V

Yeah we did this (and patented) at Pelican Imaging several years ago. This is nothing new. I see lawsuits coming up.

2 weeks ago
Jon Stern

This patent seems to be for an array with each camera having a different FOV.

Quote translated from a German site: "The lenses are placed in a certain curvature within the matrix arrangement, so that the different lenses can make photographs from multiple perspectives."

2 weeks ago
Lars V

Thanks, Jon. Presumably a convex curvature. Or possibly multiple perspectives refers to parallax resulting from offset, which is then used to estimate depth. I hate reading patents.

2 weeks ago
Jon Stern

It seems to be curvature of the mounting plane, not just parallax.

2 weeks ago
UllerellU

I could never take photos with a cell phone. Not even if in the future they were able to offer good image quality and low light capacity. For me the hergonomy and the pleasure of use is everything. I have replaced my compact RX100 for an LX100 for the same reason, I have made good shots with the RX100 in these years, but every time I use it I realize that I do not enjoy everything I would like, regardless of the result. And I do not like to shot without viewfinder, nor in my third camera body. And with the RX100 I have a physical button for the exposure lock, apertura ring, focus ring for fine MF, exposure compensation in two touches, shooting mode in two touches and functions such as measurement mode, white balance etc. in a quick submenu. Despite not being a camera that is too pleasant to use, it is a million times more pleasant to use than a mobile phone.

2 weeks ago
Paul_B Midlands UK
Paul_B Midlands UK

Wait till you try a Leica Q ...such a lovely thing to make photos. I used to like Sony till I got this.

2 weeks ago
mxx
mxx

The cell phone may not be great on ergonomics, but if it's the only camera you have on you, it sure is very handy.

2 weeks ago
UllerellU

I always carry my compact, even on business trips, it is easier for me to forget the phone at home, in the hotel or in the car before I forget the camera...

2 weeks ago
madeinlisboa
madeinlisboa

LOL! Another proof that cellphone companies are going completely nuts

2 weeks ago
JochenIs

Nices sales argument. My LG G3 is still working fine btw. :-)

2 weeks ago
QuangFoto
QuangFoto

No kidding. I replaced my G4 with the V30+, and this thing is amazing.
https://youtu.be/nAth8g05tfs

2 weeks ago
sfft
sfft

What about a patent that cools off motherboards in LG mobile phones?

2 weeks ago*
duartix

I'd buy that!

2 weeks ago
Marty4650
Marty4650

One good sensor will outperform 16 mediocre ones. No matter how many computational tricks you use.

Just remember.... those camera makers can use computational tricks too whenever they feel the urge to do so. And real cameras will ALWAYS have an ergonomic advantage. There is a fundamental conflict between "pocketable" and "can do everything you want done."

If you need images for web display, then a six year old smartphone will do the trick. If you need something significantly better, then you will need a real camera with a much larger sensor. So it really makes very little sense to make smartphones "more like a camera" or cameras "more like a smartphone." These are two different markets with two different type of user, ever since the bottom of the camera market collapsed.

Right now people are spending thousands of dollars for full frame cameras that are only marginally better than crop sensor cameras. And they do this because those marginal differences are very important for them.

2 weeks ago
duartix

Define mediocre.
Because if mediocre is 2x worse, with 16 cameras sensors you are still 8x ahead in light gathering capability.
Computational photography has come to stay. If camera manufacturers don't get it (some have already and have started producing SR from 4/8 exposures) , they will slowly continue to wither.

2 weeks ago
Michael Long

Reminds me of the early days when digital was just beginning to replace film and all of the film guys were talking about how digital would "never" outperform film.

It also strikes me that the OP simply doesn't realize just how many "computational tricks" are already being used to pull an image off a Bayer color filter array.

Next, "real" cameras have ergonomic advantages.... coupled with ergonomic disadvantages, namely size and weight.

Finally, marginal differences are in fact important... to an ever shrinking number of people. IQ is steadily increasing and each gain makes the final result "good enough" for an ever growing set of use cases.

IQ is important... but it's not the sole consideration, nor is it even the most important one. If it were, everyone would still be shooting with 8x10 view cameras...

2 weeks ago
HowaboutRAW

Michael L:

And yet as of late 2018, the IQ of not one current smartphone camera comes close to a decent current 4/3s sensored body and a good lens.

It's pushing things to use any smartphone available new above ISO 1000 in 2018.

And remember, digital took a good while to get up to producing an APSC sensor one could shoot at ISO 800 easily. But then things quickly took off, and raw extraction software got much better, and the flashcards got much faster, much cheaper, and more capacious.

Multi-lensed cameras show some promise, but it's years before they will replace conventionially lenses cameras of fixed focal length. Right, remember there's the non-tele/zoom problem with 4 or 16 sensors.

2 weeks ago
Arastoo Vaziri
Arastoo Vaziri

Duartix, the "light gathering" is a myth DPR staff invented in order to sell more smartphones to gullible gearheads who will swallow anything if it has a pseudo-scientific reasoning behind it, but there's no truth to it. Besides, the extra sensors aren't used in tandem, so the gathered light will never be the same as per a, say, APS-C sensor
Michael Long: digital photography hasn't caught up with medium and large format film yet. And a smartphone *is* an ergonomic nightmare. I'd go as far as to say it's a PITA to shoot with.

2 weeks ago*
panther fan

"And yet as of late 2018, the IQ of not one current smartphone camera comes close to a decent current 4/3s sensored body and a good lens."

Except in some use cases, where it already does. For example hand held low light photography... the last thing people expected from a smartphone... but it beats M43 handily. And if you need deep DOF it even beats FF

Computational photography is here to stay and will probably drastically improve in the next years. Something I am not so sure about is, that cameras will develop equally fast

2 weeks ago
HowaboutRAW

panther:

In 2018, there isn't really a smartphone one can shoot at ISO 3200.

While all new 2016-18 4/3 sensored cameras can easily shoot at ISO 3200.

There are of course many more fast lenses of m4/3s cameras.

So no, you are wrong.

"Computational photography is here to stay and will probably drastically improve in the next years."

It will stay around, it is unlikely to drastically improve in the next few years. 15-20 years out is a different subject.

"And if you need deep DOF it even beats FF."

Is this supposed to be a joke? Do you know what deep DOF means?

Please don't go on about stacked images from multisensored cameras being able to reduce final output noise. Someone here already embarrassed himself badly last week with claims about the Google Pixel 3 and moving images in the foreground.

Ironically, 2014 saw the release of a smartphone one could use at ISO 3200: The now discontinued Panasonic CM1.

2 weeks ago*
panther fan

@HowaboutRAW

The fact that you talk about ISO 3200 shows that you don't quite understand current tricks in computational photography.

We live in a time, where you need to pixel peep night shots to tell the difference. Just search the internet for google night sight vs DSLR and you will see tons of comparisons, showing that the pixel crushes night photography

PS: Go and search a comparison with moving objects. And you will see that the Pixel3 works phantastic:

for example.
Or look at the bazillion low light portrait comparisons

2 weeks ago*
HowaboutRAW

panther fan:

You still have the image stacking problem; it's great for static images. Which are few and far between.

Nothing to do with pixel peeping. In 2018, there is no ISO (gain if ISO confounds you) 3200 capable smartphone.

"Just search the internet for google night sight vs DSLR and you will see tons of comparisons, showing that the pixel crushes night photography".

Again, not if something is moving the the foreground. Then then no, no smartphone in 2018 crushes any contemporary DSLR for low light. Now a DSLR from 2004 is close to that low bar the Pixel 3 pretends to leap.

2 weeks ago
panther fan

@HowaboutRAW

You didn't even bother to check it out yourself. Yes stacking has some problems. But movement surpression works really well in the pixel. And that includes complex moving subject like humans.

Yes you can push a modern FF a little further in some scenarios. And that is great. But does it matter for regular people? No it doesn't. And that is the real problem. Without a large camera market, we will simply see no fast development anymore. A real problem if you look at the grwothrate smartphone cameras have

2 weeks ago
HowaboutRAW

panther fan:

"You didn't even bother to check it out yourself. "

When you provide raws, I'll be happy to download them and extract them.

I'm not going by silly YouTube assertions.

"But movement surpression works really well in the pixel."

No it doesn't. This is well established, and that you don't need raws to check. You can find examples on YouTube of big failures of the Pixel 3. Now they're where you'd expect the failures to be--car moving across foreground.

"regular people" got along just fine with 35mm point and shoots in the 1980s and 1990s; then more serious types who cared about greater performance used SLRs. Today in 2018, DSLRs have the added advantage of much better sensors.

I'm guessing you don't remember when it was radical for a DSLR to be easy to use at ISO 3200. I'm guessing you have no idea how much AF has improved on mirrorless and DSLRs in the last say 8 years. Well I'm not actually guessing.

2 weeks ago
panther fan

" I'm guessing you have no idea how much AF has improved on mirrorless and DSLRs in the last say 8 years. "

I have. I did go mirrorless with the first FF wave. I owned an A7, A7 II and rented an A9 multiple times, because my A7 II just doesn't cut it for events.

"When you provide raws, I'll be happy to download them and extract them."

Give me a week. And I will provide you some RAWs. I will probably migrate from my Nexus 5X to the Pixel 3 in a few days. So I will happily upload some RAWs

2 weeks ago
HowaboutRAW

panther:

Great, I'll look at higher ISO raws.

2 weeks ago
Will Murray

I don't see why a sensor array in a mobile device would adhere to different principles to any other sensor array.

2 weeks ago*
Akpinxit

this is going to be some challenge for add on lens manufactures

2 weeks ago
ozturert

It's heaven for them. Just an excuse to raise prices.

2 weeks ago
zeratulmrye

Sony Semiconductor: atta boy, good job!

2 weeks ago
GodSpeaks
GodSpeaks

Seriously? I think I will pass.

2 weeks ago
Brian P Smith

This is getting to be like razor blades were years ago... just keeping adding more of the same.

2 weeks ago
duartix

Well, look at history. It worked for razor blades...

2 weeks ago
mxx
mxx

I still shave with a double blade, and it works fine, and is cheap.

2 weeks ago
Arastoo Vaziri
Arastoo Vaziri

This is getting ludicrous. It's not evolution, it's just gimmickry.

2 weeks ago
Suntan

Most likely the system is designed for better 3D mapping. IE, for augmented reality. Potentially a gimmick, but something that all the phone makers are working towards.

This isn't just "moar cameras" for the sake of more cameras.

2 weeks ago
mr_landscape
mr_landscape

Yeah, let them keep on stacking small sensor sized cr#p-over-the-top.
N808 is the best camera-phone till today.

2 weeks ago
princecody
princecody

Can the 1 inch compact pocket camera still exist with new smartphone tech?

2 weeks ago*
HowaboutRAW

Yes. And that will be true for years to come.

You "forgot" that those compacts have optical zooms.

Go find raws from the Light multi-sensored camera.

2 weeks ago*
Michael Long

You'll note, however, that the short zoom pocket camera is all but disappearing.

2 weeks ago
HowaboutRAW

ML:

Not familiar with the RX100 V and the Panasonic LX10 I see.

The LX100 II also doesn't have a particularly long zoom. But true that's more jacket pocketable.

And note how princecody specified 1 inch compacts, not 1/2.3 inch compacts.

2 weeks ago*
GeraldW

It really depends more on how many of us are willing to spend for that 1" sensor camera, and are serious enough about their photography to need one. I see the smart phone camera as the equivalent of the Instamatics and box Brownies of the past, and the low end digital P&S models. An easy to use camera for the snap shooter. Once the smart phone got decent cameras, those same buyers left the major camera brands. The result was that there were not enough buyers left to make them viable products.
What helps the 1" sensor models is the high prices. You need to be pretty sure about your photography needs to justify that kind of price. We can argue indefinitely that camera phones aren't a match to 1" sensors. For most folks, they couldn't care, and are happy with their phones.

Jerry

2 weeks ago