Camera Phones: History, Evolution And Technology

“The way that light hits objects, I think, is one of the more important things that sculpture and photography share.” – Rashid Johnson

Photography is the art of capturing still images or recoding moving objects and this process of photography is done by recording light. Camera is an important instrument to achieve photography. In earlier days, we used to have negative film rolls which are loaded into the camera (a box like structure which consists of lens and image capturing mechanism) and later images are developed from these negative films into printed images. The digital cameras took over these traditional film cameras. Now, we are having an in-built camera which is an additional and most rewarding feature in mobile phones (which does not need an additional baggage to the user to carry a separate camera) and made it easy to the user to take pictures anywhere, anytime. The main difference between the digital camera and phone camera is the sensor size (which determines how much light it uses to create an image. So, a bigger sensor can gain more information to produce better images) and the aperture (it is the unit of measurement of the size of the opening in the lens that can be adjusted to control the amount of light reaching the sensor) in the camera phones is fixed whereas in normal digital cameras it can be adjusted manually according to the need. But the aperture modification issue is also solvedy dual aperture camera lens in camera phones (e.g. Samsung Galaxy S9).

The camera phone was first patented by Kenneth Parulski and James Schueckler who were engineers at Kodak and the application is titled as “ELECTRONIC CAMERA SYSTEM WITH PROGRAMMABLE TRANSMISSION CAPABILITY”¹ (US 5666159) which was filed on April 24th, 1995 at USPTO. The first camera phone J-SH04 was successfully marketed by J-phone in Japan in November 2000, with a camera resolution of 0.11 MP which used charge-coupled device (CCD) sensors instead of CMOS (complementary metal oxide semi-conductor) sensors. CMOS sensors are widely used rather than CCD sensors in today’s camera phones. Both CCD and CMOS sensors convert light into electrons.² In the same year Samsung introduced SCH-V200 camera phone and the main difference between J-SH04 and SCH-V200 was that the J-SH04 phone allowed the user to send the pictures via e- mail.

In the subsequent years, other market players introduced camera phones with an improved version of camera resolution over the existing ones. So, in a span of 10 years (by the end of 2010) the camera resolution reached upto 16MP (Sony Ericsson S006)³.

DUAL CAMERA PHONES

The Dual Camera Phone was introduced to showcase 3D capabilities in smartphone. The LG Optimus 3D was announced in February 2011 and the HTC Evo 3D launched in March 2011 but both of them failed to attract the market. HTC continued its research and finally launched HTC One M8 in April 2014 with enhanced depth and focus effects on the image but still had defects like blurred backgrounds which was later rectified in 2016.

Followed by HTC, other market players developed the dual cameras with different configurations like going wider with the combination of wider lens and normal lens in which the resolution range differ in both the sensors. This configuration was seen in LG G5 camera phone in which first camera sensor is 16MP and second camera sensor is 8MP with 135-degree lens which allowed to shoot wide-angle images with enhanced quality.

“Mobile Camera System” (US 20160007008)⁴: A patent application by Apple Inc. That explains a similar optical zooming functionality.

The claims in the above image explains the process of capturing images with two different lenses of varying focal lengths and the images from these two camera lenses are further processed by instructions or algorithms that are computer executable and the resultant image is the integration of the above two images that is obtained from such algorithm. The first lens (with smaller focal length as compared to second lens) captures image of a first visual field which can be a stationary object (example a person or a thing) and second lens captures the image of a second visual field which is the background image of the stationary object. So, the resultant image will have clearer view of background around the still object which is obtained by optical zooming functionality of the lens in combination.

In addition, other noted configuration is the combination of Telephoto lens and Wide-angle lens where the wide-angle lens covers vast area while taking the photograph (landscape photography) and telephoto lens have narrow view and longer focal length which helps to view objects closer than they appear (portrait photography) this configuration was made available in Apple iPhone 7 plus, One Plus 5, Samsung Note 8. Normally telephoto lens have a bokeh effect, which means blurred background in the image to make the appearance of the target object clearer and closer, but with the combination of wide-angle lens along with telephoto lens, the background of the target object appear with clarity.

“Folded Telephoto Camera Lens System” (US 9316810)⁵: This is the granted patent filed by Apple Inc.

This patent claims a foldable telephoto lens camera embedded in the mobile gadgets. The electronic devices which are thin are preferable over thicker ones. This foldable lens aids in designing a thinner mobile device which is acquired by the configuration of light path folding element present between first optical axis and second optical axis of the lens system. This light path folding element is the sheet which can be expanded while the camera is in active mode ad allow for optical zooming of the lens. Through the Telephoto lens the image of the target object appears closer than the actual object distance from the camera.

Another combination of sensors is monochrome and color (RGB) sensors combination. In this configuration one sensor takes picture in color while the other sensor clicks picture in black and white.  This kind of setup can be seen in models such as Huawei P9, Nokia 8, Honor 8, Apple iPhone 8. “Electronic Device with Two image Sensors” (US 8115825)⁶: this is a granted patent that claims a device of integration of two different cameras.

The above claims describes the device which consists of two different camera lenses of which monochrome lens captures black and white image and the other lens (RGB lens) captures the color image and these two images are further processed by a computer algorithm which uses the input from the first two camera lenses and adjust the color of the image to get a high color quality image as a result. The aim of the above process of homogenizing the two different images is to counteract any blind spots (a portion of the image which low color effect or blurred portion of the image where the light rays could not reach) in the two images captures by both the lenses.

Some of the few key features of these dual camera phones are:⁷

1. The sharp auto-focus feature helps while shooting the image and focus can be adjusted after taking the image too.
2. Bokeh effect: the bokeh effect simply means blurred background in the image with accurate details and this effect can be achieved through depth sensors.
3. Zoom: the telephoto lens can provide less distorted optical zoom (zoom acquired by adjusting the camera lens) upto 2X and digital zoom (zoom acquired by adjusting the image) upto 12X
4. Image quality: with the combination of different sensors dual camera results in improved quality images as compared to a single lens camera.

TRIPLE CAMERA PHONES

Why triple phone cameras when the dual cameras can take pictures of higher quality and more depth?

The reason for more lenses is to get more enhanced quality images than dual camera phones. We can consider this trend of adding more cameras as a marketing strategy for driving up prices but it can also be seen in a positive angle as increase way to empower the user with enhanced photography capabilities. The extra cameras always improve the blur effects, enhance digital zooming, increase dynamic range, improve low-light photography, allow switchable aperture and better depth sensing capability. Practically it is not possible to have all these improvements even with three cameras but they are better than dual camera phones.

Apple Inc. has been granted a patent on triple lens camera for better resolution and color accuracy titled:

“Image capture using luminance and chrominance sensors (US 8497897)”⁸

The image sensing device has three separate sensors which have their own functionality and the results from these three sensors are finally combined to form a composite image, as claimed here. This composite image will be enhanced in color compositions.

The above patent claims both image sensing device (product claim) and the mode of operation (process claim) of that image sensing device. This image sensing device have three sensors out of which one is luminous sensor and other two are chrominance sensors. The sensors are arranged where the luminous senor is placed in between the two chrominance sensors. The luminance sensor would determine light levels of the image, while the chrominance sensors accurately capture the color data of the image. This arrangement of sensors will compensate for blind spots in each other’s field of view, ensuring color accuracy in the image is produced.

15 mobile phones that implemented triple camera lens:

Model	Configuration
Samsung Galaxy A7	24MP primary sensor + 8MP wide-angle lens with 20 degree field of view + 5MP (depth sensor)
Samsung Galaxy S10+	12 MP + 12 MP + 16 MP Triple Rear setup(wide angle lens and telephoto lens) 10MP+8MP dual front setup
Huawei Mate 20 Pro	Triple camera system with Leica optics 40MP primary sensor + 8MP secondary sensor + 20MP ultra wide-angle shooter 3x hybrid zoom and AI image stabilization
Oppo Reno 10x Zoom Edition	Periscope-style telephoto lens that facilitates 10x Hybrid Optical Zoom 13MP sensor (f/3.0 aperture) + 48MP(f/1.7 aperture) primary rear Sony IMX586 sensor with LED flash + 8MP (f/2.2 aperture) 120-degree Ultra Wide Lens
Samsung Galaxy A80	Rotating camera mechanism ( for front and rear camera) 48MP (F2.0) primary sensor + 8MP (f/2.2 aperture) 123-degree Ultra Wide sensor + 3D Depth sensor
Vivo iQOO	12 MP primary sensor with Sony IMX363 sensor + 13 MP with wide angle lens + 2MP(depth sensor) Triple Rear setup
Huawei P30 Lite	24MP (f/1.8 aperture) primary lens +  8MP 120-degree ultra wide-angle sensor + 2MP depth sensor (ToF) 5X optical zoom 32MP (f/2.0 aperture) front camera
Vivo V15 Pro	48 MP + 8 MP + 5 MP Triple Camera with autofocus in the rear 32MP front camera
Xiaomi Mi 9	48MP primary sensor with 0.8 µm pixel size and 1/2” image sensor + 16MP secondary Ultra-wide sensor (f/2.2 aperture) + telephoto lens with 12MP sensor (f/2.2 aperture) for 2X optical zoom
Oppo R17 Pro	12MP primary sensor having f/1.5 -2.4 aperture + 20MP sensor having f/2.6 aperture + TOF 3D stereo camera
LG V50 ThinQ	12 MP Primary sensor with f/1.5 aperture and 1.4μm size pixels + secondary sensor of ultra-wide-angle f/1.9 aperture 16MP camera with 107-degree view field + 12MP third sensor with f/2.4 aperture telephoto/portrait camera with 2x zoom
Sony Xperia 1	12 MP primary sensor + 12 MP wide-angle + 12MP telephoto sensor
OnePlus 7 Pro	48 MP + 16 MP + 8 MP Triple Rear setup  16 MP Front Camera
Huawei Honor 20i	24 MP + 8 MP + 2 MP Triple Rear & 32 MP Front Camera

One noteworthy feature to be considered in most of these triple camera phones is the use of ToF (time-of-flight) sensors which are not only used for augmented reality applications but also to measure subject distance for auto focusing.  Let us look into few patents which are granted for ToF camera technology.

“Time-of-flight camera with motion detection” (US 9625569)⁹

This ToF camera is used for distance measurement and also to detect the moving objects. This ToF camera operated in different detection modes (image mode or video mode) with a single ToF sensor.

This ToF sensor can detect a movement of a moving object in a power saving mode based on the differences in brightness which the moving object produces to aid auto-focusing of the lens to capture the moving object without any blurred effects.

3D TOF camera with masked illumination (US 9410800)¹⁰

This patent claims a method and system for reducing stray light and multiple reflections in 3D time of flight cameras by partly masking the camera’s illumination elements.

The above claim explains the method of masking the multiple light reflections by suppressing the amount of light generated by such illumination module which is outside the bound view limits of the sensor (the territory covered by the lens) whereas this sensor detects the modulating light signals from the illuminating module and this sensor have field of view within the field of illumination. This masking of illuminating light outside the filed of view can help in reduction of stray light, reduction of multi-path, and decrease of required dynamic range to the ToF sensor.

4 Camera Phones which adopted ToF sensors are:

HUAWEI P30 PRO	ToF sensor to detect the distance of the object from the camera to aid in auto-focusing which gives a high-quality image.
Sony Xperia XZ4	This is an upcoming model which uses 3D ToF sensors to add another dimension to the images captured.
LG G8 ThinQ	ToF sensor will provide more accurate data regarding the distance of an object from the camera with the help of infrared light. This makes the sensor faster, “more effective in ambient light” and  reducing the power consumed for these advanced features. So this ToF sensor will better function in 3D object tracking, face unlock, background blur in selfies and improved AR-centric features.
Apple iPhone XI	Apple was evaluating a time-of-flight approach that calculates the time it takes for a laser to bounce off surrounding objects to create a 3D image of the environment. The third lens will include ToF sensor which improve the iPhone’s augmented reality capabilities.

In essence, it is insignificant to increase the number of lenses without improving the quality of pictures. We can foresee phones with quad camera and many more. From the recent market trend it can be estimated that the camera phones with increasing number of camera lenses will always strive to improve the quality of the images and give an overall better photography experience to every user.

So, why not get a mobile phone with better camera features for better quality images that gives a user a professional experience in photography. So, get ready for next field trip or vacation with your best camera phone having all the professional features to capture every moment of your vacation and make it more colorful and memorable.