A brave PhD candidate from University of Vaasa, Finland, presents his thesis for public discussion a week before the thesis defense scheduled on Aug. 11, 2016. The thesis "Benchmarking of Mobile Phone Cameras" by Veli-Tapani Peltoketo is available for download and "defines a new benchmarking method for mobile phone cameras. The benchmarking combines different image quality and camera performance metrics to a comparable, single value benchmarking score. According to the research, heavy image quality algorithms used nowadays may slow down functionality of mobile phone cameras. Moreover, the research highlights the main principles to compare mobile phone cameras."
"Nowadays an image captured by an imaging sensor is practically rebuilt again in a post processing pipeline of a digital camera system. The doctoral thesis reveals cameras which image quality features are superior but correspondingly the same camera can be one of the slowest of the comparison."
"Even though the image quality of a camera can be extremely good, slowness of the camera may prevent to capture the right moment and the result is as same as an image with poor quality: the image is not used", says Peltoketo.
"The research introduces also main principles which should be taken into account when benchmarking scores are implemented for mobile phone cameras.
Firstly, all measurement methods and equations have to be public. Used metrics, measurement methods, and especially equations may include intentional or other weights which can bias the result.
Secondly, different measurement environment should be used when mobile phone cameras are benchmarked. Especially low light environment is very challenging to mobile phone cameras. Image quality and speed results may vary significantly in low light environment.
Finally, a perceptual image quality should be taken in to account when benchmarking is done. A color saturation is a good example of differences between perceptual and objective metric. An image looks better, if the color saturation is artificially increased. However, objectively the image includes a color error.
One may ask which is the best mobile phone camera? Peltoketo notes that it is quite difficult to give an unambiguous answer. However, if the results of the thesis give some answers of cameras available year 2014.
“The best image quality was measured from Lumia 1020 and Zopo C2 cameras whereas the speed performance was best in iPhone 5s Samsung S3 cameras.”
"Nowadays an image captured by an imaging sensor is practically rebuilt again in a post processing pipeline of a digital camera system. The doctoral thesis reveals cameras which image quality features are superior but correspondingly the same camera can be one of the slowest of the comparison."
"Even though the image quality of a camera can be extremely good, slowness of the camera may prevent to capture the right moment and the result is as same as an image with poor quality: the image is not used", says Peltoketo.
"The research introduces also main principles which should be taken into account when benchmarking scores are implemented for mobile phone cameras.
Firstly, all measurement methods and equations have to be public. Used metrics, measurement methods, and especially equations may include intentional or other weights which can bias the result.
Secondly, different measurement environment should be used when mobile phone cameras are benchmarked. Especially low light environment is very challenging to mobile phone cameras. Image quality and speed results may vary significantly in low light environment.
Finally, a perceptual image quality should be taken in to account when benchmarking is done. A color saturation is a good example of differences between perceptual and objective metric. An image looks better, if the color saturation is artificially increased. However, objectively the image includes a color error.
One may ask which is the best mobile phone camera? Peltoketo notes that it is quite difficult to give an unambiguous answer. However, if the results of the thesis give some answers of cameras available year 2014.
“The best image quality was measured from Lumia 1020 and Zopo C2 cameras whereas the speed performance was best in iPhone 5s Samsung S3 cameras.”
