History: Hard Battle Against Dust

...
ElectronicsWeekly quotes Sony history site on the yield of early CCDs:

"In January 1980, six years and three months after Iwama first gave orders to begin work on the CCD, the world's first CCD camera was produced. However, the production line's yield of viable CCD chips at this stage was poor and only one in several hundred was usable. It took twelve months to manufacture fifty-two CCD chips necessary to manufacture twenty-six cameras. As a result, each CCD was priced at 317,000 yen. Such a CCD would never realize the 50,000 yen target price that Iwama had specified.

"We can't call this a yield rate. It's more like an occurrence rate of usable chips," complained those overseeing the CCD production. The vicious "enemies" undermining their efforts were superfine dust particles that measured less than several microns and were undetectable by human eye. Clean rooms and dust proof attire reminiscent of space suits were introduced at all development and production sites. The battle against small dust was vigorous; people, machines, and every other thinkable source of dust were identified and dealt with.

After all the measures were applied, the yield rate improved enough to implement full mass production from 1983 at the Kokubu Semiconductor plant in Kagoshima.
"

"Once a quality problem affects an OEM customer, it requires a lot of time and effort to regain that customer's trust. Ochi and his team did have a very difficult time selling the CCD chips in 1983 when they first began OEM sales. They visited various camera manufacturers, but were turned away due to a poor track record. Sony had reduced the volume of semiconductors shipped to its customers to meet a surge in internal demand. Ochi and his team started by apologizing for this. The marketing of CCD chips was literally built on the team's sweat and tears."
9:06 AM

Videantis Takeaways from AutoSens 2016

...
Videantis posted its review of AutoSens 2016 conference. There are 5 things learned from the conference:
  1. Self-driving cars are hard. After Tesla Autopilot accidents people are more conservative about self-driving prospects. One analyst mentioned 2040 as a possibility for driverless cars to become available, beyond the horizon for most technology companies.
  2. Deep learning is hard. The nets need to be trained with huge amounts of image data, which needs to be properly annotated by hand, with some companies having hundreds of people on staff to perform this task.
  3. Image quality is hard. Image quality will remain a key topic for quite some time.
  4. We need more sensors. OEMs designing 12 cameras into their cars, and this number continuous to go up.
  5. Surround view replacing rear view. These systems are quickly becoming the rear view camera of yesteryear.
10:10 PM

First Visual Innovation Award

...
Arizona State University: The new Visual Innovation Award has been announced at IEEE International Conference on Image Processing.

Ren Ng, founder of Lytro light-field camera, has been named the first recipient of the Award. Other finalists related to the image sensing field were Achin Bhowmik, Intel VP, development and deployment lead for Intel RealSense camera technology; Brendan Iribe, co-founder and CEO of Oculus, VR; and Alex Kipman, inventor, Microsoft Kinect.
1:03 PM

Nvidia on Key Issues in Automotive Imaging

...
Image Sensors Auto US publishes an interview with Nvidia imaging architect Joshua Wise. Few points from the interview:

Q: What are the key standards issues need to be addressed as components get more complex and diverse?

Joshua Wise: There are two that are top on my list right now. The first that comes to mind is safety and compliance: an important issue in the automotive environment is the ability to self-diagnose issues. In short, the system must “know when it doesn’t know”. As more components enter the ecosystem, there is more opportunity for data to be damaged in transit — and, similarly, more components result in more health data that must be aggregated and transmitted.

The second on my list is the need for a standard in transmitting data with a high dynamic range. There are as many implementations of sending pixels with greater than 12 bits of data as there are vendors right now — perhaps even more! We’ve been working with vendors to come up with solutions that work with both modern and legacy components, but we see an opportunity to unify and standardize here.
12:52 PM

Keynote on Fast Image Sensors

...
University of Strasbourg and CNRS Prof. Wilfried Uhring presented his keynote "High Speed Image sensors" at SIGNAL 2016 conference on June 27, 2016 in Lisbon, Portugal. Few slides out of 33:


Talking about the IO speed, 25GPixel/s limitation is somewhat obsolete by now. For example, PCIe 4.0 standard defines 32 lanes of 16Gbps each, with the aggregate bandwidth of 512Gbps. Assuming 10b per pixel, one can get 51GPixel/s I/O speed by just buying the PCIe 4.0-compliant IP. And PCIe 4.0 is not the fastest interface these days.
12:56 PM

A Palace of Bird Beaks

...
In class, I've been learning about race and inheritance. This week we read a folktale from the Yemen Oral Tradition called  A Palace Of Bird Beaks. The story told of an all-powerful king whose wife wanted him to build her a palace constructed only of bird beaks. At first he intends to fulfill her request, calling all the birds in the world to him and commanding them to give up their beaks. Then, a single bird, a hoopoe, who originally fails to answer his call comes forward and presents him with three riddles, each chosen to remind him of the need for empathy. The story ends with the king realizing his mistake and rewarding the hoopoe for its wisdom.

Empathy
Human against self

The most important theme in the tale was empathy, or lack of it. Empathy is defined as the ability to share and relate to another's feelings. In the story, the hoopoe attempted to make the king feel empathy through the riddles it asked him, but didn't expect him to actually be lenient, as shown when "The hoopoe bowed its head... and it waited in silence to hear the harsh punishment of the king."

However, the hoopoe also showed great wisdom in its choice of riddles, as each inspired a new kind of empathy or understanding in the king. The first riddle reminded the king of a belief they both shared (that all creatures should be free), leaving him to conclude that what he was doing was enslaving his fellow beings. The second riddle told him of an emotion they both shared (sadness) and reminded him that he was causing sadness in the soon to be beak-less birds. The final riddle taught him of the importance of a birds beak - "something delicate enough to put food in a babies mouth yet strong enough to bore holes in the hardest wood."

Everyone has the ability to have empathy, however it is a survival trait still ingrained in many people to instinctively put themselves or someone they care about above other people or species. The king in the story was in example of this. His power and eagerness to use it to appease his wife blinded him, and it took the hoopoe's riddles to remind him to have empathy for the birds as well.

Survival of the Fittest
Human against human

The king, in a way, is a representative of people who believe themselves superior to other people based on differences of appearance or personality. He allowed himself to be inconsiderate of others in pursuit of his wife's request, to temporarily believe he was better then them, Many people are similarly inconsiderate of others when they're focused on themselves or people like them alone, and can believe they are superior to others. However, we are not genetically created to act a certain way towards others are personality only has a little basis in our genes (such as how certain genetic disorders cause behavioral issues like aggressiveness or shyness). However, the main force shaping personality is our environment. It is the influence of society and the people around us that encourage certain traits within us and suppress others.

Because of this, it would be incorrect to view our genes as racist. Racism is a social construct and racist people have learned their behaviors from society, not from their genes.  Genes are what they are - and since the diversity of humans is part of why we are so successful as a species, genes cannot be biased towards one particular variety of humans.

Power In Nature
Human against nature

The king and his mistakes could not only represent how humans sometimes view other humans, but also how we as a race view nature and the rest of the world around us. Traditionally, our role in the environment has been one of imbalance and excess. We have always been a species that strives for expansion and innovation, something which is likely the reason we have such an advanced civilization today. However, what most believe sets us apart as a species from the rest of nature is our ability to understand and not by restricted by our place in nature.

Unfortunately, as with the king, the power and perspective we have has, in a way, blinded us to the impacts we have made on the environment and our fellow inhabitants of it. Like how the king intended to remove an essential part of an entire species, humankind have caused 322 species extinctions in the past 500 years. We have treated the environment we live in horribly in our quest for advancement, and only recently have we begun to realize the impacts of what we've done.

It is our job, as the species that created the damage, to use the influence of our power to have empathy and fix what we have damaged. We need to speak up for the species we have not protected like we have our own and stop using harmful environmental practices. There will be no helpful hoopoe for us, reminding us to have empathy whenever we act selfishly. It is up to us to recognize when we aren't having and empathy and up to us to do something about it.

Low-Power Event-Driven Image Sensor Architectures

...
"Low-Power Event-driven Image Sensor Architectures" presentation by Laurent Fesquet, Amani Darwish, and Gilles Sicard of University Grenoble Alpes and CEA-LETI reviews different enent-driven approaches. The presentation was prepared for First International Conference on Advances in Signal, Image and Video Processing held on June 26 - 30, 2016 - Lisbon, Portugal. Few slides from the presentation:

11:09 AM

Toshiba Develops Switchable 2D/3D Sensor

...
Toshiba develops an image sensor that can be switched between 2D and 3D imaging modes. To get a switchable microlens array for 3D light field mode, Toshiba sensor uses a liquid crystal for 8,000 gradient-index (GRIN) lens array. The lenses have 50um pitch and can be switched between 2MP-resolving 2D and a depth-resolving 3D modes by application of +/-4V control voltage:

12:38 PM

Sunny Report

...
Guosen Securities publishes its research report on Sunny Optical - Chinese maker of lens and camera modules. Few quotes:

"Aside from rapid pixel migration (8MP & above over 70% in 1H16), Sunny will benefit from the rapid adoption of dual camera modules in FY16-18. In 2016, we already see Huawei, LG and Xiaomi adopting dual cameras ahead of Apple and Samsung and we expect other competitors very likely to adopt the feature in their upcoming flagship phones to stay competitive. We expect 4% adoption in FY16 rising to 23% through FY18. Beyond that, we predict smartphones will adopt even more cameras to support 3D and augmented reality functions. We forecast HCM revenue to rise at 24% CAGR through FY16-18 while margins should be on a mild uptrend supported by dual camera ramp up."

11:50 AM

Fracking and Earthquakes: The Tightest Link Yet

...

Stanford scientists have found the best evidence so far that injections of wastewater from hydraulic fracturing (fracking) oil and gas wells definitely cause earthquakes.  The next question is, how will the Texas Railroad Commission and the oil and gas industry respond?  But first, the scientists' study.

As readers of this blog may know, fracking involves the injection of special mixtures of water and proprietary stuff at extremely high pressure into specially drilled wells that penetrate oil- and gas-bearing formations which normally would not produce enough to be worth drilling into.  The producing wells are not the problem.  The problem is that a byproduct of the process is a huge amount of wastewater contaminated with salt, chemicals, and sometimes even radioactive stuff, and these days you don't just dump it out on the ground or into a nearby stream.  The drillers gather it up with tank trucks and ship it to disposal wells, where it is squirted several kilometers deep into rock formations under tremendous pressure. 

It's these disposal wells that seem to be associated with spates of earthquakes in north Texas and Oklahoma, which up to 2000 or so were some of the most earthquake-free areas in the U. S.  Fortunately, most of the earthquakes have been small—around 3 on the "moment magnitude" scale, which replaced the old Richter scale in the 1970s.  But a 4.8-magnitude quake on May 17, 2012 in the East Texas town of Timpson (about halfway between Lufkin and Longview) knocked down a brick wall, and turned out to be the largest such quake ever recorded in that area in recent times. 

Stanford geologist William Ellsworth, working with an international team of geophysicists, remote sensing experts, and others, decided to build a model of the subsurface rocks to see if they could reproduce the conditions that may have led to the earthquake.  Fortunately, that part of Texas is well-understood geologically, and Ellsworth's team obtained data on how much wastewater was injected into two pairs of wells, each at a different depth.  They also found and enhanced satellite-radar data that can measure movement at the earth's surface as slight as 1 millimeter per year.  They put all this data into a "poroelastic layered Earth model," meaning they accounted for porosity and elasticity—how holey and how flexible the rocks are.  They also knew about existing faults, and ran their model to predict both how much the surface might bulge after getting some 800,000 cubic meters of wastewater injected into it per year for several years.  Then they compared their model's predicted bulge to the measured bulge, which was several centimeters, and got pretty good agreement between their model and the actual satellite data.

That told them that another number their model produced—the increase in pore pressure—was also probably right.  When pore pressure increases by about 10 times atmospheric pressure (1 megapascal or more), this has been shown to cause earthquakes.  The mechanics are complicated, and I'm not a mechanical engineer.  Basically, the reason fault lines under pressure don't slip is that there is a lot of force squeezing the two sides together, and the resulting friction keeps things stationary.  But when you have increased pore pressure on the order of 1 megapascal, that somehow decreases the squeezing force and the thing starts to slip.  And slip it did, causing Timpson's quake and others.

Although most of the bulging occurred around the eastern pair of wells, the western wells were where the earthquake happened.  Ellsworth's team could explain this by citing differences in the porosity and elasticity of the rocks around each set of wells. 

So the scientists have made a model of the rocks under Timpson, injected their rock model with wastewater, and observed both a surface bulge that matches what satellites actually measured, and noted pore-pressure changes of a size that is known to cause earthquakes elsewhere.  And in fact, an earthquake happened.  Looks pretty conclusive to me.  But I'm not a Texas Railroad Commissioner.

What have railroads got to do with oil and gas production?  It's a long story, but basically, the Texas Railroad Commission (TRC), which originally did regulate railroads, backed into the business of granting permits for oil and gas production in the 1930s, and as time went on nobody has had the temerity to change its name.  It apparently did some useful work in the 1930s by putting the brakes on absurd overproduction and keeping oil prices from vanishing.  Nowadays, its regulatory duties are different, and involve environmental concerns as well as the usual support and encouragement of the industry it is charged with regulating. 

In reports describing the Stanford study, attempts by reporters to get a reaction out of the TRC were initially unsuccessful.  The Commission's mission statement has three bullets, saying it serves Texas through (1) "our stewardship of natural resources and the environment" (2)  "our concern for personal and community safety" and (3) "our support of enhanced development and economic vitality for the benefit of Texans."  Judging by the Commission's past reluctance to admit any causal link between fracking and earthquakes, their mission statement's bottom line, about enhancing development and economic vitality, appears to be taking precedence over the other two items, just as a company's bottom line tends to take precedence over other concerns.

Ellsworth and company have confirmed what many other geologists, as well as numbers of ordinary citizens, have been suspecting for a long time.  Most, if not all, of the increased earthquake activity in regions near wastewater injection wells can probably be attributed to those wells. 

By and large, Texans are reasonable people.  Fracking has been an economic blessing to many parts of the state, and it's unlikely that anything like the blanket fracking bans in New York and Maryland could happen here.  But now that there is reasonably good evidence of the connection between wastewater wells and earthquakes, it would only be reasonable for people who have lost property or been injured in such events to ask for compensation from the owners of the wells.  Of course, any time lawyers get involved, reason may fly out the window, but I think we can work these issues out without either continuing to deny that there's any association at all, or saying that fracking is an invention of the Devil and must be abolished from the planet.  Let's hope so, anyway.

Sources:  I referred to a report published online by the Dallas Observer on Sept. 23, 2016 at http://www.dallasobserver.com/news/new-scientific-study-cites-direct-evidence-that-texas-quakes-are-manmade-8736998, one in the Dallas Morning News at http://www.dallasnews.com/news/state/headlines/20160923-texas-quakes-caused-by-injection-wells-scientists-determine.ece, and the paper by M. Shirzaei, W. L. Ellsworth, K. F. Tiampo, P. J. González, and M. Manga, "Surface uplift and time-dependent seismic hazard due to fluid injection in eastern Texas," Science, vol. 353, Issue 6306, pp. 1416-1419, as well as the Texas Railroad Commission website www.rrc.state.tx.us.

Snapchat Launches Spectacles with Camera

...
BBC, WSJ, Business Insider, Snap: Snapchat launches its first hardware product, a camera equipped sunglasses named Spectacles. Simultaneously the company changes its name to Snap to reflect that its activity expands beyond the smartphone application. Now, Snap home page says:

Snap CEO Evan Spiegel wearing Spectacles

Spectacles’ camera uses a 115-degree lens, said to be much closer to the eyes’ natural field of view. On a tap on the button near the hinge, it records up to 10 sec of video from your first-person vantage (30 sec, according to BBC). Each new tap records another clip. Spectacles will be available this fall for $130.

One of the differentiators of the Spectacles camera is the round format of its video output. The rectangular format is said to be the one of the past when photos were made to print on paper. Now, they are used in a different way:



2:53 AM

The Threat Of Arctic Albedo Change

...
Arctic sea ice extent in 2016 was the lowest since satellite measurements started, when extent is averaged over the period from March 20 to September 22, as illustrated by the image below.


As the added trend also illustrates, this decline in Arctic sea ice extent looks set to further accelerate and result in a dramatic fall in albedo. The trend points at zero sea ice over this entire period in less than two decades from now.

Zero sea ice on a single day looks set to occur much earlier; a similar trend points at minimum sea ice extent reaching zero in about a decade from now, as illustrated by the image below.

Above image also shows average sea ice extent data for the period January 1 to September 22, i.e. the year to date (blue line). The added trend points at zero being reached in 2037. The data show that Arctic sea ice extent also was the lowest since satellite measurements started, when extent is averaged over the period from January 1 to September 22.

Finally, the image also shows data for the average sea ice extent over the entire year. Data for 2016 are not available yet, but it does look like 2016 will also be have the lowest sea ice extent when averaged over the entire year.

Anyway, the period between the equinoxes of March 20 and September 22/23 is most important, as the Arctic receives most sunlight during this period. This is illustrated by the image on the right and by he image below, from an earlier post, which further shows that the amount of solar radiation received by the Arctic at the time of the June Solstice is higher than anywhere else on Earth.


Thick sea ice covered with snow can reflect as much as 90% of the incoming solar radiation. After the snow begins to melt, and because shallow melt ponds have an albedo (or reflectivity) of approximately 0.2 to 0.4, the surface albedo drops to about 0.75. As melt ponds grow and deepen, the surface albedo can drop to 0.15, while the ocean reflects only 6% of the incoming solar radiation and absorbs the rest.


As Professor Peter Wadhams, University of Cambridge, once calculated, a collapse of the sea ice would go hand in hand with dramatic loss of snow and ice cover on land in the Arctic. The albedo change resulting from the snowline retreat on land is similarly large as the retreat of sea ice, so the combined impact could be well over 2 W/sq m. To put this in context, albedo changes in the Arctic alone could more than double the net radiative forcing resulting from the emissions caused by all people of the world, estimated by the IPCC to be 1.6 W/sq m in 2007 and 2.29 W/sq m in 2013.

Professor Peter Wadhams on albedo changes in the Arctic
Collapse of the sea ice could occur even faster than decline of sea ice extent may indicate.

Rapid loss of sea ice thickness has taken place over the years, as discussed in a recent post. A trend based on PIOMAS volume data (preliminary for 2016) points at a collapse around December 2021/January 2022, as illustrated by the graph below.

Indeed, Professor Peter Wadhams warned about this in 2012: "global warming will increase the intensity of extreme weather events, so more heavy winds and more intense storms can be expected to increasingly break up the remaining ice, both mechanically and by enhancing ocean heat transfer to the under-ice surface."

Thin sea ice is more vulnerable to the stronger storms that can be expected to hit the Arctic Ocean during the northern summer more frequently, and they could push huge amounts of ice out of the Arctic Ocean.


The sea ice acts as a heat buffer by absorbing energy in the process of melting. In other words, as long as there is sea ice, it will absorb heat and this will prevent this heat from raising the temperature of the water in the Arctic. Once the sea ice is gone, this latent heat must go elsewhere.

As the sea ice heats up, 2.06 J/g of heat goes into every degree Celsius that the temperature of the ice rises. While the ice is melting, all energy (at 334J/g) goes into changing ice into water and the temperature remains at 0°C (273.15K, 32°F).

Once all ice has turned into water, all subsequent heat goes into heating up the water, at 4.18 J/g for every degree Celsius that the temperature of water rises.

The amount of energy absorbed by melting ice is as much as it takes to heat an equivalent mass of water from zero to 80°C. The energy required to melt a volume of ice can raise the temperature of the same volume of rock by 150º C.
This buffer is now largely gone and further decline of Arctic sea ice means that a lot more heat will be absorbed by the Arctic.

As the water of the Arctic Ocean keeps warming, the risk increases that methane hydrates at the bottom of the Arctic Ocean will destabilize. Increases in temperature due to albedo changes and methane releases in the Arctic will go hand in hand with further feedbacks, in particular increased levels of water vapor in the atmosphere.

Here's the danger: As decline of the snow and ice cover in the Arctic continues and as more methane gets released from the seafloor, temperatures will rise rapidly, triggering further feedbacks such as a rise of water vapor in the atmosphere. Keep in mind that what makes heat unbearable is a combination of high temperatures with high humidity levels. Furthermore, water vapor is a potent greenhouse gas that will further accelerate the temperature rise. Taken together, we are facing the possibility of a 10°C temperature rise within one decade.

The image below, from the extinction page, shows that we may well be on a trend that is rising even faster than the rapid temperature increases in 2016 may indicate. Indeed, a large part of global warming is currently masked by aerosols and, as we make progress with the necessary shift to clean energy, the full wrath of global warming looks set to become manifest soon.


Risk is the product of probability and severity. The risk of a 10°C temperature rise is incalculably high. On the severity dimension, the impact of such a temperature rise is beyond catastrophic, i.e. we're talking about extinction of species at massive scale, including humans. On the probability dimension, this outcome appears to be inevitable if no comprehensive and effective action is taken.


Above danger assessment adds a third dimension, i.e. timescale. A 10°C temperature rise could eventuate within one decade and this also makes the danger imminent, adding further weight to the need to start taking comprehensive and effective action, as described in the Climate Plan.


Strange Applications for Image Sensors

...
Inspired by Ig Nobel prizes this week, here is a roundup of the recent prize-worthy patent applications that incorporate image sensors:

Philips application US20160262521 "A system and a method for guiding a user during a shaving procedure" by Jacek Lukasz Kustra, Guido Theodorus Gerardus Volleberg, Stefan Pfundtner, Karl Catharina Bree, Petrus Johannes Arnoldus Hubertus Kuijten, John Cornelius Petrus Van Der Ven, Harry Broers proposes "the system comprising an imaging module configured to register an image of a part of the body of the user, an image data analyzer configured to determine a local hair-growth direction based on data in the image, a controller configured to generate instructions about a direction in which a hair cutting device is to be moved over the body in dependence on the determined local hair-growth direction, and a feedback system configured to provide the instructions to the user."


Samsung patent application US20160267895 "Electronic device, method for recognizing playing of string instrument in electronic device, and method for providng feedback on playing of string instrument in electronic device" by Dae Young Jeon, Yeon Su Kim, Yeong Min Kim, Jung Min Park, Kyung Seok Oh proposes "The electronic device includes an image sensor configured to sense a motion of a bow to the string instrument, a vibration sensor configured to sense a vibration generated by the string instrument, and a control module configured to determine a fingering position of a user with respect to the string instrument using the motion of the bow and the vibration."


Bosch Und Siemens Hausgerate patent application US20160201981 "Refrigeration appliance with a camera module" by Stephan Kempfle, Gerhard Wetzl solves a problem of rolling shutter distortions when the refrigerator door opens. The distortions might cause difficulties in the food identification. The application proposes to rotate the camera as the door opens, so that it's always pointing to the same spot 125:

5:06 AM