NOTE
NOTE
NOTES/ALERTS
For the most current version visit www.phantomhighspeed.com
Subject to change Rev April 2016
Boosting Sensitivity
In this series of articles, we’ll explore this topic in more detail and show you how This is the first in a series of articles
to boost the light sensitivity of your digital high-speed camera when required on camera sensitivity.
by an application. And, we’ll discuss the trade-offs you will make by doing
so. We will tackle this topic at a high level and even make a few simplifying A high-speed camera’s light sensitivity
assumptions. But, don’t worry; the results will be meaningful and relevant to your can be an important specification when
everyday use of a digital high-speed camera. considering a camera purchase – especially
for applications that are “light starved”.
We’ll cover a number of key concepts in this series. Other key specifications such as maximum
resolution, or frame rates at key resolutions
are unambiguous and clear. However, light
Topic #1 sensitivity, usually specified by an ISO
Photo Site / Pixel: number, is more open to user or vendor
manipulation. Simply put, the effective ISO
The sensor used in today’s digital high-speed cameras is typically a CMOS
rating of a camera can easily be modified
technology integrated circuit comprised of millions of pixels. Each pixel (or photo
by any of several manipulations, the most
site) converts photons that strike it into electrons that accumulate charge at the
popular being the use of gain.
pixel. Think of the pixel as having a bucket that holds the electrons. The more
photons that hit the pixel in a given time, or the longer we wait under a constant
illumination, the more electrons that will accumulate in the bucket. This bucket
is called a “well”. And, the number of electrons that the well can hold before it
reaches saturation is called the Full Well Capacity (FWC) of the pixel.
In normal use, a pixel will “count” photons, turning them into electrons and
accumulating a proportional charge during the image exposure time. If only a few
electrons accumulate, that pixel will have a very small charge. If a lot of electrons
accumulate the charge will be proportionally larger. If the pixel well fills up with
accumulated electrons and starts to “overflow” it will saturate.
At the end of the exposure time, the accumulated charge in each pixel is read
out and converted into a digital number representation of the charge by an
analog-to-digital converter (A/D). The resulting number represents the charge
that accumulated during the exposure time. A low number represents black
and a large number represents white. In a 12-bit sensor, a value of 0 would be
pure black (no charge accumulated) and a value of 4095 would represent pure
white. And every digital number between 0 and 4095 would represent a different
scale of gray in-between pure black and pure white. Because this is now a
1
NOTES
when it’s too fast to see, and too important not to.
®
Boosting Sensitivity
2
At Vision Research, we use the ISO saturation-based The standard then allows the resulting ISO value to be
methodology, usually called ISOsat. rounded up to the next standard ISO number. The standard
ISO numbers range between 10 and 8000. There is a
pattern to these numbers that you can see in the last 10
valid numbers from the spec:
1000, 1250, 1600, 2000, 2500, 3200,
4000, 5000, 6400, 8000
While the spec doesn’t call out any valid numbers above
8000, we in the high-speed industry have assumed it is
okay to use this sequence, but multiply each valid number
Here is how the measurement works:
by 10. So, for modern-day high-speed cameras you will
1. Illuminate the sensor under test with a constant likely see ISO specs with values of:
and prescribed light source. The illumination can be
10000, 12500, 16000, 20000, 25000, 32000,
Daylight or Tungsten (the specific spectrum for each
40000, 50000, 64000, or 80000
of these is described in the standard).
2. Measure the lux value of the illumination. So, a measured value of 34000 will be rounded up to
3. Start exposing the (uncharged) sensor to the light and 40000 for the purposes of specifying the camera sensitivity.
measure the time it takes to reach saturation. This means that two cameras with the same specification
may actually have somewhat different native sensitivities.
4. Do some math with the lux value and exposure time
as variables in this equation: Why round up? Besides the convenience of a set of
standard values, the rounding up of a base measured value
ISOsat = 78 / (exposure * lux) helps ensure that when ISO values are used to judge/
calculate proper lighting and exposure, nothing in the image
will saturate. (Alternatively, if light and exposure are fixed, a
Pretty simple! light meter will tell a photographer what ISO rating to use to
get a good shot without the risk of saturation.) Remember,
For a given illumination level (lux), the less time it takes to
the standard is written for digital still cameras. It provides an
reach saturation (exposure in seconds), the higher the ISO
analog to the ASA / ISO ratings for film that photographers
value.
relied upon for proper film selection for a given scene.
An ISO value that is measured using Daylight illumination
will sometimes be marked with a “D”. If there is no specific
illumination noted in the value, then you can assume it
is the Daylight value. A value measured under Tungsten
illumination must be marked with a “T” in its value.
Tungsten lighting will have more light with a spectrum
toward the “infrared” wavelengths (typically, wavelengths
between about 700nm and 1100nm). Assuming the sensor
is sensitive to those wavelengths (and most monochrome
sensors are), there will be more photons available to the
3
NOTES
when it’s too fast to see, and too important not to.
®
®
Boosting Sensitivity
sensor over any given exposure time. This is good to know But, with a monochrome camera, that extra spectrum is
when using a monochrome camera in a light-starved advantageous, as we’ve discussed.
application. You can effectively boost the sensitivity of the
So, when we at Vision Research measure and specify a
camera by using Tungsten illumination.
Tungsten-based ISO value for a monochrome camera, we
It turns out that the ISO 12232 standard wants us to filter do not use the prescribed IR cut filter resulting in a value
out this extra spectrum of light before making the ISO that is more representative of the way the camera will be
measurement under Tungsten illumination! Doing so will used in real-world applications.
make the ISO value of a monochrome sensor appear to be
In this second article in our series on Boosting Sensitivity,
much less that it really is capable of. So, what is going on?
you’ve learned, at a high level, how the ISO 12232
Remember, the ISO 12232 specification is written for specification for measuring ISO of a Digital Still Camera
measuring the ISO value of Digital Still Cameras. And, works. A key point is that for a given amount of illumination,
probably 99.99% of them are color cameras. Turns out, you shorter exposure times to saturation result in higher ISO
don’t want that extra spectrum in a color camera. It causes values. We’ve also covered the two lighting methods
what is called IR contamination, which results in false colors allowed, and explained why VRI does not conform to the
on a color camera. So, an IR cut filter is used to restrict measurement standard when using Tungsten illumination
the offending light spectrum and prevent IR contamination. with a monochrome sensor.
Topic # 3
Gain If you have a light-starved application that needs more
sensitivity, you can increase the effective sensitivity by
In the previous topic we discussed the ISO 12232 method adding gain to the image. Our customers do it all the time.
for measuring ISO. At Vision Research, we make this
measurement with camera gain and gamma set to a value There are really three ways to add gain, but only one is
of “1”. That is, no gain and a linear gamma. We call that typically available to the user. The camera designers may
“native” ISO because it is the base ISO of the camera add gain to the analog value of the photo site charge before
without any manipulations to boost ISO that might cause sending it to the A/D. This is a normal design practice to
undesired side effects (such as a decrease in signal-to-noise match the charge of the full well to the range of the A/D and
ratio or dynamic range—we’ll get into those topics more in helps get maximum performance from the camera. It has
part 5). trade-offs, but they are generally good for the end user.
4
A second place to add gain is to the digital number output setting on their camera in low light situations to boost the
from the A/D before storing the pixel value as raw data sensitivity. The result, however, is often an unsatisfactory,
in the camera. Doing this means that some of the digital noisy image.
numbers that represent white will be pushed beyond the bit-
depth limit of the camera (i.e. numbers greater than 4095 Earlier, I mentioned that boosting gain to boost ISO is
for a 12-bit camera) and those pixels will appear saturated. equivalent to lowering the full-well capacity of the sensor.
The user can then adjust either the lens aperture or the Here is the logic behind that claim. Refer back to the
digital exposure time so that there is no longer saturation description of how ISO is measured and formula used from
and a nice image results. This sounds like a great solution part two of this series.
to increase sensitivity, but there are some downsides. Most Under constant illumination at some lux value, you increase
notably you now have fewer digital numbers representing the exposure of the camera until the sensor “just saturates”.
the full swing of the camera from black to saturation. This It “just saturates” when it reaches the FWC.
means less dynamic range in the image. We will also soon
learn that boosting gain is equivalent to lowering the full- If I add gain to boost the sensitivity, then the effective ISO
well capacity (FWC). And, that means a lower signal-to-noise value goes up. We have not magically created any more
ratio (SNR). Finally, the amplified values of the pixels are incoming photons by adding gain, and it takes less exposure
stored in the raw image data. So, the full dynamic range and time to saturate, so the well must be smaller. Another way
SNR of the image is now lost. Forever. to say it: we reach saturation with fewer electrons, so by
definition the well is smaller.
The third way is to use image-processing controls to
manipulate the raw image data for presentation. Adding gain
in this way does pretty much the same thing as the second
method above, but the gain is only added to the visualization
of the data. There is the same boost in apparent sensitivity.
There is the same effective lowering of the FWC with a
resulting decrease in SNR and dynamic range. But, the
full range in the raw data is protected. Only the visualized
version is changed and that can be analyzed or saved with
the gain added. And, if you ever need to go back to the
original raw data, it is available in the raw file. (In a Phantom
camera, any image processing done to the live image prior
to recording will be saved as meta-data with the raw file.
This means that you will always “see” the manipulated
image because the meta-data will be applied when you Higher gain means an effectively smaller FWC. And, the
view it or save it. But, the raw data does not change and the relationship is linear. If I had a FWC of 20,000 electrons (e-)
visualized image can be further manipulated with image- with a gain value of “1” (remember, a value of “1” means
processing controls at any time, including removal of gain “no gain” because whatever is coming into the gain equation
and recovery of the full dynamic range and SNR of the is simply multiplied by 1), and I increase the gain to a value
image.) of “2”, my effective FWC is now 10,000 e-.
The new value of ISO obtained by adding gain is called I’d really like to discuss the side effects of using gain to
the Exposure Index or EI. Anyone with a modern digital still boost ISO, but that requires a discussion about noise first.
camera has probably had the opportunity to increase the EI So that is the topic of part 4 of this series. Stay tuned.
5
NOTES
when it’s too fast to see, and too important not to.
®
Boosting Sensitivity
Topic # 5
Dark noise is the charge that accumulates with no light SNR and Dynamic Range
stimulus over long exposure times. Signal-to-noise ratio (SNR) and dynamic range are two ways
of specifying image quality. In general, the higher they are
Readout noise is really just the dark noise over very short
the nicer the image will look. And, perhaps more importantly,
time periods. It is the noise in an otherwise perfectly black
the higher they are the better the “data” you have in the
frame and is an intrinsic noise in CMOS sensors. Since
image. If you are making scientific measurements based on
6
the brightness of a pixel, or tracking particles using particle decrease SNR. This means there is less separation between
image velocimetry (PIV) or a similar technique, the quality of the signal and the noise in an image and there will likely be
the data in the image is critical. If you need to match images more visible temporal noise in the image.
from two or more cameras for 3D modeling, you want the
data to be as good as possible, for example. Dynamic Range (DR) is the range of values from the
darkest to the brightest image that a sensor can provide.
Signal-to-Noise ratio (SNR) is the ratio of the signal in Often expressed in decibels (dB) or f-stops (stops). It is a
electrons (e-) to the total temporal noise at any charge level. function of the full-well capacity and readout noise. If two
cameras have the same readout noise, the camera with the
I’ve been told that people generally stop reading articles greater FWC will have the greater DR.
the first time a formula appears. Even my early reviewers of
this article stopped reading at this point since I included a It seems like this range should be infinite--that there should
formula! Still, I’m going to stick some formulas in here, but be an infinite number of values of “brightness” between
please don’t stop reading because of them. Just skip them. black and white. But, the first limitation on that is the
number of electrons in the FWC. Even with each added
electron creating a new value, the number of values would
then be limited to the number of electrons at FWC. But, we
then use A/D converters to quantize that range of values
into discreet digital number representations. So two charge
For best image quality, it usually makes sense to set the values that are close together may get quantized into the
camera up to use its full range. So, there will usually be same value during the analog to digital conversion. Meaning,
parts of the image that are dark and parts that are bright. It the most values we can have on a perfect sensor is 2^n,
is useful then, when specifying SNR, to calculate the SNR at where “n” is the bit-depth of the A/D. (Here come some
saturation (or full-well capacity). more formulas you can skip.)
The total noise is the sum of two RMS values, one for the
readout noise and one for the shot noise. If I remember
correctly, to sum two RMS values, you first square them then
add them together then take their square root.
Boosting Sensitivity
That is pretty good considering for a high resolution digital 2 simply reduces the FWC used in the formulas from
still camera an SNR of > 40 is often considered acceptable. 26,300 to 13,150.
The resulting SNR (at saturation) is 80. The DR is 53 dB
or 8.82 stops. So, we halved the signal-to-noise ratio and
decreased our dynamic range by 1 stop. Not too bad, we
still have an acceptable image and have boosted the native
ISO from 6,400 (native ISO for a monochrome camera
under daylight illumination) to 12,800. However, this would
be expressed as an Exposure Index (EI) of 16,000 to
(I’m rounding these numbers off for simplicity). conform to one of the standard ISO values.
That is probably enough for today’s article. Just remember,
you can boost sensitivity by adding gain. Adding gain
effectively reduces the full-well-capacity of the sensor.
Reducing the FWC means you have a lower signal-to-noise
ratio and dynamic range. Reducing SNR and DR is okay,
up to the point it starts to affect your experiment by either
creating an overly noisy image or decreasing the data set
The above are all calculated with a gain of 1. Let’s boost available for data analysis.
the gain to 2 and see what happens. Boosting the gain to
We’ve covered a lot of ground and I think now is a good • An ISO rating is commonly used to specify a camera’s
time to summarize what we’ve learned: light sensitivity.
• ISO ratings are determined by following the
Important point #1: Even ISO values that conform to ISO measurement and specification methodology outlined
12232 cannot be compared directly as these values are in ISO 12232:2006 (which I’ll call just 12232 from
likely “rounded up” from some lower native value. The here on).
specification was not created with the goal of comparing • A camera’s ISO specification is not necessarily an exact
cameras. Rather, the specification was created to allow representation of the camera’s native sensitivity since
photographers a way of judging/calculating appropriate the method allows for “rounding up” the measured
lighting and exposure to mimic the behavior of a film value to a standard value.
camera and prevent saturation (that’s explains the “round
up”). The ISO specification simply tells you the maximum Important point #2: Tungsten ISO specs are valid under the
possible value of the camera’s sensitivity—the actual ISO 12232 methodology. However, the measurement method
value can be, and probably is, less. Two cameras with artificially limits the measured value for monochrome
different actual sensitivity values could have the same ISO cameras. So, while we still use the standard’s methodology
specification. The best way to compare to cameras is in a at Vision Research, we do not use the prescribed IR cut filter
8
for tungsten-based illumination of a monochrome sensor, • The application of gain has the effect of decreasing the
making our tungsten ISO specifications for monochrome total number of electrons a photosite (pixel) can hold
sensors more realistic and closer to what you will see in before saturation (called the full-well capacity).
practical use.
Important point #4: Nothing is free. While gain can be used
• The ISO 12232 method for determining a camera’s ISO to boost sensitivity, the result is a decrease in achievable
specification was written for digital still camera (DSC) SNR and DR yielding lower quality images, possibly with
evaluation. Thus it is written for color cameras. visible temporal noise.
• When doing measurements to the standard, you can • There are two “kinds” of noise in a sensor: fixed noise
use either (or both) of two specified light spectra – and temporal noise.
daylight or tungsten.
• All camera vendors work hard to minimize fixed noise
• Tungsten has more light in the near IR region and is an and the best way to compare cameras is in a side-by-
excellent lighting choice for monochrome cameras in side evaluation under identical setups.
light-starved applications because of this.
• Temporal noise is noise that is not fixed and changes
• However, the measurement method calls for an IR cut over time. And, there are two types of temporal noise
filter to be used to eliminate much of this useful light that most affect the images from a high-speed digital
spectrum in order to prevent “IR contamination” of camera: readout (or dark) noise and shot noise.
color images.
• Readout noise is the noise present in a perfectly dark
• It makes no sense to limit the light spectrum used for a image at the start of the exposure time. It can be
monochrome camera, artificially limiting its sensitivity. affected by sensor and camera design.
Important point #3: The most important point is that gain • Shot noise is the noise that accumulates during the
can be used to boost sensitivity! However, where gain is exposure time and is a function of the number of
added is important. All camera vendors will add gain to photons collected.
the analog value of the sensor to match it to the A/D range • The amount of temporal noise in an image determines
if needed. This gives the camera the ability to realize its its visual quality as well as the usefulness of the image
full range of values in the raw sensor data. Adding gain in quantitative/analytical measurement techniques.
to the digital output of the A/Ds prior to saving the pixel
value as raw data means a loss of original pixel value and • Image quality is often specified with Signal-to-Noise
permanently decreases signal-to-noise ratio and dynamic Ratio (SNR) and/or Dynamic Range (DR). Maximum
range. The third way of adding gain with image processing is achievable SNR and DR are functions of full-well
a non-destructive way to increase sensitivity. capacity and one or both noise sources.
• As full-well capacity decreases, maximum SNR and DR
• One can use gain to increase the measured sensitivity will also decrease yielding a lower quality image.
of a camera.
Now, let’s look at some examples using hypothetical
• Gain can be applied in three places: prior to analog-
cameras.
to-digital (A/D) conversion, prior to storing the digital
representation of the sensor value as raw data, or Here are the specs for two cameras that have the same
as an image processing step where gain is used to resolution, speed, minimum exposure times, etc.
manipulate the raw data for presentation, visualization
or conversion to a different file. Camera A: ISO 20000 D Mono
Camera B: ISO 10000 D Mono
9
NOTES
when it’s too fast to see, and too important not to.
®
Boosting Sensitivity
That’s it. That’s all the information you have. Two, otherwise What if I told you that the FWC for Camera A is 16,000 e-
identical cameras but one has an ISO spec of 20,000 and and for Camera B it is 26,500 e-?
the other of 10,000.
I’d hope you would then ask for one more key specification:
Which is the better camera? What is the readout noise? Because, with readout noise,
I hope you scratched your head before answering. If you are you can now apply the formulas from part 5 of this series
considering sensitivity specifications, and only sensitivity and determine maximum signal-to-noise ratio and dynamic
specifications, Camera A appears to be the “better” camera. range! (Don’t worry, I’m not going to make you do this.)
It seems to be twice as sensitive. Let’s say both cameras, since they are using the same
Now, what if I told you that the ISO measurements, before CMOS sensor technology, have a readout noise of 25 e-.
rounding up, were 16,300 and 9810 respectively? Camera Now, armed with those additional specs, which is the
A is still more sensitive by this unadjusted measure, not by “better” camera?
twice but by 60%. Not even a full stop.
Here is a table of all the relevant information:
Measured ISO Read-out Shot DR DR
Gain FWC SNR
Sensitivity Spec Noise Noise (dB) (Stops)
On paper, Camera A appears twice as sensitive. In a side-by- one stop. However, Camera B has a higher SNR and has 0.7
side comparison, it would be 60% more sensitive, less than stops more dynamic range. Which is the “better” camera?
A gain of 1.7 raises the measured sensitivity of Camera B I say “non-destructively” because if I add the gain using
from 9810 to 16,677, which we can round up to 20,000 image processing tools applied to the original raw data, I
as the valid ISO specification. This effectively reduces the achieve not only a camera output equivalent to Camera A,
FWC to 15,588 e-, pretty close to the 16,000 e- of Camera but if needed I can always go back to the raw data from
A. And, now the SNR and DR values are identical (to the Camera B, apply a different set of image processing values
precision we are calculating). and regain the higher original maximum SNR and DR!
If I can non-destructively raise the ISO of Camera B to equal This technique of adding gain to change the effective ISO is
Camera A, and still have the same SNR and DR values as a common feature of modern DSLR cameras. An adjusted
Camera A, which is the “better” camera? sensitivity rating is called an Exposure Index or EI, and we’ll
talk more about that in a subsequent article.
10
Topic #7
How to Add (Linear) Gain using PCC
We’ve discussed using gain to boost sensitivity and the There is a set of tools called Image Tools available by
trade-offs that come with that technique. Now, let’s briefly clicking on the “painter’s palette” in the top toolbar of PCC.
look at how to add gain using PCC.
Image Tools gives you a lot of control over how your image to convert the raw images to some other format such as
appears. However, all adjustments made here are non- ProRes or h.264. But, even then you have the original raw
destructive. They control how the image appears but they do cine file available if you want to go back to the raw data or
not change any of the raw data. The controls you see will be make different adjustments.
a little different between a monochrome camera and a color
camera. I’m not going to go into all the adjustments available. Let’s
just focus on two things: the histogram view of the image
Adjustments made in image tools are applied to the raw data, and the linear Gain adjustment. Here is how it looks on
data and “baked into” any file you create when you choose a monochrome camera.
11
when it’s too fast to see, and too important not to.
®
NOTES/ALERTS
In this example, the gain is set to “2”. You can make the gain adjustment by
typing the desired value in the text box. Or, you can use your mouse to drag the BOOSTING
SENSITIVITY
slider.
The histogram shows the distribution of all available digital number values of
pixels in the currently visible frame of the cine file. Remember, for a 12-bit
camera, we can have 4096 different values ranging from 0 (totally black) to
4095 (just at the point of saturation).
This is useful because as you add gain, you’ll “spread” these values out over the
full range and you will see some values approaching saturation. The histogram
helps you know when you’ve boosted gain too much and driven some values to
saturation. On the other hand, spreading the values out over the full range helps
you to achieve greater apparent sensitivity.
Focused
Since 1950, Vision Research has been designing, and
manufacturing high-speed cameras. Our single focus is to
invent, build, and support the most advanced cameras possible.
In this case, the gain is set to “3” and you can see that the values are pretty well
spread out over the full range.
100 Dey Road
Wayne, NJ 07470 USA
+1.973.696.4500
www.phantomhighspeed.com
12