Computational Photography: Geometric Optics

Quentin Bammey

How are images formed?

Camera Obscura (pinhole camera)

• Natural phenomenon, then ideal model definind perspective projection
• The hole ``selects’’ the rays that pass through it, allowing the formation of an inverted image.

Natural camerae obscurae

• Nautilus
• Solar eclipse

CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=18395466

CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=4657582

Replica images of a partial solar eclipse, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=362665

Pinhole cameras throughout history

• 1646
• 1858

Illustration of “portable” camera obscura in Kircher’s Ars Magna Lucis Et Umbra (1646)

A camera obscura tent drawing aid in physics lessons (Gagniet)

A perfect model?

A perfect model?

• Only a small set of rays from any given point will hit the screen \(\Rightarrow\) very dark images
• Pinhole too big: too many rays are averaged, blurring the image
• Pinhole too small: The image is dark, and diffraction effect also blur the image
• Pinhole pane too thin: Many thin materials are not opaque enough, light will leak from other regions than the pinhole
• Pinhole pane too thick: Rays that should represent the corners of the image will hit the pane within the pinhole instead, leading to vignetting

Lenses for a better image formation

Convex and concave lenses

• Convex lenses are converging
• Concave lenses are diverging

Snell’s law

Refractions in optics are goverened by Snell’s law: \[\frac {\sin \theta_1}{\sin \theta_2} = \frac {n_2}{n_1}\]

Snell’s law

Refractive indices

The refractive index of a medium is the ratio between the speed of light in a vacuum to the speed in the medium

Material	Refractive index
Vacuum	1
Air (0 °C, 1atm)	1.000293
Water (20 °C)	1.333
Glass	\(\simeq 1.3-1.7\)

Snell’s law in practice: Optical manhole

• Snell’s window
• explained

From underwater, we can only see the surface through a cone.

https://commons.wikimedia.org/wiki/File:RefractionReflextion.svg

Ideal lenses

• Ideal lens
• Focal points
• Focal planes

For simple lenses calculations, we usually assume:

• Thin lens: the lens’ thickness is negligible compared to the radii of curvature of the lens
• Paraxial approximation: the angles between the rays and the optical axis of the camera are small
  • Allows for simplification of trigonometric functions: \(\sin x\simeq x\), \(\cos x\simeq 1 - \frac {x^2}{2}\) (\(x\) in radians)
  • Accurate to 0.5% for angles up to \(10^{\circ}\).
• Central approximation: the rays are close to the centre of the lens.

A lens has an optical axis and two focal points on it:

• The optical axis is the line perpendicular to the lens that passes through its centre
• Rays that pass through a focal point before hitting the lens will emerge parallel to the optical axis
• Similarly, Rays that are parallel to the optical axis before hitting the lens will converge to a focal point of the lens. For divergent lenses will look as if they came from the focal point.

The focal planes are the planes perpendicular to the optical axis at the locations of the two focal points.

• Rays that pass through the same point on the focal plane before hitting the lens will emerge parallel.
• Similarly, Rays that are parallel before they pass throught the lens will converge to a point on the focal plane

Dioptric power / focal length

• dioptric power \(P\) [\(\mathrm m^{-1}\)]: power to which an optical system converges or diverges light. Also called optical/refractive/focusing/convergence power.
• Positive dioptric power = converging lens, negative dioptric power = diverging lens
• focal length \(f\) [\(\mathrm m\)]: Distance between the lens and the focal points, inverse of the dioptric power
• Both are defined using the refractive index of the lens, \(n_d\), and the lens’ two surfaces’ curvature radii \(r_1\) and \(r_2\), using the simplified lensmaker’s equation: \[ P = \frac 1 f = \left(n_d-1\right)\cdot\left(\frac 1 {r_1} - \frac 1 {r_2}\right)\]
• The full lensmaker’s equation also depends on the actual lens thickness \(d\): \[ P = \frac 1 f = \left(n-1\right)\cdot\left(\frac 1 {r_1} - \frac 1 {r_2} + \frac {(n-1)d}{nr_1r_2}\right)\]

Image formation

• The object and image size are related to the focal length: \[ \frac 1 f = \frac 1 {do} + \frac 1 {di}\]
• Scale of reproduction: \[m = \frac {hi}{ho} = \frac {di}{do} = \frac {di-f} f = \frac f {do-f}\]

Optical aberrations

Chromatic abberations

Chromatic aberrations happen when different wavelengths of light have different focal points.

This jewel has purple fringing on the bottom and green fringing on top. https://commons.wikimedia.org/w/index.php?curid=30060843

Back to Snell’s law

Angles of refraction and focal points are determined through Snell’s law:

\[\frac {\sin \theta_1}{\sin \theta_2} = \frac {n_2}{n_1}\]

The angle of refraction changes with the refractive index

Refractive index and wavelength

The refractive index of a medium actually depends on the wavelength!

Cauchy equation: the refractive index is determined as

\[n(\lambda) = A + \frac B {\lambda^2} + \frac C {\lambda^4}\]

We usually only go up to the second-order term:

\[n(\lambda) = A + \frac B {\lambda^2}\]

We can thus note that \[1 < n(\lambda_r) < n(\lambda_g) < n(\lambda_b)\]

Note: The Cauchy equation can only be used in the visible range (more complex equations are needed in the UV and IR regions)

Astigmatism

Lenses that feature astigmatism will feature different focal points for rays that propagate in perpendicular planes.

https://commons.wikimedia.org/w/index.php?curid=4799508

Spherical aberrations

In ideal lens calculations, we assume the rays pass close to the lens’ centre. In practice, this is not the case!

The focal length of a real lens actually varies depending on how far the ray is from the lens’ centre when it passes through it, causing spherical aberrations (named so because this phenomenon happens due to lenses being spherical).

Coma

Similarly, off-axis point sources will appear distorted and will have coma, as if they had a tail.

Uncorrected (left) and corrected (right) astro-photography with coma. https://commons.wikimedia.org/w/index.php?curid=24971290

A single lens can be chosen to have no coma only at a set distance from the lens.

Avoiding aberrations

• Some aberrations (e. g. astigmatism) are caused by defects in the lenses: high-quality lenses will feature fewer aberrations
• Some aberrations (e. g. coma) can only be removed at a set distance, or not at all: combine several lenses to remove/minimize them
• If all else fails: digital processes can correct these aberrations

Chromatic aberration correction

Combining lenses against aberrations

An achromatic doublets realigns the red and blue focus. https://commons.wikimedia.org/w/index.php?curid=10721236

Photographic lenses

A typical photographic lens

Characterizing a photographic lens

Photographic lenses combine several elements. To characterize them, we specify their:

• Focal length
• Angle of view
• Aperture (depth of field)
• Resolving power

Angle of view

Angle of view: Angular extent to which a scene is imaged by the camera

Angle of view / perceived perspective

• Human eye perspective: \(\simeq 50^\circ\) vertically
• Compressed perspective: a smaller angle of view (\(<50^\circ\)) will compress the perspective: telephoto lens
• Extended perspective: a larger angle of view (\(>50^\circ\)) will show a wider portion of the scene than the human eye: wide angle lens

Vignetting

If the lens does not fit the sensor, there is vignetting on the image

Vignetting

If the lens does not fit the sensor, there is vignetting on the image

Sensor format

• The standard sensor format, called full-frame sensor, has a sensor size of \(24\times36\) mm
• On a full-frame sensor, a normal perspective lens will have a focal length of approximately 50mm
• On the same lens, smaller sensors will only receive the centre crop of the lens output, thus compressing the perspective: you have to calculate the full-frame equivalent

Sensor format

https://commons.wikimedia.org/w/index.php?curid=3163749

Angle of view

https://commons.wikimedia.org/w/index.php?curid=1052034

Angle of view and compression

Smaller angles of view compress the image, larger angles of view (smaller focal length) extend it and make the front details more prominent

Angle of view and compression

Smaller angles of view compress the image, larger angles of view (smaller focal length) extend it and make the front details more prominent

Angle of view and compression

Smaller angles of view compress the image, larger angles of view (smaller focal length) extend it and make the front details more prominent

Depth of field

{width=80%)

Circle of confusion

Only the points that are on the object plane will be imaged as a point. The points before or behind will be imaged as circles of confusion.

Circle of confusion and aperture

Only the points that are on the object plane will be imaged as a point. The points before or behind will be imaged as circles of confusion.

Closing the aperture reduces the amount of light that is imaged (thus the image is darker), but also reduces the sizes of the circles of confusion.

Depth of field and aperture

Image taken with an aperture f/22, https://commons.wikimedia.org/w/index.php?curid=118009180

Image taken with an aperture f/1.8, https://commons.wikimedia.org/w/index.php?curid=118009181

Depth of field and aperture

The diaphragm of a lens, or f-number (commonly written as f/#: f/22, f/1.8, etc), is the ratio of the focal length and aperture diameter: \[N = \frac f d\]

A larger f-number means the aperture is smaller

Different apertures on a lens, https://commons.wikimedia.org/w/index.php?curid=78136658

Knowing the depth of field

• The depth of field is the distance between the nearest and the furthest object from a lens that will be in focus.
• Only objects on a specific planes are imaged as real points: is the depth of field null?

• We need human measurements to know how large a circle a confusion can be to still look like a point: the maximum permissible circle of confusion
• One possible standard: largest circle of blur on a sensor that will still be perceived by the human eye as a clean point, when printed at 30 cm (diagonal size) and viewed at a distance of 50cm

Hyperfocal distance

• The hyperfocal distance is the distance with the largest depth of field
• It is the closest distance at which a lens can be focused while keeping objects at infinity sharp
• When the lens is focused at this distance, all objects from half the hyperfocal distance to infinity will be sharp

Depth of field and hyperfocal distance

• The depth of field varies proportionnally to the f-number (larger f-number \(\Rightarrow\) smaller aperture \(\Rightarrow\) larger depth of field)
• It varies proportionnally to \(\frac 1 {f^2}\)
• It varies proportionnally to the square of the focused object distance
• The diameter of the maximum permissible circle of confusion \(C\) can be obtained from the f-number \(N\), the focal length \(f\) and the hyperfocal distance \(H\): \[C = \frac {f^2}{N\cdot H}\]

Extending the depth of field

Lightfield / plenoptic cameras capture the direction of the incoming rays in addition to the value, thus they can reconstruct a perfect depth of field

Characterizing the lens resolution

Point spread function (PSF)

PSF: measured response of a system to a point source input, https://commons.wikimedia.org/w/index.php?curid=877065

Diffraction of a point source

Due to diffraction when the light crosses the aperture, a point source will render as a spot rather than a perfect point

Angular resolution

Rayleight criterion for the minimum resolvable detail:

• A system is said to be diffraction-limited when the first diffraction minimum of one source point’s image coincides with the maximum of another
• Angular limit of resolution: \(\Delta\theta = \frac{1.22\lambda}d\)
• Spatial limit of resolution: \(\Delta l_{min} = \frac{1.22f\lambda}d = 1.22 N\lambda\)

Due to diffraction when the light crosses the aperture, a point source will render as a spot rather than a perfect point

Diffraction-limited system

Sharpness

The sharpness of a camera is the apparent contrast at its edge.

{width=“80%”}

Resolution

• Limiting resolution is the highest frequency at which light and dark parts of an image can be distinguished
• Evaluated visually or through an instrument, depending on the application
• The sampling interval is the distance between adjacent pixel centers
• The sampling interval sets the Nyquist limit, the highest resolvable spatial frequency, as the inverse of the sampling interval

Sampling area

Large sampling areas result in higher sensitivity, less aliasing, but less apparent sharpness

Visual Evaluation of a system

Modulation Transfer Function (MTF)

• The MTF shows how well the original contrast has been captured
• Depends on the whole system (optics + sensor)
• Several contrast metrics, such as the Michelson contrast: \[M = \frac {L_{max}-L_{min}}{L_{max}+L_{min}}\]

Line Spread Function (LSF) and Edge Spread Function (ESF)

Line Spread Function:

• 1D response of a system to an infinitely long line source
• Convolution of the PSF with an infinite line

Edge Spread Function:

• 1D response of a system to a straight boundary between two infinitely long regions
• Convolution of the PSF with a step function

LSF (top) and ESF (bottom)

Spatial Frequency Response (SFR)

• MTF for a sampling system
• Measured from sampling a tilted edge
• Discrete Fourier Transform of a single LSF

Tilted square test target

Test targets for measurement