The 3D reconstructed shape of a tiny (0.8 by 0.3 mm) surface mount resistor on a printed circuit board. This was made using only a microscope; no fancy laser scanning required!
3D scanning through a microscope is a bit different to normal 3D scanning; mostly because when you look down a microscope at an object it looks very different to what you might expect from day-to-day life. The most immediately obvious effect is that out of focus areas are very out of focus, often to the point where you can barely see what is there. This effect comes down to the angle over which light is collected by the lens capturing the image; your eye or a camera lens in everyday life, or an objective lens when using a microscope.
Three images of the surface mount resistor. The three pictures are taken at different focus distances so different parts of the image are clear and others blurred. The blurred parts are very blurred!
In every-day-life when using a camera or your eyes distance from the lens to the object is normally long, it may be several metres or more. As a result the camera/your eye only collects light over a small of angle, often less than one degree. In comparison microscopes collect light from an extremely large range of angles, often up to 45 degrees. The angle must be this large because the objective lens sits so close to the sample. A wider angle of light collection makes out of focus objects appear more blurred. In photography terms the angle of light collection is related to the f-number, and large f-numbers (which have a large angle of light collection) famously have very blurred out of focus portions of the image.
The upshot of this is that in a microscope image the in focus parts of an image are those which lie very near (often within a few micrometers) to the focal plane. It is quite easy to automatically detect in focus parts of an image by using local image contrast (this is actually how autofocus works in many cameras) to map which parts of a microscope image are perfectly in focus.
In this series of images the most in-focus one is image 6 because it has the highest local contrast...
... using edge detection to emphasise local contrast in the image really highlights which one is perfectly in focus.
In this series of images the most in-focus one is image 55 instead.
The trick for focus 3D scanning down a microscope is taking the ability to detect which parts of an image are in focus, and using this to reconstruct the 3D shape of the sample. Going to the 3D scan is actually really easy:
It may be a simple idea, but this method gives a high quality 3D reconstruction of the object.
- Capture a series of images with the focus set to different distances.
- Map which parts of each of these images are perfectly in focus.
- Translate this back to the focus distance used to capture the image.
This concept is very simple; if you know one part of an object is perfectly in focus when the focus distance is set to 1mm, that means it is positioned exactly 1mm from the lens. If a different part is perfectly in focus when the focus distance is 2mm, then it must be positioned 2mm from the lens. Simple!
It may be a simple idea, but this method gives a high quality 3D reconstruction of the object.
The reconstructed 3D shape of the resistor, using 60 images focused 0.01mm apart, mapped to a depth map image. The lighter bits stick out more from the surface, and the darker bits stick out less.
Using the depth map to reconstruct the resistor reconstructed in full colour in 3D! Pretty cool for something less than 1 mm long...
Does that seem impressive? Then check out the videos:
A video of the original focus series of images captured of the resistor.
The reconstructed 3D shape.
A 3D view of the resistor, fully textured.
This approach is, roughly speaking, how most 3D light microscopy is done in biological and medical research. It is very common practice to capture a focal series like this (often called a "z stack") to get this 3D information from the sample. 3D imaging is most useful in very thick samples where you want to be able to analyse the structure in all three dimensions, an example might be analysing the structure of a tumour. My research on Leishmania parasites inside white blood cells uses this approach a lot too. The scanning confocal fluorescence microscope was actually designed to maximise the value of this 3D effect by not only blurring out of focus parts of the image, but also eliminating the light all together by blocking it from reaching the camera.
Software used:
ImageJ: Image analysis.
Blender: 3D viewing and rendering.
