Optical computing for augmented-reality microscopy

Augmented-reality microscopes overlay information on what the user sees, for example by highlighting cells of a certain size or shape. Doing that with a camera and a computer adds processing time, which limits real-time use.

We do the selection optically. A phase mask, computed in advance by optimising it against a physical model of the microscope, is displayed on a spatial light modulator placed in a Fourier plane of the imaging path. Objects that match the target shape or size are highlighted, and the highlighting follows them as they move through the field of view, with no continuous image processing or tracking.

The paper was selected for the cover of Optics Continuum, Volume 5, Issue 8 (August 2026).

The setup is built around a Zeiss Axio Observer 7 with a 655 nm laser and a 4f relay to the modulator. The project page has the full method, results and a video.