A more precise method of observing the microworld

“Our team of Quantitative Optical Imaging operating at the Institute of Micromechanics and Photonics in the Department of Micromechanics and Photonics at the Faculty of Mechatronics of the Warsaw University of Technology is working on utilizing optical measuring methods to test objects on a microscale,” says Dr Maciej Trusiak. “Inspired by the work of Professor Małgorzata Kujawińska’s team, we are mainly testing transparent objects, which do not absorb light, including most biological and part of technological objects,” he explains.  

As it is impossible to make observations of the researched objects with a naked eye or a camera, scientists are using phase microscopy enabling them to increase contrast, by intensifying the sensitivity to refraction, not its absorption in the samples. This facilitates observing objects without interfering with them, e.g. through fluorescent colouring. This method was invented by Frits Zernike, a Nobel Prize winner. WUT researchers utilize a modern quantitative version of classic phase microscopy.  

“We can very precisely measure phase delay introduced by a sample using techniques of interferometry and digital holography. Endogenic contrasting factor provides us with a distribution of refractive index combined with the optical thickness of the researched object,” explains Dr Trusiak.

Transition phase

It is necessary to record a band image to move from intensity to high-contrast phase delay. The image shape encodes the information on the searched phase distribution. The more locally bent band, the higher the phase change. Scientists at the Faculty of Mechatronics at WUT are working on increasing the decoding bandwidth of this information.  

“The method of quantitative phase microscopy has its physical and numerical (computing) limitations,” explains Dr Trusiak. “Striving for the best phase imaging resolution, we must consider the physical resolution and numerical phase recovery bandwidth. In our work we focus mainly on the numerical aspect of increasing the phase imaging bandwidth to be able to image the tiniest details of the researched object,” he adds.

The team of scientists utilizes the Hilbert transform method which handles well the overlapping spectrum, enabling one to produce a more detailed resulting phase image. The method requires processing the initial band distribution. WUT scientists implement it in two ways.

“Our objective was to improve the quality of initial processing with simultaneous use of modern neural networks and methods of image decomposing. We are introducing algorithmic novelties to work on both platforms,” says Dr Trusiak.

Algorithm work

Single-frame algorithm of band image analysis - Variational Hilbert Quantitative Phase Imaging.

To achieve the intended objective, scientists determined a map of the density and orientation of bands (indispensable for the correct application of Hilbert transform) based on convolutional neural networks, among other things. The team from the Faculty of Mechatronics has applied a learning set based on computer simulations solely, which is rare for phase recovery.

“We were familiar with assimilated phase distribution so we knew what the neural network output should look like,” – explains Maria Cywińska, MSc. “With well-defined input and output of a neural network, during the learning process, instead of knowing the dependence between the band image and density map or orientation in advance, we can use a network which acts as a “programmer” and defines that dependence,” she explains.

An example of analysis of real data (upper rectangle) and maps of band density obtained by a neural network (lower rectangle) in the learning process in simulations.

To make this happen, the scientists had to define the architecture of a convolutional neural network which would allow it to learn to function based on smaller “blocks”, distributing the image into smaller parts. 

Network architecture

“Our network did not follow one pathway during the learning process. We divided it into a few pathways, changing this way the dimensionality of the image and letting the network divide a big “problem” into smaller “problems”. As a result, in each branch the network could focus on different areas of the tested image,” she explains.  

Networks trained this way can be used to filter a band image: the noise is reduced, a coherent band module is extracted and incoherent background, which is a sum of the intensity of two bands used for creating a band image, is removed. The variable decomposition method, devised by Maria Cywińska, MSc, is used for removing the background. What does it involve?

“From iteration to iteration we approach the solution we are looking for. Depending on the change in the character of band images, the number of iterations we need to compute to select the bands from the background and noise also changes,” explains the doctoral student. “So far we have used the stop criterion for those computations. Unfortunately, it drastically prolongs the time of these computations.”

In the analysis, Maria Cywińska, MSc, decided to use a network through teaching it to map the behaviour of the stop criterion in the set of band images.

“Once taught, a network will determine the number of necessary iterations in a split of a second and will hugely accelerate these computations in future applications,” says Maria Cywińska, MSc.

According to the scientists at the WUT Faculty of Mechatronics, the new method reduced the filtration time even by 80%.

The computations using the network show overestimation in the number of iterations. However, with so many iterations, this does not affect the quality of filtrations.

Observations of fish

Scientists from the Faculty of Mechatronics could share their experience and knowledge with the researchers from the team of Professor Balpreet Ahluwalia from the Arctic University of Norway in Tromso, who are observing fish cells in terms of their movements and how they are affected by microplastic contamination.

So far fish cells have been thought to be moving along pathways resembling scooping water. Thanks to the collaboration, that concept has been disproved. Scientists could observe the behaviour of cells thanks to a dynamic phase image and did not notice any movements like that during their observations. Their observations prove that the movement of fish cells resembles drifting. 

Continuation of work

Scientists are still developing neural networks both in terms of theory, architecture, and new applications. They focus on improving decomposition methods in terms of mathematical foundations, from numerical implementation to applications in phase microscopy.

“We are doing all of this to be able to see more under phase microscopes without practically physically changing them,” says Dr Maciej Trusiak. “We like to think that we create fully numerical apertures which increase the information bandwidth of phase microscopes without interfering with them. Algorithmically, we are increasing their capacity which is the foundation of a dynamically developing field in computational microscopy,” explains the scientist.

Cheek epithelial cells – the same sample. Left: close-up with limited resolution. Right: close-up under a microscope with algorithmically increased capacity. Photos from a computational microscope utilizing the method of Fourier ptychography.

However, that is not all the researchers are working on. Doctoral student Mikołaj Rogalski, MSc, and adjunct Piotr Zdańkowski, DSc, have constructed a system of a ptychographic microscope and written special software for it. “Thanks to that, we can not only introduce quantitative phase contrast but also increase its resolution through lighting a sample from many angles and numerical “stitching” of all information,” explains Dr Trusiak.

Scientists are planning to develop methods of neural networks and image decomposition to facilitate faster and more precise reconstruction in ptychographic microscopy. “Currently, it is one of the biggest challenges in computational microscopy and has a direct relationship with information bandwidth which we dealt with within the research grant awarded in the FOTECH-1 competition,” says Dr Trusiak.

"DEEPCOMPOSITION: Advanced data decomposition techniques aided by deep learning algorithms for enhancement of the space-bandwidth product of selected full field optical measurement methods." is funded within the grant of  Priority Research Photonic Technologies under the programme "Research University – Excellence Initiative" implemented at the Warsaw University of Technology.

Research team:

Maciej Trusiak, DSc, Professor Krzysztof Patorski, PhD, DSc, Piotr Zdańkowski, DSc, Julianna Winnik, DSc, Maria Cywińska, MSc, Mikołaj Rogalski, MSc, Filip Brzeski, MSc, Wiktor Krajnik, BSc