6/12/21 – DeepMirror Spark for single instance segmentation in biomedical images
Written by Max
Almost all biomedical image analyses start with the acquisition of a few images, be it a clinical trial in which MRI scans are acquired from patients or a laboratory trying out new treatments on cultured cells. To extract clinical & scientific insights from these images, researchers and clinicians could benefit from recent advances in Artificial Intelligence (AI), for example to automatically detect specific regions of interest in these images, or to automatically classify images into multiple categories. While these tasks can be carried out reliably by human experts, this takes a lot of time and money. AI can help but needs to be trained with giant datasets that have been painstakingly curated by hand. Human-driven data curation (also called annotation) often take months or years and poses a barrier to the application of AI to new problems.
But what if one could reduce the number of annotations required to train AI? With reduced annotations, researchers & clinicians would be able to get AI up and running for new applications in no-time, accelerating breakthroughs in the biomedical sector. This would have tremendous real-world applications since while the amount of raw data is often vast, the number of validated data is not. At DeepMirror, we set out to tackle this challenge by considering how humans learn. To learn to recognize cats, for example, humans do not need a giant dataset of annotated cat images. Often a few are enough and by cross referencing with other images a human can accurately locate cats in their vicinity. We wondered if we could replicate this with an AI technique called semi-supervised learning. In semi-supervised learning, AI is trained with a few known & validated datapoints, as for example a few images in which cats have been annotated, and a large dataset of non-annotated examples. After almost 2 years of iterating on semi-supervised learning algorithms, we built a modern framework that focusses on real world applicability, i.e., is easily adjustable for any kind of AI task.
Meet DeepMirror Spark, a semi supervised AI training platform that uses specialized data augmentations (i.e. distorting data so that it “looks” like new data), adversarial learning (using two competing AI with the objective of improving AI performance), and other techniques, to decrease the number of required annotations by up to 100 times!
Our early clients often struggled with instance segmentation, which is the accurate detection of individual object shapes from images. Instance segmentation can for example be used to detect the size of cancers in radiology images and the abundance of biomarkers in histopathological images. To test and benchmark our DeepMirror Spark platform, we applied it to a customized biomedical instance segmentation architecture based on the popular UNet (Figure 1).
AI training performance
This UNet-based implementation of DeepMirror Spark alone can be used for a myriad of different image analysis tasks. In Figure 2 & 3, for example, we trained a UNet to detect Islets of Langerhans in human pancreas images with just 46 annotated images, and brain tumors in MRI scans with only 75 images. In both cases conventional (i.e., non semi-supervised) training led to lower training performance and worse segmentation.
Ablation Studies on radiology & cell biology image data
To test DeepMirror Spark’s performance as a function of annotated images we performed ablation studies, i.e. we took several publicly available and internally generated image datasets and varied the number of labelled images. In doing this experiment we were able to quantify the improvement that DeepMirror Spark has over conventional training, for small datasets (Figure 4-5).
We observed that DeepMirror Spark enabled us to reach the maximum Average Precision (AP) score with only 5-300 images depending on complexity of the data. Conventional training of the same network failed to reach the same AP scores with any number of images from a small dataset.
What’s next
We are now working on extending the DeepMirror Spark platform to perform other tasks beyond instance segmentation. In the next few weeks, we will add image classification, semantic segmentation and more. Additionally, we are working on integrating other datatypes such as genomic sequences and molecular structures for gene editing experiments and drug discovery! Stay tuned for further blogposts on all these areas 🙂
If you want to add AI to your projects in the fastest way possible, get in touch with us at enquiries@deepmirror.ai to do a pilot study on your dataset!
Figures
