Predicting therapy resistance in tissue slides: a reminder of AI’s power in cancer image analysis

In 2020, widely read outlets such as Scientific American highlighted a striking achievement: Google’s artificial intelligence (AI) system could outperform clinicians in detecting breast cancer from mammograms (www.scientificamerican.com/article/google-ai-tool-can-pinpoint-breast-cancer-better-than-clinicians/). The original study, published in Nature (www.nature.com/articles/s41586-019-1799-6), quickly became emblematic of a new era in medical AI in which algorithms could not only support clinical expertise but sometimes surpass it. Five years on, that breakthrough continues to shape oncology, although some of its most significant impact is emerging in adjacent areas.

Breast cancer is unusual among solid tumors because millions of women undergo routine imaging annually, creating an abundance of data. This allowed Google’s model to learn from a vast library of images. In most cancers, however, the situation is different. Clinicians lack a simple, repeatable visual snapshot of disease progression. Instead, the only consistent window into these tumors comes from biopsy tissue processed into thin slides. Remarkably, tissue slides may hold far more information than any mammogram.

Projects like CancerScan build on the principle that the tumor microenvironment, a dense, chaotic ecosystem of cancer cells, immune infiltrates, blood vessels, and stromal structures, encodes key information for predicting resistance evolution. Unlike mammograms, which show a silhouette, histopathology reveals the microscopic terrain where cancer learns to outmaneuver therapy. Every pixel offers environmental cues, from gradients of hypoxia to pockets of immune suppression and architectural bottlenecks where resistant clones may take root.

Yet image data alone is limited. Decades of biological research hold much of what we know about cancer, its pathways, vulnerabilities, and escape strategies. Knowledge graphs offer a way to fuse these domains, linking what the tissue shows to what science has already uncovered. In CancerScan, this approach links histopathology with existing knowledge giving AI a deeper anchor in cancer biology.

Although the original AI-mammography breakthrough dates to 2020, its influence endures. That study revealed AI’s ability to detect subtle image signatures of disease. CancerScan extends this vision by leveraging far richer image sources and using them to construct digital twins of each patient’s tumor. These twins capture microenvironmental structure and evolutionary dynamics, enabling resistance trajectories to be forecast before they appear. This shift has the potential to move oncology from reactive treatment to truly anticipatory care.