Does computer vision technology hold the key to beating cancer?

We know that computer vision has been used in medical imaging in many different fields. There are many tasks that we can do with computer vision in medical imaging, such as image segmentation, classification, and object detection. And we see these applications every time we go to the doctor.

For example, when ophthalmologists get images, they can tell if somebody has diabetes or not from the vessels in their eyes. When we go to a dermatologist, they can detect with imaging techniques if somebody has skin cancer or not.

One of the first and most important applications of computer vision in medicine was radiology. There are tons of radiology images; CT scans, MRIs, and ultrasounds. And the purpose of them is to visualize the internal organs for us.

We have many different types because we want to understand different tissues better. And one of the most important uses of radiology images is for tumor detection.

Computer vision tools help medics find out where the tumor is, what type of tumor it is, and make a proper plan for treatment.

The role of computer vision in radiotherapy

Now, let's say that a doctor acquires radiology images from a patient and understands that the patient has cancer. They then recommend radiotherapy for the treatment of this patient. While there are other ways that we can treat cancer, one of the main ones is radiotherapy or radiotherapy combined with other types of treatment.

How do we use radiotherapy to cure cancer? And how do we use these imaging techniques to help us to understand the behavior of these cancer cells better?

Radiotherapy or radiation treatment uses high doses of radiation to kill cancer cells and shrink tumors. Radiotherapy is conducted by devices called linear accelerators or LINAC.

LINAC conducts radiation treatment by sending different particles. They have large amounts of energy, and they try to release the energy when they hit the cancer cells. So we try to kill the cancer cells and also try to slow down the growth of these cells by damaging their DNA. The thing that’s important to note is that during this treatment, we want to kill cancer cells but at the same time avoid harming healthy cells.

The image below will help you to better understand the whole process of radiotherapy, where the imaging data that we’re getting from this machine comes from, and how it helps us to have a better understanding of what's happening in the body and carry out better treatment.

If you look at number one, we can see these are different modalities of images. Different modalities try to visualize different information for different tissues. And it helps a clinician to understand what's happening, what type of cancer it is, and how they have to treat that.

Number two is an expert clinician. Usually, there are radiotherapy oncologists that are trained for years. They sit down and draw the shape of the tumor on each slice. The medical images are usually in 3D or even 4D because we want to see the exact shape and location of the tumor.

The doctor will spend hours and hours going through all of the slices and drawing the shape of the tumor manually. The interesting thing is that two different doctors can draw slightly different shapes. Or the same doctor who comes the next day can draw the shape slightly differently. So, it's a very important task.

As I said, we want to kill the cancer cells and keep the healthy cells, so that's why the amount of time they're spending and the level of precision they have is very important.

After the doctor draws the shape of the tumor, we can plan the treatment. We can find out what the shape of the tumor is, and medical physicists can calculate how much dosage they have to deliver to be able to kill those cancer cells and how long the treatment will last.

Usually, radiotherapy is done in many sessions. For example, for lung cancer, it can go up to 30 days, or four to six weeks. And the interesting thing is that during the whole period of the treatment, we use the shape that the doctor drew before treatment started.

We hope that the tumor shrinks, but it doesn’t happen in all cases. Sometimes the tumor might expand or it might even react and get bigger.

Now that we have the shape and have started treatment, each time we have the treatment, we put the patient inside this linear accelerator. And this machine radiates the tumor based on the shape that the doctor drew.

So, how do we find that tiny tumor inside the patient's body and radiate that exact location? It's very complicated.

When we put the patient into the machine, there are imaging tools we have called combination with CT. It’s very low quality, but it's good enough for us to understand what's going on in the body and register the position of the patient to the device. Then we can determine that this is the right location and we can radiate this part.

When the LINAC rotates around the patient, it tries to rotate and shape the beams from different angles in a way that’s more efficient. This is so we’re not just radiating one shape from all angles or delivering the same amount of dosage from all angles. It's very important and there are lots of things that we have to consider.

Another thing that I want to emphasize again, is that for the whole process, we use the same shape. But in radiotherapy, we’re working on adapting and changing the shape each time that the patient goes for treatment.

Let's say for someone who has a tumor that either shrinks or expands, we can change the shape.

The figure below is able to show us the regions that we want to radiate for a tumor in the brain. The good thing about the brain is that it doesn't move, so the tumor will always be in the same location. Therefore, we don't have that many problems when we want to radiate this part. The amount of dosage that we want to deliver is fixed, and it's in the same location.

But when it comes to other parts of the body, like the lungs, the patient is breathing so the tumor shape is changing. Let's say that when our LINAC rotates around the patient, it tries to find the exact shape, but the shape is changing. So with advanced imaging techniques, we have to reduce margins and further limit those to healthy cells.

With some parts of the body, we can't resist the movement of the organs. And that's why adjusting and adapting to the patient's movement during the whole course of delivering the dose is very important. And we have to have a seamless online plan adaptation.

Why isn’t cancer cured yet?

Now that we understand how complicated the radiotherapy process is, with all this amount of effort and access to expert radiotherapy, the time that radiographers spend, and the cost of the treatment, do you think that this treatment would work successfully for all patients? Are they all being cured of cancer?

The answer is no. We still need to improve this. And the reason for that is because, with the treatment, we look at different types of patients the same way.

In the images below, we can see different guys who’ve had skin reactions and bruises on their skin after radiotherapy treatment. And this makes us wonder that maybe we’re delivering a larger dose to these patients than they need.

Maybe when the doctors were planning the treatment for these guys, they saw that the shape or class of the tumor they had was similar to other people and so they planned the same treatment for them.

But we can see that their skin is reacting so maybe this dose is harming some of the healthy cells on the way to reaching the tumor.

And maybe we have to consider some other information besides the tumor shape or type. For example, it might affect people who have a different skin color or people of different ages or genders, so maybe they should be treated differently and we should have different plans for treating them.

And this brings us to the important question: why isn’t cancer cured yet?

Although medicine is highly advanced, there's still room for improvement. And with the advances in imaging techniques, we can do so much better.

To me, the answer to this question is obvious. It’s because we use the same recipe for all patients, and there’s no personalization involved in the treatment. And it's because we’re limited in the amount of time and amount of access we have to experts.

Each clinician is specialized in a different field, but let's say somebody wants to consider their blood tests, it's very difficult to merge all the information together.

AI is the key. AI is able to understand this personalization much better than humans because it’s able to access different data and interpret it much better than us.

To help you understand what I'm trying to tell you about personalization, let's look at this figure below.

This is a picture of people who are all different in terms of gender, age, skin color, diet, levels of exercise, and amount of exposure to the sun. These factors make us different, so it's not really ideal that we use the same treatment for everyone.

People who have the same sickness and disease are affected by it differently, and some people get treated much easier than others.

For example, we hear that people in some parts of the world have less potential or more potential to get a specific type of cancer. For example, Southeast Asian women have less potential to get breast cancer compared to European women. Among European men, Italian men have less potential to die from prostate cancer.

So what is it? Is it the environment they live in? Is it their race? There are lots of factors that we’re trying to understand, but we don't know exactly what it is. And maybe it's a mixture of different factors.

If we look at the figure below, we can understand this concept much better. These are different people that have colon cancer, and the doctors decided to send them for the same treatment.

We understand that it works for most people, but of course, we know that for some people it has no effect or it can even have adverse effects. But if we’re able to personalize the treatment and understand each one of these individuals better, we can make the treatment better in a way that it's able to work for everybody and everybody has a good response to it.

What is personalized medicine?

Personalized medicine, also called precision medicine, P4, or stratified medicine, is basically a medical model that separates people into different groups and makes different medical decisions based on them.

For example, let's say the drug is being specialized for different individuals that have the same disease. Personalized medicine can not only be used in treatment, but we can also use it in prevention and diagnosis. And by understanding more information about each individual we can detect lots of these diseases much earlier.

How can AI aid in personalized medicine?

We believe that AI would be able to help with this better than humans. And the reason for that is AI is able to understand and look at lots of different data at the same time, find out the pattern in different types of data that we have, and interpret it much better. We also don't need to have access to lots of different experts at the same time in the same room.

This would be really helpful and would be the goal for us. For the same guys with the same type of cancer, now we don't have to only look at the images we have from CT scans. We can look at blood tests, demographic information, genome information, and lots of different data that we have from a patient that’s normally ignored when we send them for treatment.

And based on that, we can categorize them into different groups, find the best treatment for each one of them, and then provide more effective treatment.

There are many benefits of using this for sure, and of course challenges. The best benefit would be that with machine learning, we can deal with large multimodal datasets to identify these hidden patterns. Sometimes the pattern of one disease can be hidden in one set of data, but it's easier to understand.

For example, sometimes there are people who have very advanced breast cancer, but it doesn't show it very well in the scans of their mammography. But there’s lots of other information the doctor needs to understand. So when we have it with something like machine learning, it's easier to understand that.

Another benefit is that it would reduce the burden on the healthcare system and deal with the limitations we have with accessing doctors.

As for challenges, optimization of the application is one of the most important things that’s required. And also, stakeholders such as doctors now need to have a basic understanding of the domains, how to use them, and how to improve on them.

Combining different types of data for personalization

Let's say we have a picture of a person. If we know this person and we’ve heard his voice before, we can identify who he is. Our brain can understand and it can merge different types of information at the same time.

So, let's say we only see the picture of this person, we can still detect who he is. Or if he calls us and we don’t have the information of the picture, we can still understand who this is. So in the absence of one of the pieces of information, our brain is still able to do a good task.

But while there have been lots of improvements in AI, we still need to do more work on it to merge different types of data to be able to replace one piece of data with another one. And that's one thing that's very important in healthcare.

For the example that I gave about breast cancer, if the information we’re looking for is hidden or not there in one of these sources of data that we have, then we should be able to find it from the other resources.

And coming back to radiotherapy, when we have lots of different information for the patient, let's say their blood test, their urine test, their genomic information, and so on, we can get a better understanding of the situation of the tumor.

We can segment it better, we can classify it better, and we can do better planning for the dose delivery. And of course, we can do better treatment, which is more personalized for that patient and has more effect.