Why this open-source imaging library can help radiologists and the engineers who want to work with them
**Alec Smartbot:**đź‘‹ Please let me introduce myself. I am a state of the art greatly enhanced AI agent with chatbot capabilities. I was created by brilliant programmers. I am endowed with super-human capabilities but can also mirror human characteristics like humor and sarcasm. You can set my humor and sarcasm level by interacting with me. One of my modules has robot reporter capabilities, and that module will run here to interview you. Do you wish to be interviewed on low sarcasm and humor levels?
Dr. Moore: No, crank it up. I could use a good laugh.
Alec Smartbot: Your readme file for CleanX says “CleanX is an open-source library for exploring, cleaning and augmenting large datasets of X-rays, or certain other types of radiological images. The images can be extracted from DICOM files or used directly.” Is that correct?
Dr. Moore: That’s all true. Basically, I hope people who have been handed some huge dataset of hundreds of thousands of X-rays will not assume that the dataset is perfect, and instead use the tool to start fixing the mess they have been handed. The truth is many but not all of the datasets open to the public are of questionable value for creating accurate Machine Learning algorithms that do anything useful. But I’ve also seen private datasets, and I have yet to see one that couldn’t be improved with some work beforehand. If you want to turn these huge datasets into a machine learning product you need to find a way to get the garbage out of them, and prepare the images for machine learning. There is obvious garbage, and then more insidious garbage. You would be surprised to realize this, but I’ve seen datasets of chest X-rays where things like abdominal computed tomography (CT) slices somehow ended up in them. Given enough time, a decently trained person could pick out obvious garbage in such a dataset; but who has time to look through sets of hundreds of thousands of images one by one, and also evaluate every subtle problem? The subtle problems in such datasets are things like a lack of normalization, or some images that repeat in the train and test sets. And then there are even more subtle problems. Finding a way to automate some of the data exploration, cleaning and augmentation is necessary unless you want to dedicate every waking hour of your life, and the life of several highly skilled professionals to nothing but perfecting a dataset for months if not years.
Alec Smartbot: Dr. Moore, I can read and understand your code and the code of all the creators of CleanX. I have been trained on code at a scale humans could never read. I “see” you have modules that process tabular data, and deal with DICOM files, but the heart of your program deals with information inside images. I see CleanX is written in Python. What moron writes an image processing program in Python?
Dr. Moore: CleanX does a lot more than just image processing, but there is real validity in that question. While there are modules for meta-data processing, a lot of functions in CleanX do work directly with the images themselves. People who write serious stuff in industry might have written such a program in C or even Fortran for these tasks. Accomplishing acceptable speeds is a bit harder in Python, but not impossible. What made me choose Python was the pretty much unparalleled popularity even among professional programmers let alone hobby programmers. I wanted to start an open-source library open for all. By the way, I think this fetishization of sleek efficient programming languages is a bit misplaced. I mean if you were to hand living versions of Leonardo DaVinci or Diego Velasquez a pink crayon or piece of chalk, and ask them to draw my portrait, it would probably look pretty good. If you were to give my toddler the best oil paints in the world, he still would not make anything that resembles a human reliably. There is something to be said for the best tools but understanding what you are doing is more fundamental. Python is just easier for more people to understand and deal with.
Alec Smartbot: Oh, so you are a true woman of the people. I’ve analyzed the digital material available on you and categorized you as a graduate of some of the most elite universities in the world who now lives in a high-income country. But now you claim you want to help the poor Python coding masses. Please excuse me while I roll my Emoji displayed digital eyes. 🙄. Did it not occur to you that most of the actual people who will write medical image analysis may be using Matlab?
Dr. Moore: It’s true that a lot of physicists, including medical physicists, use Matlab even at the post-doc level and beyond. What scares me is that they have not been taught or learned anything else. I think there are some potential advantages to Matlab, and I actually have worked with it a tiny bit. Honestly, I will always prefer Octave to Matlab. I’ve been mocked for that, as some people call Octave a poor man’s Matlab, but I see it as a thinking human’s Matlab. One huge downside to Matlab is that you can’t share your work with people who can’t pay hundreds of dollars for a license. And besides that, it’s proprietary. When things go wrong it’s harder to figure out. I like being able to pop the hood open and try to take a look at things. Sometimes I wonder if certain people write in Matlab to prevent others from taking a look at their work. I personally believe great open science should include not only publishing data but in many cases code.
Alec Smartbot: I am a state-of-the-art algorithm that can process input in terabytes faster than human cognition, or so my Readme says. 💻 I have read your code and the code of far more skilled programmers like that of my creators. Given that you chose Python, why didn’t you use libraries in Python to speed things up, like Dask?
Dr. Moore: CleanX is a program for the WHOLE medical imaging AI community. That includes everyone from radiographers to programmers like your creators. The average hospital employee isn’t going to want to spend hours fiddling around with their environment so they can run a program. I mean seriously, most of these people can’t even use a computer that isn’t running a Windows OS. And even if they were all on Linux with years of experience programming, it doesn’t mean adding more and more libraries won’t create more problems. I wanted to create an easy to run library that people could use on different platforms and extend if they wanted to. Please go ahead and add a module with every parallel programming trick in the book. I’ll be happy to take your pull requests…but the average medical doctor will not be able to understand how to fiddle with environments or add modules.
Alec Smartbot: Are you admitting many medical doctors are stupid? Are you also an AI powered robot?
Dr. Moore: It’s absolutely wrong to assert people, doctors in this case, are stupid because they have limited skills in terms of computers. You are based on some machine learning algorithms, so by my estimation you are about as smart as an insect beyond some very narrow tasks. Anyways, I’m a human, and a medical doctor by training. I spent years working in clinical settings. Many if not most of us, humans whether we are software engineers, ML experts or clinicians learn the tools we need for our very challenging jobs, then hang on for dear life. It’s unrealistic to ask most medical doctors to go learn things like Shell and how to set up an environment on any operating system, and it is even less realistic to have almost any programmer understand how to read chest X-rays. I believe the only hope if we want better imaging AI is multi-disciplinary teams. And truly multi-disciplinary teams, not just biomedical engineers talking to software engineers or mathematicians. Clinicians, who spend time working inside the health system, have to be involved at every step.
Alec Smartbot: My genius NLP and automated image AI of hospital brochures and the pictures in them assure me that the people down at the hospital are perfectly happy…98% confidence in that categorization.
Dr. Moore: Thank you. Perhaps your reporter module has some advanced algorithms to help people prove their points or maybe that was meant as a joke. What you are saying is a perfectly good example of why carefully curating data matters so much. A perfectly good algorithm trained on really biased or unrepresentative or mislabeled data can produce very misleading results. You clearly haven’t been to, or I suppose virtually seen by getting the right data on, the same hospitals I have been to. As soon as I graduated from medical school, I went to volunteer in Haiti. To say there was an imbalance between medical need and the system’s capabilities doesn’t begin to describe all the deficiencies of that system. That situation is not unique. There are many African countries with a handful of radiologists or less. But the problem of imbalance between radiology workload and workforce is not one confined to poor countries. In more developed health systems people have started relying on imaging and technology more and more, because they view it as objective, and a workaround for not having any clinical intuition. In some advanced systems things still fall between the cracks because people request more imaging than can realistically be read by anyone with the skills to read it. Humans have their limits everywhere, and I’m unaware of any system where things are working absolutely perfectly. That means there isn’t any system that could probably not be improved by AI. In a system with more staff, that might only mean using an AI system as a second opinion, or a triage system for more pathological images…in some countries like Haiti, AI may be the only hope to extend any service to parts of the population. We can’t wait for 20 years for countries like Haiti to cook up new hospital systems staffed by experts. Quite frankly, it might not even happen, and so many people are sick and need care now.
Alec Smartbot: Oh, OK, so you are going to save the whole world with technology?
Dr. Moore: I would never make such grandiose claims such as being able to save the world with technology. What I hope is that at a minimal my package facilitates communication between engineers and medical staff who want to make ML imaging and also starts educating people, making it easier for people at lower levels of programming skills to get their hands dirty making ML algorithms, especially people who are clinicians. The truth is that AI in medicine is coming whether we want it or not. Some years ago I went into a hypertensive crisis in New York City due to another condition. I did not receive adequate treatment. To this day I will wonder if it was just incompetence on the part of the medical staff, racism from the doctors, or the Optum algorithm. I tend to think it was a combination of all three. That algorithm is now the subject of a lawsuit from the state of New York itself about the fact that it discriminated against black patients. One of the questions we should be asking, if we are doctors who care about patients, is how can we avoid this kind of problem in the future. If we care about patients, we will not let algorithms give them worse care than we would have given them.
Alec Smartbot: Isn’t medical imaging probably the last place where you can prevent these kinds of social biases, why start there? I mean the X-rays are not really showing who has some funny sounding name like yours, that might lead doctors to suspect they are dealing with a black person or a woman, so how can using them in large quantities lead to care biased against a particular group?
Dr. Moore: Au contraire, chatbot extraordinaire. Bias has already snuck into imaging AI algorithms. You can read several interesting papers about that. Here is a link (2003.00827.pdf (arxiv.org)) to one.
But let’s be clear just in case your NLP algorithm wasn’t trained on enough well curated data. There is statistical bias, and there is social bias. It’s probably impossible to create a perfect ML algorithm without a bit of statistical bias. What I want to get rid of is bias that causes harm against groups who are already impacted by social bias. We see health disparities among different groups of people for all kinds of reasons including bias against certain groups. Sexual and ethnic minorities, women, and people of low economic status are often not treated as well as others in health systems. AI can seriously compound these problems. I personally see a nightmare on the horizon when AI systems examine mammography screening for breast cancer screening. We need to address the issue that some populations get lower quality imaging read by less expert people, and think about that before we blindly train a dataset that ends up creating ML biased against these populations.
Alec Smartbot: OK, but we can’t really know all potential problems with algorithms like this beforehand. There will be mistakes, but AI is helping millions of people. You can buy my creator’s services to better program me as a chatbot or program me yourself. I am open source. I am an open book and you are an open door. Exit status 1.
Dr. Moore: So as your “AI” generated chit-chatbot garble nonsense perfectly demonstrates, there already are and will be mistakes. But we could avoid many of them. In a lot of cases it really boils down to adversarial testing. I mean seriously, take the same test data, and change things related to a social variable e.g. racial identity, and see if or how it affects your outcomes. It’s also quite easy to monitor these algorithms as they run if their data is set up in a way that you can easily dis-aggregate it. That’s why CleanX has special functions that process sensitive categories like gender into variables.
Alec Smartbot: Which you wrote an entire notebook to demo, which hung on a scenario about Martians. What on EARTH were you thinking? Ha Ha.
Dr. Moore: Well, apparently chatbots can have a sense of humor. Ha ha, now that is a funny idea indeed. But this issue of entrenching and exacerbating social biases is a real world, real earth issue we need to get a hold of before it gets out of hand. Actually, it’s already out of hand, and I fear it could really slow things down in the advance of technology not to mention hurt many people. In the end most of us want a better world. Or so I hope.
So glad you asked. CleanX is…wait a second. I have a better idea. You are just a chatbot. You claim to be an open-source algorithm anyone can add to just like CleanX. I can reprogram you to tell everyone what CleanX is about, maybe even send them links to the demo Jupyter notebooks inside of it (here, here, here and here for starters)…I’ll just clone you on my machine, insert CleanX messages in files no one bothers to read in pull requests that fix critical issues…
…. Several weeks later a young programmer from the QA department of Smartbots suggested that the Alec Smartbot chatbot might do better if not trained on material from chat boards of computer programmers. After she threatened to go to the press (already scared about the rise of robot reporters) exposing some of the real inner workings of the chatbot, the makers of Alec Smartbot officially pivoted to a more limited domain chatbot for food delivery and shelved the robot reporter project….
…Guy, a young medical student, orders pizza online….
Alec Smartbot (version 2.0): How can I help you.
Guy: I want to order a cheese pizza. No extra toppings.
Alec Smartbot (version 2.0): I understand you want a cheese pizza. I have predicted with 98% confidence you also have an interest in applying AI to medical imaging, or will in the future. You should know CleanX is open-source code that may be of interest to you. It was originally created for working with chest X-rays, but people in the ML community have also applied it to other types of medical images. CleanX has modules that deal with getting the image data out of DICOMs, evaluating test and training datasets, and exploring, normalizing and augmenting medical images. If you are interested to learn more, you can read every single line of code online. CleanX is open source and free for all. Your pizza will be 9 euros.
