Given recent AI fervors, I would like to give a perspective from the medical device world and how the current narrative will end with an outright rejection for most uses of it.
I've had this in my head for a while and it is just a way to get it out.
When you release a new product for deployment in hospitals (AKA, patient wards, general and ICU), there is always an initial uptick of complaints from nurses. Are some of those bugs in your product? Yeah. But another set of complaints is usually bound to a distrust of the device, an inherent defensive nature of nurses. They think the device is there to replace some of their functions.
While hospital administrators (and many other industry admins) would love to be able to replace most of their staff to lower costs, at the end of the day medical devices are designed with nurses and doctors in mind. Not admins. We see a complication in their workflow and want to address it. Blood pressure taken manually can take up to 5-7 minutes (including putting the cuff on, instructing the patient in what to do), blood pressure taken by a machine can take 1-3 minutes (slap the cuff on, ask patient not to move). This example shows 3 things: a reduction in usage of the nurses day to day time, a reduction in needing to teach the patient proper form for the measurement and a reduction in complexity of task.
Overall, nurses end up gaining in the process, it becomes an automated task and they can very easily hop in and out of a room after just setting the machine to record. Nurses hated these machines at first; some claimed that they knew better than them despite the fact the machines are calibrated to the same accuracy as the manual measurement. The messaging employed from the device industry was then clear: it's not here to replace you, it is here to help.
Now imagine trying to get nurses to use a blood pressure monitor, all the while there is a group of people outside the hospital ranting about how they're going to replace every single nurse with these machines. These fans of the monitor are not doctors, nurses, or medical staff at all. They for some reason hold resentment to those professionals for being able to carry out the craft. They think it is an easy job (it's not) and that it could be relegated to machines. This is what AI promoters are like. They do not understand the fabric of the industry they are trying to automate and fundamentally misunderstand what people need to actually improve their workflow.
If the nurses example is anything to go by, having these disruptive voices in the room will just push any professionals to swear off most uses of AI. As long as the pushback is bigger than the movement (assisted by if these have any utility at all), AI will be relegated to niche utilities and forgotten. You cannot sell your product and insult your customer at the same time, and the failure to recognize that will be its downfall.
