+1 612.405.4059

Product Testing – It Can Really Hurt

Imagine a pioneering surgeon inventor and entrepreneur dedicated to proving that his latest medical device is safe by implanting it within the muscle of his own rib cage… by himself. Then, months later, removing it… again, by himself. The surgeon told me that it was a lot more difficult taking the device out than it was putting it in! While the ethics and safety of his procedure were certainly questionable (at best), the experience certainly showed the surgeon’s personal conviction in the design of his device. To him, the classic biologic response of tissue growth into the device surface was evidence of biologic safety and proper choice of materials. To an engineer, however, this experiment was rife with flaws that make it useful only as a great story over a beer with colleagues.

Good experimental design leans heavily on statistical analyses, and in the world of statistics a sample size of one is meaningless. So how many surgeons, willing to operate on themselves, would it take to provide good statistical evidence? That sounds like a setup for a bad joke (at least to an engineer) and the question obviously has no practical answer. But the engineer would still have a few other questions about the device. Was it built using a validated process? Was it made with implant-grade materials? Was the cleanroom certified and the device properly sterilized? And, how well can the results be applied to the rest of the surgeon population?

A more important question from an experimental perspective in this crazy scenario, however, is one of appropriateness. Not the ethical issue of using live surgeons in a medical device experiment, but rather in how the device was intended to be used. In this case, the device wasn’t meant to be implanted in the intercostal space, and it was later found that devices explanted from their intended location in a clinical study showed no tissue ingrowth at all. The dedicated surgeon’s rather painful experiment ended up having no scientific value since the biologic response he saw (and felt!) wouldn’t translate to the actual implantation site.

While his approach was clearly non-traditional, the surgeon was one of the smartest people I’ve ever met. He didn’t do anything without a purpose, and he knew how to calculate risks. While his intent was to provide convincing anecdotal evidence before objective testing could be performed, he never said if it was to satisfy his own curiosity or to show proof of safety for potential investors. In that regard, I’ll leave the question of appropriateness to the ethicists. But the story is a good one for engineers to think about when designing a test or for anyone who will be using product testing results when trying to convince others of device safety. Maybe over a beer.