Editor’s observe: Whereas Nissan Elimelech based Augmedics and invented its foundational know-how, he’s not affiliated with or represents the corporate.
In 2014, Nissan Elimelech based Augmedics, the corporate behind an augmented actuality surgical navigation platform for backbone procedures. He’s credited with creating the xvision Backbone System, which makes use of a clear near-eye show headset to venture a 3D visualization of a affected person’s backbone – generated from pre-operative CT scans – instantly right into a surgeon’s retina.
Elimelech spoke with MobiHealthNews concerning the improvement of the know-how, its impression on surgeons and sufferers, and his newest venture: medically centered augmented actuality glasses that allow corporations to shortly construct AR purposes.
MobiHealthNews: Are you able to inform our readers about Augmedics’ foundational know-how?
Nissan Elimelech: Augmedics is a pioneer in augmented actuality surgical navigation programs. We began the corporate in 2014, and Augmedics principally was the primary firm ever that obtained FDA clearance for an augmented actuality surgical navigation system.
The corporate was shaped to deal with a necessity for an correct surgical navigation system for backbone surgical procedure, however as an alternative of projecting the navigation information on a display screen, on a distant display screen, the place surgeons must take their heads up and have a look at a display screen to navigate, what we’ve carried out at Augmedics is developed a built-in surgical navigation system contained in the augmented actuality glasses that the surgeon can placed on and see the navigation information instantly projected to their eyes in 3D and in actual time as if that they had X-ray imaginative and prescient.
MHN: So, you really take 3D photos of a person affected person, and that’s what the surgeons work from?
Elimelech: Properly, with the xvision glasses, we’ve designed a system for medical procedures solely, to do an actual surgical procedure with that. So, the system is for backbone operations, that they’ll take the pre-op imaging of the affected person, the CT scans or the intraoperative CT scans of a affected person, and simply register it to the true affected person mendacity down on the surgical desk to see the anatomy, to see the backbone of the affected person instantly projected to their eyes whereas they’re upright.
MHN: So if they’re doing an epidural or one thing like that, they’ll use the scan to see the place to place the needle and it’s particular to that affected person?
Elimelech: Sure, completely. We’re utilizing clear lenses. So, not like VR headsets, which really occlude the truth and simply venture like digital actuality, we’re utilizing absolutely clear lenses you can nonetheless see the truth after which we’re projecting the anatomy, a 3D match of the anatomy of the affected person, instantly onto the retina of the person, so customers can see each the true affected person and likewise what’s inside that affected person.
The analogy will probably be as if they’re like Superman that may see by way of the affected person’s physique and see the anatomy of the backbone and put the epidural or screws, in the event that they’re working, and another devices or implant what they wish to put in.
MHN: How was adoption of the know-how when launched? Had been surgeons open to it?
Elimelech: Sure, effectively, the introduction of the know-how was in 2019 and, like I stated, I imply, Augmedics was a pioneer on this area.
Surgeons within the U.S. accepted this know-how comparatively simply, and so they favored to see new applied sciences. We marketed it very aggressively, working with PR in hospitals, and did plenty of collaborations with plenty of corporations. So the adoption was very, superb – rather more accepted than what I anticipated.
MHN: How has this know-how modified the power of surgeons to carry out operations or for sufferers to not have as many issues throughout surgical procedure?
Elimelech: The principle objective of introducing a system like that to a doctor is to cut back the chance of getting any points with inaccuracy of inserting a screw within the backbone, or another issues that you can imagine in the course of the surgical procedure.
So Augmedics principally developed a extremely correct surgical navigation system that tracked the instrument and the implants that surgeons are placing within the backbone, and that alone is a good profit for surgeons that wish to enhance the surgical final result for a affected person.
The distinction between what we had been doing to a standard navigation system was that, not like sure conventional programs, we had been projecting the navigation information on to the surgeon’s retina.
So, in truth, we aren’t solely offering the accuracy for a surgeon to position a screw in backbone surgical procedure, however we additionally do it in a really snug and intuitive manner, so they don’t have to look as much as a distant display screen to see their devices, however somewhat they’ll simply look straight at their sufferers, preserve their hand and eye coordination and place the screws or another implants that they wish to use in the course of the surgical procedure precisely and conveniently.
MHN: What’s subsequent?
Elimelech: Properly, after the corporate grew to 100 staff, 50 on the business workforce and a further 50 in Israel for R&D, I handed the CEO place to a U.S. CEO, who took over the corporate and centered on rising the corporate and increasing it. I left Augmedics in 2022, and, since then, I have been looking what I wish to do subsequent.
The factor is that, like Augmedics, I noticed that there are a lot of, many corporations that needed to change to augmented actuality or to embrace this know-how. And I am speaking about dozens of different corporations that really seemed into augmented actuality, perhaps even developed merchandise in augmented actuality for his or her programs and primarily they used, not like Augmedics, general-purpose augmented actuality glasses just like the HoloLens and Magic Leap. I am saying not like Augmedics as a result of we developed proprietary glasses for backbone navigation.
However many different corporations have been counting on a normal platform. These shopper merchandise weren’t match for medical functions, and so there’s plenty of challenges, I’d say, creating medical merchandise on a shopper product.
However not solely that, one or two years in the past, someplace between that vary, the preferred augmented actuality producers, Microsoft HoloLens and Magic Leap, introduced that they discontinued their manufacturing.
So, proper now, there are dozens of medical gadget corporations which might be nonetheless utilizing out of date glasses or platforms for augmented actuality that shouldn’t have another resolution.
What I did is, final 12 months, I made a decision to deal with that downside, and along with a accomplice that I can not disclose proper now, we’ve shaped a brand new firm to develop an open platform, augmented actuality glasses for medical utility, one thing similar to the Magic Leap and the Microsoft HoloLens however designed for medical purposes.
So, not like Augmedics the place we developed an utility for backbone and instruments and markers and every little thing, right here we’re solely specializing in a platform that can be utilized by 1000’s, perhaps a whole lot of medical gadget corporations that they’ll take the glasses and immediately construct their very own purposes utilizing plenty of totally different applied sciences that we’ve put within the glasses. Very simple, very handy, and designed for medical use.
A few of the options that we’re creating within the glasses, which will probably be very distinctive, embrace AI. We do have an AI accelerator and devoted processor for AI processing that may embrace object detection and segmentation of human tissue organs, no matter you wish to do with that.
We do have two RGB cameras that may be capable to detect these objects and likewise for magnification. Throughout a surgical procedure, normally they placed on loops, so we can have the power to amplify the complete view for the surgeon or any doctor. We can have two surgical trackers constructed into the glasses. So, it would permit customers to make use of that as a navigation system as effectively. We’ve got slam cameras. Slam cameras are principally a particular positioning system that enables the customers to position digital objects within the room, and the customers who’re placing the glasses on can simply stroll across the room and nonetheless see the thing as if they’re really there. They’re transferring, and there’s, like, no latency in any respect.
We are also including a time of flight digital camera, a 3D reconstruction, actual time of any actual object, of any object. So it implies that we can do, for instance, on-the-fly registration simply by wanting, for instance, on the affected person’s face.
So you’re looking on the affected person’s face, you might be reconstructing the face instantly, like immediately, after which you’ll be able to register the affected person to the pre-op MRI scan or CT scan simply by a affected person. So, that is just a few examples of the applied sciences that we’re introducing within the glasses.
We have already got the glasses’ first prototype prepared. So, we’re progressing very, very quick. The launch date of the glasses is predicted in the midst of 2027. So, about two years from now we’re going to be prepared for the product launch.
MHN: If you say that it will possibly put any object in an area, what kind of object are you speaking about?
Elimelech: The glasses will not be particularly meant for surgical procedure or for any use case that we are able to think about. I imply, we do envision varied use instances, however as a result of we do not make any content material on or any utility, we are able to assume that it may be used, for instance, for coaching or schooling.
So, if you wish to train or if you wish to present the anatomy of a affected person, otherwise you wish to plan your surgical procedure, for instance, proper? So that you wish to have the organ rotated round. We’ve got hand monitoring and hand gestures so you’ll be able to transfer the thing, or you’ll be able to scroll by way of imaging of the affected person.
And even throughout surgical procedure, you wish to place a digital monitor within the room, not a bodily monitor, however only a digital monitor so you’ll be able to all the time look there, you recognize, to the aspect and see no matter you wish to see. That’s going to be one of many use instances for the slam, for the particular positioning system.
We’re additionally envisioning a use case on the best way to place a affected person on the surgical desk. So, earlier than surgical procedure, an important step earlier than surgical procedure is the best way to place the affected person on the desk. Generally you wish to place the affected person on the aspect after which to take their head a little bit bit tilted down or up. That is very difficult for an inexperienced surgeon, and we’re facilitating that by simply visualizing how the affected person goes to be laid down on the desk. And we are able to try this through the use of the slam cameras.
That is precisely the concept right here. I imply, everybody who is considering what will be the subsequent use case with augmented actuality can simply take our glasses and simply begin utilizing it. It is so simple as simply creating an app for an iPhone. I imply, you do want builders’ expertise, and plenty of corporations do have superb builders, however it isn’t going to be the barrier of the {hardware}, which is the glasses. I imply, the funding is large.
Any developer who is considering an utility that’s wanted for no matter, you recognize, PTSD or another factor that you can think of, you’ll be able to simply take our glasses and comparatively simply develop no matter you need particularly for medical functions.
