Haptic simulator is a system enabling interaction with a virtual object which always requires a computer for calculation of object's physical properties. Haptic simulators and simulations are important incitements for the development of haptic devices e.g. Surgical training modules, flight simulators etc.
Haptic devices can simulate a number of surface properties such as friction, adhesion, texture etc. Surface friction describes the viscose (velocity-proportional) friction of a contact point on a surface. Surface adhesion describes a force binding the movement of a contact point to a surface. This concept allows simulating magnetic or sticking effects. Roughness describes an uniform, sinoid structure of a small, defined amplitude making the movement of a contact point on a surface appears rough. In addition to that, haptic can account for object stiffness and can be utilized for impact or penetration phenomenon. Both of these effects have been utilized for this device.
During laparoscopy, trocar is grabbed with a cylindrical grip in one hand and then the tip is steered in tissue by twisting the base. For a realistic simulation, we needed to construct a custom haptic device which provides sufficient linear haptic feedback in the direction of insertion. Also, there is a torquing motion at the base while penetrating the tissue, and frictional forces are encountered while rotation and while performing trocar retraction, which makes rotational haptic feedback a necessity. Commercial devices such as Sensable Phantom Omni [Web] and Sensable 6 Degree of freedom Phantom device [Web] are easy to program for customized haptic applications and easy to interface with realistic graphic rendering.
For the simulation of trocar insertion, we have used a Sensable Phantom device programmed to simulate a virtual haptic environment with subject's thorax portion in the view and a trocar probe. This gives the trainee a realistic interaction interface. The twist motion available with 6 DOF Phantom can be utilized for torque feedback, which is useful for simulating the rotator motion and frictional resistance at the trocar tissue interface. The detailed working of the haptic loop and methodology underlying the haptics is illustrated below. Note that we wanted to construct a realistic simulator which gives an effective feedback to the trainee; however the level of details and haptic realism has been considerably simplified at some places to improve performance.
3.2 Commercially available Haptic APIs considered for simulation
In the area of software development, several convenient haptic APIs are also available to develop applications. There are several well known APIs available for creating Haptics applications from scratch such as CHAI3D and Ghost SDK. CHAI3D is a well known open source multi platform haptics rendering API, used by a lot of researchers, in conjunction with PHANToM and FALCON devices, for desktop based Haptics applications. It internally uses C++ coding style with scene graph based data structures to help the programmer design applications quickly and efficiently, with built in code for dental, fluid and gel based simulations [Ref 32]. GHOST SDK is another well known Haptics API provided by Sensable [Web] with static, dynamic, transformational and spatial force feedback capabilities. This API is easily extensible and provides support for Sensable devices [Web].
SOFA is one of the open source medical simulation frameworks that was considered for this simulator. SOFA provides a common software framework for various medical simulation needs while enabling component sharing and integrating new algorithms in a highly advanced and extensible framework [Ref 33]. It provides an active support for GPU based computing for implementation of fast non linear based solver but we found that the development time with SOFA is more because of the learning curve. SOFA uses a scene graph based approach but sometimes fails to provide support for standard haptic devices. Hence, we plan to integrate SOFA in our application in the next phase of the simulator.
3.3 Challenges in Haptic Modeling
We faced several challenges while modeling the haptics behavior for abdominal tissue and to increase the fidelity of the simulation. Some of these challenges have been addressed in the current version of the simulator and for others, the work is in progress.
Modeling organs and placing them at their anatomically correct position with respect to outer abdominal tissue layers
Constructing haptic mesh and perform collision detection with haptic probe
Model physical properties of the tissue layers at the surgical site
Simulating behavior of trocar insertion, resistance to penetration force and haptic feeling of braking force
Simulating resistance to torque and friction force at the trocar tissue interface
Synchronizing haptic and graphics meshes and their deformations
Performing realistic graphic rendering and simulation of trocar, organs and abdominal tissue
3.4 Force profile and differential haptic rendering methodology
The proposed haptic simulator uses a differential method for haptic rendering of deformable objects. The choice of the model, dictates haptically rendering the force and contact interaction between a non linear elastic surface with a linear haptic model. The resulting equation is of the nature,
F (Δx) = −k.Δx + a. Δx 2 + b.Δx3 + …. Equation I
Where, F = Reaction force experienced by the surgeon performing trocar insertion and
Δx = deformation of the spring with respect to its length at rest.
We fit a polynomial of the desired degree, to a force profile obtained from literature, to obtain appropriate haptic response. By setting the constants at a specified value, we can obtain the desired behavior for the deformable surface. To avoid unnecessary computation and minimize delay between desired position and computed position, we initially use a lower order polynomial and also have an adequate value for degree of polynomial to approximate the force profile in a satisfactory manner, without loss of contour information or data points. Hence, we use an optimum degree of a polynomial in the end.
We plan to use multiple force profiles constructed from a correlation or mapping between different factors affecting the force profile (patient age, Body Mass Index and level of exercise) for actual simulator. Here we show a sample force profile to demonstrate this concept. The force profile data for this particular case was obtained from previous research [Ref 1]. We fitted and used a polynomial of the above form for this data for haptic rendering of this force profile as shown in figure 2. The resulting coefficients of an 8th degree polynomial are given below:
C0 = -0.0680, C1 = 1.8386, C2 = -20.1777, C3 = 114.9088,
C4 =-359.4543, C5= 595.9162, C6 = -453.6914, C7= 130.5271,
The resulting equation of approximating polynomial for this force profile is given by,
F (Δx) = ∑ Cn. Δxn, Equation II
where n varies from 0 to 8.
Also, when we researched for force profile data from multiple sources, it was found that the data registered with real time force measurement for trocar insertion shows a lot of discrepancy and it is hard to establish a general shape of the data, for a particular case. For example, force profile shown in figure 3(a) [REF], is reproduced from reference [Ref 6], and shows a spike shaped exponential curve. This type of force profile can be easily simulated using a model with two linear springs attached in parallel as shown in figure 3(b) [REF]. For the first linear stiffness region, for displacement d1, the spring stiffness is set to a small value k1. After force equal to dead force is exerted, the second spring is activated which increases the effective spring stiffness to ke = k1 + k2. However, after reaching peak force value, sudden decrease in force is experienced which simulates the entry of the trocar in rectus abdominis. We plan to carry out a number of user studies with both linear and non linear types of force profiles to obtain a scientific feedback from expert surgeons, about which force profile simulates the tissue behavior realistically. For now, we find that for end users who do not have expert skill, it is hard to distinguish between the haptic sensation with a linear and non linear force profiles.
Figure Simplified force profile to simulate spikes experienced due to braking force
Figure. Nature of characteristic force profile generated by a trocar, multiple peaks indicate the forces required to overcome fascia and peritoneum resistance respectively [Ref 13]
A swine model was used in another research work to replicate the abdominal tissue behavior at 10 mm of Hg pressure [Ref 13]. The characteristic nature of force profile obtained in this research, confirms the fact that there are multiple peaks encountered during penetration for overcoming different tissue layers (fascia and peritoneum layers respectively). After the braking force magnitude is reached, a sudden loss of resistance or giving away of the tissue is experienced which should be effectively simulated with the lowering of haptic resistance during the simulation.
3.5 Haptic loop explained
Figure Block diagram of haptic loop for haptic rendering
A block diagram of haptic loop for surgical simulation is shown in above Figure used for our application. The simulator consists of four blocks: the trainee surgeon (human operator) interacting with windows based application, the haptic device mechanical subsystem, an electronic interface connecting the mechanical subsystem with the haptics API and the anatomical model of the patient (environment) and the tool constructed within Haptic API framework. The trainee surgeon interacts with the simulator by positioning the haptic device, which exerts forces back that are similar to the actual patient interaction. The choice of the haptic model dictates the computation of desired Cartesian forces and torques on the tool based on physical properties selected for the tool tissue interaction. The FPGA controller calculates the motor torques for the haptic device based on the sensed position and/or force and the desired interaction forces and torques coming from the simulated anatomical model.
The motor torques are then sent to the haptic device, and any updates to the model are sent to the visual interface, typically a 3D graphical representation of the environment. The haptic control and visual computations may be done on different computing platforms, depending on the system architecture. In a typical implementation, the haptic device must be updated at a fairly high rate (on the order of 1000 Hz) to ensure stability and a responsive interface. On the other hand, the visual interface can be updated at a much lower rate, about 60 Hz for a recent generation display.
Graphics API form an essential component of the entire software which add a level of realism to the rendering. Graphic rendering has to take care multiple issues such as rendering meshes, applying textures over meshes, bump mapping, uv parametric mapping of textures, camera and scene graph objects. In addition to that, trocar insertion simulation implies that tissue deformation and tissue cutting problems are taken care of in real time.
3.6 Description of the Simulator
While designing the simulator, a number of things have been considered such as the maximum force/ torque application capability of the haptic device, ergonomic considerations in order to generate realistic haptic behavior and advanced visualization techniques to generate tissue deformation and trauma.
One of the most important factors of virtual surgical simulation is the graphic and haptic realism of the underlying tissue model. The tissue membrane is constructed as a non elastic membrane connected with virtual springs, under tension. A thorax model is penetrated with a realistic trocar instrument, which is modeled based on Endopath Excel, trocar obturator assembly. The simulated surface acts as a non elastic member under tension connected by virtual springs. The underlying visualization uses a number of spheres connected by virtual springs which simulate the behavior of deformable tissue due to equivalence between mesh formed by connected tensile springs and finite element discretized mesh [Ref 13].
Figure. Diagram of the trocar insertion simulation
A scene graph utilized for the application contains meshes for tool and abdominal model, virtual cameras and lights for the environment. The root object for all objects in the scene graph is a cGenericObject class which inherits from a general abstract type cGenericType. The generic object creates a tree structure of objects using a standard template vector class of children objects in children member. This application's scene graph has one root node class for every object in the scene called cWorld. This class is essential for further communication with graphics and haptics. Any effects applied to the parent node in the scene graph tree, are applied to all the subsequent child nodes if they are not locked. This property is very useful for managing
We have used cShapeSphere class extensively throughout the simulation mainly for collision detection purposes and for simulating the tool interaction with the membrane. Deformable objects in the scenegraph are constructed using gel simulation class however; we have implemented our very own haptic model based on force profiles computed from real data as discussed in section 3.4. The gel based simulation merely helps to create the graphic rendering for soft tissue deformation, since we found that this algorithm is much faster than NURBS based surface deformation algorithms. We also tried a thinplate spline algorithm which had performance issues and hence was rejected.
Fig. Haptic simulator in action, along with virtual envirnment
The main idea behind the deformation is a skeleton model made of nodes (cGELSkeletonNode) and links (cGELSkeletonLink) between them. Nodes are represented as spheres with a given radius and mass connected with elastic links with spring physics deï¬ned by elongation, ï¬‚exion and torsion properties. Every node has its physical properties (linear damping, angular damping, gravity ï¬eld deï¬nition) and provides methods to control force and torque. Considerable control over gel dynamics engine is achievable using different spring stiffness and damping parameters however, in this application, we have used constant values for these local properties.
PHANTOM Premium 1.5/6 DOF is used since the trocar insertion simulator needs torque feedback along with 3 DOF force feedback. The proposed system consists of a 3D scene and 2D menu as shown in figure 6. The 3D scene consists of human body and trocar, and the sub windows are included to give many different viewports to show underlying organs. The 2D menu can be used to control the system and edit haptic properties (stiffness, friction, and trocar force profiles) if necessary.
Haptic User Interface
The non linear force profile selected for the simulation consists of three regions: an initial constant spring stiffness region where the force increases linearly with displacement; a breaking force threshold that decreases suddenly the resistive force and a transition to a dead band where the resistive force is constant. The simulator allows the users to change these properties (K i.e. constant spring stiffness, breaking force, and dead band force) if desired. The default values for these properties are computed based on the mathematical data fitting technique as discussed in the previous sections. In the next section we also discuss a FEM based approach to model the haptic properties.
Users can learn about the trocar insertion with assistive haptic feedback throughout the task. Simulator is currently capable of providing basics of trocar insertion using haptic sensation. The simulator can also save the haptic information (position, force, and torque) based on expert interaction, and the system synthesizes the experts' recorded haptic information.
A Haptic User Interface (HUI) is also available for haptic modeling [Ref 8], which uses a closed loop feedback approach to gradually refine the underlying haptic mathematical model, based on expert surgeon interaction. In this virtual simulation, HUI focuses on editing force profile constants and improving the haptic feedback produced by the 6 DOF Phantom device. We plan to use this approach in the future once we have collected a large amount of sample data from expert surgeons.
3.7 Future Enhancements
We have described the underlying methodology of a six-degree-of-freedom haptic simulator for trocar insertion during minimally invasive surgical procedures. However, the fidelity of the simulation can be increased in future.
In the current algorithm for haptic rendering, the surface deformation results in curvature of the tissue and local change of properties. It is a complex phenomenon which involves simultaneous deformation, cutting forces and displacements caused by torquing motion.
Haptic perception for this phenomenon is not as accurate as it should be.
Current simulator only uses a virtual deformable plane which generates the haptic response based on underlying force profile programmed beforehand however; this force profile doesn't take into account local tissue properties and factors such as Body mass index, age of the patient and realistic surgical conditions.
More research and standardization is needed to construct a force profile based on real patient data.
Real CT data and image processing algorithms should be used to construct the 3D organ models and to place the organs at their anatomically correct positions.
Current system has to be improved by conducting user study consisting of a control group and an expert group. This will help to characterize the expert skill for this research problem. It establishes a standard to follow with regards to the procedure as well as process parameters.