International Journal of Robotic Engineering
Volume 6, Issue 1
An Affordable Haptic Device with Force and Torque Outputs
Yang Zheng, Xu Liu, Bin Zhao, Baibo Wu and Kai Xu*
Table of Content
Figure 1: a) The CombOmni device with the new....
a) The CombOmni device with the new stylus; b) Manual orientation matching is needed while switching the teleoperation from Arm L1 to Arm L2, if the device does not generate torque outputs.
Figure 2: A flowchart of the methodology.....
A flowchart of the methodology framework.
Figure 3: Coordinate's attachment: a) The CombOmni's....
Coordinate's attachment: a) The CombOmni's coordinates; b) Coordinates of each Omni device; c) Coordinates of the stylus elbow and the stylus handle; d) Schematic of the force decomposition from the two Omni devices for the desired wrench output of the CombOmni.
Figure 4: An intuitive and less optimal connection....
An intuitive and less optimal connection between the two Omni devices: a) The reachable axis directions (in red) of the original stylus.
Figure 5: Design optimization of the stylus....
Design optimization of the stylus elbow: There are four optimal configurations with the highest averaged φ3 angles; Since they are considered equivalent, one was adopted.
Figure 6: Workspace of the....
Workspace of the CombOmni.
Figure 7: The new stylus of the CombOmni...
The new stylus of the CombOmni: a) An exploded view.
Figure 8: Pose read-outs of the CombOmni while...
Pose read-outs of the CombOmni while held by a denso manipulator.
Figure 9: Experimental results: a) Tracked stylus...
Experimental results: a) Tracked stylus positions along three lines in the x-axis, y-axis, z-axis directions; b) Position reading errors.
Figure 10: Experimental quantification of the...
Experimental quantification of the dexterous workspace.
Figure 11: Experimental setup for the force...
Experimental setup for the force and torque quantifications.
Figure 12: Force outputs of: a) The 1st Omni...
Force outputs of: a) The 1st Omni device; b) The 2nd Omni device.
Figure 13: Force and torque outputs of the CombOmni....
Force and torque outputs of the CombOmni at: a) Point A; b) Point B; c) Point D; d) Point H.
Figure 14: Tracking performance of: a) A force...
Tracking performance of: a) A force profile; b) A torque profile.
Figure 15: Trajectory tracking inside a wrench....
Trajectory tracking inside a wrench subspace at Point A.
Figure 16: Stiffness quantifications at: a) Point....
Stiffness quantifications at: a) Point A; b) Point B; c) Point D; d) Point H.
Figure 17: Trajectory of the WpH1 in: a) The...
Trajectory of the WpH1 in: a) The virtual sphere; b) The virtual cube.
Figure 18: Virtual environment: a) Sketch map...
Virtual environment: a) Sketch map; b) Insertion of the virtual pin; c) The virtual pin's movement in the virtual hole.
Figure 19: Surgical teleoperation with the coordinate...
Surgical teleoperation with the coordinate assignment of: a) The camera arm and the grasper arm; b) The CombOmni; c) Teleoperate the camera arm; d) Teleoperate the grasper arm.
Table 1: Denavit-harttenberg parameters of each omni device.
Table 2: Comparison of the specifications.
- Hannaford B, Okamura AM, Haptics (2008) In: Siciliano B, Khatib O, Springer handbook of robotics. Part D: Springer, 719-739.
- Bergamasco M, Avizzano CA, Frisoli A, Ruffaldi E, Marcheschi S (2006) Design and validation of a complete haptic system for manipulative tasks. Advanced Robotics 20: 367-389.
- Kawasaki H, Mouri T (2007) Design and control of five-fingered haptic interface opposite to human hand. IEEE Transactions on Robotics 23: 909-918.
- Blake J, Gurocak HB (2009) Haptic glove with mr brakes for virtual reality. IEEE/ASME Transaction on Mechatronics 14: 606-615.
- Pacchierotti C, Sinclair S, Solazzi M, Frisoli A, Hayward V, et al. (2017) Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. IEEE Transactions on Haptics 10: 580-600.
- Wang F, Qian Z, Lin Y, Zhang W (2021) Design and rapid construction of a cost-effective virtual haptic device. IEEE/ASME Transactions on Mechatronics 26: 66-77.
- Arata J, Kondo H, Ikedo N, Fujimoto H (2011) Haptic device using a newly developed redundant parallel mechanism. IEEE Transactions on Robotics 27: 201-214.
- Steger R, Lin K, Adelstein BD, Kazerooni H (2004) Design of a passively balanced spatial linkage haptic interface. Journal of Mechanical Design 126: 984-991.
- Birglen L, Gosselin C, Pouliot N, Monsarrat B, Laliberté T (2002) SHaDe, a new 3-DOF haptic device. IEEE Transactions on Robotics and Automation 18: 166-175.
- Tholey G, Desai JP (2007) A general-purpose 7 DOF haptic device: Applications toward robot-assisted surgery. IEEE/ASME Transactions on Mechatronics 12: 662-669.
- Long GL, Collins CL (1992) A pantograph linkage parallel platform master hand controller for force-reflection. IEEE International Conference on Robotics and Automation (ICRA).
- Tsumaki Y, Naruse H, Nenchev DN, Uchiyama M (1998) Design of a compact 6-DOF haptic interface. IEEE International Conference on Robotics and Automation (ICRA).
- Stocco LJ, Salcudean SE, Sassani F (2001) Optimal kinematic design of a haptic pen. IEEE/ASME Transactions on Mechatronics 6: 210-220.
- Ueberle M, Buss M (2002) Design, control, and evaluation of a new 6 DOF haptic device. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
- Kim K, Kyun W, Youngil Y (2003) Design and analysis of a new 7-DoF parallel type haptic device: PATHOS-II. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
- Faulring EL, Colgate JE, Peshkin MA (2006) The cobotic hand controller: Design, control and performance of a novel haptic display. International Journal of Robotics Research 25: 1099-1119.
- Bernstein N, Lawrence D, Pao L (2013) Dynamics modeling for parallel haptic interfaces with force sensing and control. IEEE Transactions on Haptics 6: 429-439.
- Najdovski Z, Nahavandi S, Fukuda T (2014) Design, development, and evaluation of a pinch-grasp haptic interface. IEEE/ASME Transactions on Mechatronics 19: 45-54.
- Lee G, Hur S-M, Oh Y (2016) A novel haptic device with high-force display capability and wide workspace. IEEE International Conference on Robotics and Automation (ICRA).
- Hoshyarmanesh H, Zareinia K, Lama S, Sutherland GR (2021) Structural design of a microsurgery-specific haptic device: Neuro Arm PLUS Prototype. Mechatronics 73: 1-14.
- Guthart GS, Salisbury JK (2000) The intuitiveTM telesurgery system: Overview and application. IEEE International Conference on Robotics and Automation (ICRA).
- Hannaford B, Rosen J, Friedman DW, King H, Roan P, et al. (2013) Raven-II: An open platform for surgical robotics research. IEEE Transactions on Biomedical Engineering 60: 954-959.
- Xu K, Goldman RE, Ding J, Allen PK, Fowler DL, et al. (2009) System design of an insertable robotic effector platform for single port access (SPA) surgery. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
- Xu K, Zhao J, Fu M (2015) Development of the SJTU unfoldable robotic system (SURS) for single port laparoscopy. IEEE/ASME Transactions on Mechatronics 20: 2133-2145.
- Kobayashi Y, Sekiguchi Y, Noguchi T, Takahashi Y, Liu Q, et al. (2015) Development of a robotic system with six-degrees-of-freedom robotic tool manipulators for single-port surgery. International Journal of Medical Robotics and Computer Assisted Surgery 11: 235-246.
- Niemeyer GD, Guthart GS, Nowlin WC, Swarup N, Toth GK, et al. (2002) Camera referenced control in a minimally invasive surgical apparatus. Intuitive Surgical Inc Assignee, 23.
- Lin Y, Sun Y (2011) 5-D force control system for fingernail imaging calibration. IEEE International Conference on Robotics and Automation (ICRA).
- Xu K, Zhao B, Zhang Sa, Zhang Z, Xing N (2017) Design of a haptic master device for teleoperation applications. IEEE International Conference on Robotics and Biomimetics (ROBIO).
- Zhang Sa, Chen Y, Li Q, Zhao B, Xu K (2018) Kinematic optimization of a continuum surgical manipulator. IEEE International Conference on Robotics and Biomimetics (ROBIO).
Yang Zheng, Xu Liu, Bin Zhao, Baibo Wu and Kai Xu*
State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, China
Kai Xu, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan RD, Minhang District, Shanghai, 200240, China.
Accepted: May 18, 2021 | Published Online: May 20, 2021
Citation: Zheng Y, Liu X, Zhao B, Wu B, Xu K (2021) An Affordable Haptic Device with Force and Torque Outputs. Int J Robot Eng 6:032.
Copyright: © 2021 Zheng Y, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
A haptic device with both force and torque outputs is preferred in many applications. It generates faithful interaction between an operator and a virtual or remote environment and facilitates teleoperation. However, researchers do not have sufficient access to such a haptic device due to the facts that the commercially available ones are usually expensive and the state-of-the-art research prototypes are not readily reproducible outside the original laboratories. This paper presents the design and the experimental characterization of the CombOmni, an affordable haptic device with both force and torque outputs. The CombOmni device is constructed from two Sensible Omni haptic devices. The Omni device only provides force outputs and its original structure doesn't allow effective connection between two such units. By carefully designing a new stylus with two stylus elbows that better connects the two Omni devices, the CombOmni can be easily constructed. Kinematics-based design optimizations and the system integration are presented. The CombOmni has a translational workspace larger than 100 × 200 × 200 mm3, and its rotational workspace is also quantified. With the force outputs calibrated on the Omni devices, the CombOmni was shown to provide force outputs up to 5.8 N and torque outputs up to 150 mNm, before saturating the Omni devices. The stiffness is 2.22 N/mm, 1.96 N/mm and 4.26 N/mm in the XYZ directions. Then the CombOmni devices were used to render two virtual environments, to simulate a virtual scenario of pin insertion and to teleoperate two surgical manipulators. With these experimental characterizations, the CombOmni might become a viable option for a haptic device with both force and torque outputs in a cost-effective way.
CombOmni, Design optimization, Haptic device, Kinematics, Phantom omni
HAPTIC devices reproduce kinesthetic and tactile sensation between real/simulated environments and an operator. Depending on the specific purposes, their forms and theoretical foundations can be quite different . In virtual reality applications, haptic devices usually interact with one's fingers or hands to generate realistic feelings of the simulated surroundings [2-6]. In teleoperation applications (e.g., robotic surgery), a haptic device on the master console enables kinesthetic interactions, sensing the kinematic status of its handheld stylus, while generating agile and programmable wrench outputs (preferably including both force and torque components). The programmable wrench outputs of the haptic devices are usually generated by setting different output levels on the actuators at a high frequency (e.g., at 1 KHz), according to the devices' dynamic models.
Examining the existing haptic devices that can be utilized for teleoperation, categorization can be proposed regarding the core function. Please note that an additional end effector, such as a gripper, can always be incorporated, leading to increased number of the device's DoFs (Degrees of Freedom). But this additional DoF is not taken into consideration in the proposed categorization below, because the device's essence does not change with or without this additional DoF. Another categorization examining the control method is in .
• 3-in-3-out devices sense 3-DoF inputs (usually positions) and generate 3-DoF outputs (usually forces). Examples of commercial products mainly include the Novint Falcon (Novint Technologies, Inc.), the delta.3 device and the omega.3 device (Force Dimension, Inc.). Examples of research prototypes include the joystick mechanism , the DELTA-R device , the SHaDe device , the ECUST-haptic robot , etc. Among them, the SHaDe device only has rotational DoFs. It takes orientation inputs and generates torque outputs.
• 6-in-3-out devices sense 6-DoF inputs (positions and orientations) and generate 3-DoF outputs (usually forces). Examples of commercial products mainly include the Omni device (now as Touch from 3D Systems, Inc.) and the omega.6 device (Force Dimension, Inc). A representative research prototype is the laparoscopic interface in .
• 6-in-n-out devices sense 6-DoF inputs (positions and orientations) and generate multi-DoF outputs (including forces and torques components). Early examples include the designs that used hybrid serial-parallel structures [11,12]. Then the delta.6, delta.7 and sigma.7 devices (Force Dimension, Inc.), as well as the Phantom Premium (3D Systems, Inc.), have emerged as commercially available products. But they are substantially more expensive than the 6-in-3-out products. The research prototypes include the haptic pen , the VISHARD6 , the PATHOS-II , the haptic cobot , the modified DELTA-R device (equipped with a 3-DoF rotation mechanism) , the CU haptic interface , the haptic interface for pinch & grasp , the Virtual Power device , and the neuroArmPLUS device .
Among the existing haptic devices, structural characteristics can be seen as follows. Serial structures are often used to enlarge the workspace, while parallel structures are utilized to increase stiffness and force output capabilities. Serial-parallel hybrid structures were also used to reach a balance between adequate workspace and proper stiffness.
6-in-n-out devices are highly preferred in teleoperation and virtual reality tasks since they generate both force and torque outputs, possibly leading to more faithful interaction between the operator and the haptic device. However, the major limitation on the current development is that researchers do not have sufficient access to these 6-in-n-out haptic devices due to the following reasons: i) The 6-in-n-out products (unit price usually above USD $50,000) are substantially more expensive than the 6-in-3-out ones (unit price around USD $3,000 or less); ii) The state-of-the-art research prototypes cannot be readily reproduced outside the original laboratories.
Furthermore, the 6-in-n-out haptic devices are especially useful in robotic surgeries. For example, when a pair of haptic devices are used to teleoperate a surgical robotic system with multiple arms, the operator needs to switch the mapping between the haptic devices and the teleoperated arms (including a vision arm and a few manipulation arms) [20-25]. Force and torque outputs of the haptic device can actively change the position and orientation of its stylus so as to quickly establish the mapping between the stylus and the arm's end effector [21,26]. As shown in Figure 1b, if the haptic device does not generate torque outputs, the operator will have to manually orient the stylus to match the orientation of the surgical end effector. This manual matching process can severely hinder the teleoperated robotic surgical process. What's more, force and torque outputs from the haptic device can better render the motion status of the teleoperated arm by slowing down excessively fast inputs or preventing the operator from moving or turning to cautioned regions.
This paper hence proposes an accessible solution for constructing a 6-in-5-out haptic device, the CombOmni device. It combines two Sensible Omni devices (now called Touch supplied by 3D Systems, Inc.), as shown in Figure 1. The core incentive is to provide a convenient and cost-effective option that is simple enough for the researchers who prefer 6-in-n-out haptic devices. The presented experimental characterizations showed that the constructed CombOmni has comparable specifications to those expensive 6-in-n-out ones. The CombOmni is developed with its intended applications in robot-assisted surgeries. It can certainly find other applications in teleoperation or virtual reality tasks.
The CombOmni combines two Omni devices that can only generate force outputs. The original structure of the Omni device doesn't allow effective connection between two such units. Hence the core contributions of this paper lie on the design and kinematics-based optimization of the new stylus that effectively connects the two Omni devices. The new stylus can be easily 3D printed and installed with minimal modifications to the Omni devices. A similar idea was proposed to use two Novint Falcon devices to generate 5-dimensional outputs  (forces in three directions and torques in two directions). Another similar setup using two Omni devices for a master device in a surgical robot is in . In the two aforementioned examples, the way to connect those two haptic devices was not optimized. Contrarily, the CombOmni used a carefully optimized connection to substantially improve the available workspace. The CombOmni concept was previously proposed in . The kinematics-based optimization was further confirmed and the wrench generation statics was derived. The device's performance was then intensively characterized via a series of experiments and application demonstrations. Due to the facts that the specifications of the devices in [25,27] are not available, the CombOmni was compared to a few commercially available products in the "Characterization of the Positioning Accuracy" subsection. It shows that the CombOmni may become a viable option for a haptic device with both force and torque outputs in a cost-effective way.
This paper is organized as follows. The "Design and Optimization of the CombOmni" section presents the design overview and the kinematics-based optimization of the CombOmni device. The construction of the stylus and the infrastructure is elaborated in the "System Description of the CombOmni" section. Comprehensive experimental characterizations are reported in the "Comprehensive Experimental Characterizations" section, while the conclusions are presented in the "Conclusions" section. Besides, a flowchart of the methodology framework is presented in Figure 2 for readers to more conveniently follow the paper's contents.
Design and Optimization of the CombOmni
Since the CombOmni uses two Omni devices, it is critical to find an efficient way to connect them in order to effectively utilize the workspace of each Omni device. After all, the CombOmni's intended application is teleoperation, and it is vital to have a properly sized workspace. The size of its workspace directly affects how the teleoperated subjects can be guided to move around to accomplish a task. Having an acceptable size of the workspace is a primary goal for the design of the new stylus. The CombOmni's other specifications, such as positioning resolutions, stiffness, force output ranges, etc., indirectly depend on the original specifications of the Omni device. It should be noted that if the intended application had relied on other specifications of the CombOmni, a different optimization scheme should be used.
The Omni device consists of six revolute joints, as illustrated in the "Nomenclatures and Coordinates" subsection and "Kinematics of the Omni Device" subsection. As shown in Figure 1 and Figure 3, the proximal three joints (θi1, θi2 and θi3) position the gimbal center, while the distal three joints (θi4, θi5 and θi6) orient the stylus. If the original styluses of the two Omni devices are directly connected (indicated in Figure 4), the CombOmni will have very limited workspace due to the limited motion ranges of the θi4 and θi5 joints of the Omni device. An optimization based on the kinematics is hence presented in the "Design and Optimization of the New Stylus" subsection to design a stylus elbow that better utilizes the θi4, θi5 and θi6 joint motion ranges of the Omni device. The new stylus is desired to reach a wider range of orientations within the workspace. This is particularly important for surgical teleoperation where the end effector may need to be teleoperated to approach a target organ from various angles.
With the stylus elbow designed, the two Omni devices are connected more effectively. Then another optimization is presented in the "Optimized Arrangement of the Two Omni Devices" subsection for determining the relative positioning of the two Omni devices to form the CombOmni.
Nomenclatures and coordinates
The following coordinates are defined to describe the CombOmni's kinematics, referring to Figure 3.
• World Coordinate is located at the CombOmni's base between the two Omni devices.
• Omni Device Coordinates (i = 1, 2; j = 0, 1, 2, ..., 6) are attached to base and the jth link of the ith Omni device according to the Denavit-Hartenberg rules.
• Stylus Elbow Coordinates (i = 1, 2; k= 1, 2) are attached to the stylus elbow that is mounted on the ith Omni device.
• Handle Coordinates (i = 1, 2) are attached to the CombOmni's stylus handles that are rotatable on the ith Omni device, respectively.
Kinematics of the omni device
Referring to Figure 3b, the kinematics of each Omni device can be described by assigning seven coordinates according to the Denavit-Hartenberg rules. For visualization clarity, Y axes of the coordinates in Figure 3b are hidden. The homogeneous transformation matrix relating the adjacent coordinates is as follows.
i = 1, 2, j = 1, 2, ..., 6 (1)
The homogeneous transformation matrix from to is written as in Equation (2).
i = 1, 2, j = 1, 2, ..., 6 (2)
The Denavit-Hartenberg parameters of each Omni device are listed in Table 1.
Please note that the θi4 and θi5 joints (the panning and the tilting motions) have limited ranges while the θi6 joint (the rolling motion) has the biggest rotation range.
Design and optimization of the new stylus
As shown in Figure 4, an intuitive way to connect the two Omni devices is to simply connect the original styluses coaxially. One can soon see that this connection is not the optimum. The axis direction of the original stylus mainly depends on the θi4 and θi5 angles (even though affected by the θi1, θi2 and θi3 angles). Please refer to the joint motion ranges in Table 1. Then the reachable directions of the axis of the original stylus can be visualized in Figure 4a, in the view of . The red surface indicates reachable directions.
If the original styluses are connected coaxially, this limited reachable direction range will severely limit the workspace of the CombOmni.
It is highly preferred that the axis direction of the new stylus can be oriented arbitrarily, when the new stylus is used for the connection of the two Omni devices.
The stylus elbow is hence introduced in the new stylus design as indicated in Figure 3a. In Figure 3c, the axis of the original stylus is rotated to the axis of the new stylus by two rotations of δ1 and δ2.
Referring to Figure 3c, the δ1 angle is the rotation from to about the shared axes. The δ2 angle is the rotation from to about the shared axes. The axis is in line with the new stylus handle axes
The stylus elbow is the introduced key design idea. With the stylus elbow, the stylus axis is now oriented using the θi4, θi5 and θi6 angles (instead of using just θi4 and θi5 angles). Essentially, the θi6 motion range is utilized to arbitrarily orient the axis of the new stylus. The axis of the new stylus is then addressed to have omnidirectional reachability. With the omnidirectional reachability, positioning one Omni device with respect to the other will not involve verifying the connection feasibility. Then the CombOmni's workspace can be conveniently optimized by varying the relative pose between the two Omni devices, as reported in the "Optimized Arrangement of the Two Omni Devices" subsection.
Infinite pairs of the δ1 and δ2 angle values can realize the omnidirectional reachability of the new stylus axis. An optimization was carried out to determine the δ1 and δ2 values.
With the stylus elbow installed, the coordinate transformation matrix is written as in Equation (3). At the same time, can also be parameterized by the ZYZ Euler angles as in Equation (4).
Where can be extracted from Equation (2); are simple rotations about the axes.
Omnidirectional reachability of the new stylus axis depends on the δ1 and δ2 angle values. An adoptable pair of δ1 and δ2 values means that there exists at least one set of the θi4, θi5 and θi6 values within their joint limits to realize an arbitrary pair of the φ1 and φ2 values. Then for this pair of φ1 and φ2 values, the φ3 value will have a motion range: φ3 can be varied without violating the joint limits of the θi4, θi5 and θi6 joints.
To determine the δ1 and δ2 values, an optimization was carried out to maximize the averaged φ3 range for any φ1 and φ2 values. The optimization was implemented in an enumerative manner as follows. The optimization for Omnidirectional reachability may seem to put more significance on φ1 and φ2. This is because the ZYZ parameterization is used and the φ1 and φ2 values fully determine the direction of the axis.
The δ1 and δ2 values were varied within [-90°, 90°] in increments of 1°. Then for a pair of δ1 and δ2 values, the φ1 and φ2 values are varied in increments of 5° to generate all possible directions of the new stylus axis, while the φ3 value is varied in increments of 1°. The δ1 and δ2 value pair is admitted only if for each axis direction generated by the φ1 and φ2 angles, there exists at least one φ3 value under which the θi4, θi5 and θi6 values obtained from Equation (5) are within their joint limits. Equation (5) is obtained from Equation (3) and Equation (4).
Then the averaged φ3 motion range under all the axis directions generated by scanning the φ1 and φ2 angles are plotted in Figure 5 with respect to the δ1 and δ2 angle values.
From Figure 5, it can be seen that there exist four optimal configurations with the highest averaged φ3 angles. They can be considered equivalent since they are symmetric in the plot coordinate. Then the configuration of δ1 = 37° and δ2 = -37° is adopted. In this optimal configuration, the averaged φ3 value is 202.1°.
Optimized arrangement of the two omni devices
With the stylus elbow, the new stylus axis can be arbitrarily oriented. Because of the fact that the θi6 joint is partially used to orient the new stylus axis, the new stylus has a reduced rotary motion range about its own axis Hence an additional joint is incorporated between the stylus handle and the stylus elbow. The mechanism is explained in the "System Description of the CombOmni" section.
The handle coordinate is attached to the stylus handle that is installed on the stylus elbow of the ith Omni device, as shown in Figure 3c. is rotated from for an angle of θHi. and share their origins and the Z axes. The origins of are treated as the CombOmni's input point, while the orientations of the and can simulate the jaws of a gripper (or other medical instruments) in a robot-assisted surgery.
The length of the new stylus is the distance between the two gimbal centers (indicated in Figure 1) of the two Omni devices. It is obvious that this length should be made shorter in order to generate larger workspace of the CombOmni. As explained in the "System Description of the CombOmni" section, the distance L is 80 mm, considering the necessity of including the required components and avoiding interference.
The second optimization determines the relative position of the two Omni devices, in order to better use their given specifications. For the convenience in assembly, the two Omni devices are assumed to be placed on the same plane. Three arrangement parameters should be determined: The relative position (px and py) and the orientation (γ) of the 2nd Omni device with respect to the 1st one.
With the stylus elbows and the stylus handles installed, the CombOmni's workspace is obtained as follows. The workspace (W-1) of the 1st Omni device is defined as the volume that the gimbal center of the 2nd Omni device traverses when the 1st Omni scans its joint space. This definition of W-1 makes the arrangement optimization procedure straightforward, as shown below.
Then the workspace (W-2) of the 2nd Omni device concerning its own gimbal center, in is generated, as shown in Figure 6. Moving the 2nd Omni device will translate and rotate its workspace as well. When the 2nd Omni device is arranged with respect to the 1st one, the volume of the overlapped workspace (between the workspace W-1 and the movable workspace W-2) can be obtained. The placement of the 2nd Omni device with respect to the 1st one is determined when the overlapped workspace volume is maximized as in (6), varying the three placement parameters: and
The simulated annealing algorithm was implemented in MATLAB to solve this optimization. The initial values of were used. And the results of were obtained. The structural parameters were rounded to for simplicity. The CombOmni's workspace is then plotted in Figure 6, by putting the two Omni devices at in
As indicated in Figure 6, the overlapped workspace has an irregular shape. Hence a rated workspace is defined with its center at [0 mm 0 mm 150 mm]T in and size of 100 × 200 × 200 mm3. Then the experimental characterizations for the CombOmni were mostly conducted concerning this rated workspace, as presented in the "Comprehensive Experimental Characterizations" section. The rated workspace is chosen so to form a balanced comparison with the commercially available products in Table 2. If the workspace is chosen to be bigger (but still within the CombOmni's actual workspace), the reachable orientations of the axis may be more restricted at the points that are close to the boundaries of the CombOmni's actual workspace.
When the arrangement of the two Omni devices is determined, the complete kinematics of the CombOmni can be derived as follows.
The origin of (base of the 1st Omni device) is placed at in , while the origin of is located at in . and are coaxial but in the opposite directions.
The homogeneous transformation matrix of in can be obtained as in Equation (7).
Where ,is the rotated angle between and . θH1 is measured by the rotary potentiometer as explained in the "System Description of the CombOmni" section.
Similarly, the handle frame is attached to the other handle. The transformation matrix is derived as in Equation (8).
Where θH2 is the rotated angle between and , and
Statics of the combomni
The motivation of designing the CombOmni is to use the force outputs from the two Omni devices to generate wrench (including force and torque) outputs to form an affordable 6-in-5-out haptic device. The decomposition of the desired wrench output is as follows.
Referring to Figure 3d, Equations (9) and (10) hold.
Where are the force outputs from the first and the second Omni devices for the desired force output of the CombOmni; are the force outputs from the two Omni devices for the desired torque output.
Note that because the CombOmni is a 6-in-5-out device, the unachievable output dimension in an arbitrary desired wrench output is removed by the projection in the left-hand-side of Equation (10). This is needed because the forces from the two Omni devices cannot generate a torque about this axis. This component has to be excluded to solve Equationa (9) and (10).
Then the total force outputs from the first and the second Omni devices are obtained as in Equation (11). Please note that, Equation (10) does not give unique solutions to and , as they can contain opposite components in the direction. An additional condition was integrated to set the with zero components in the direction.
i = 1 and 2 (11)
System Description of the CombOmni
The CombOmni is composed of two Omni devices whose relative position is determined as in the "Optimized Arrangement of the Two Omni Devices" subsection. Referring to the relative position, an acrylic board was cut to mount the two Omni devices.
The original stylus of the Omni device can be unplugged as in Figure 1a. Apparently, a 6.35 mm stereo audio jack plug was used to connect the stylus. Since the distance between the two gimbal centers should be minimized to increase the workspace, this connector was hence cut off, in order to reduce possible interferences during movements.
The new stylus, which was used to connect the two Omni devices, consists of two stylus elbows, two stylus handles, two potentiometers and a few miscellaneous components. The stylus elbow is installed on the shortened stylus connector. A screw is tightened to lock the stylus elbow on the connector, as indicated in Figure 7.
Referring to Figure 7a, the following parts are assembled axially on a ∅2 mm central shaft: A pair of miniature ball bearings, a cover plate, a potentiometer (SV01A1 from muRata Manufacturing Co., Ltd.), the stylus elbow, the stylus handle, a torsional spring, the other stylus handle, the other stylus elbow, the 2nd potentiometer, the 2nd cover plate, and another pair of miniature ball bearings. Two nuts are tightened on the shaft's two ends to axially fix all the parts.
The stylus elbow houses the potentiometer that is retained by the cover plate. A groove is reserved in the stylus elbow to pass the wires of the potentiometer. A D-shaped shoulder of the stylus handle is inserted into the potentiometer so that rotation of the stylus handle with respect to the stylus elbow will be measured by the potentiometers. The two measured angles (θH1 and θH2) between the stylus elbow and the stylus handles are needed for calculating the handles' kinematics.
The CombOmni's two Omni devices were connected to a computer via IEEE 1394 bus. APIs (Applications Programming Interface) from the OpenHaptic™ libraries are available for reading the joint values & the poses of the Omni devices, as well as generating programmed forces at the gimbal centers. Besides, a multi-function card (PCI-1710 from Advantech Inc.) was plugged into the computer for reading the potentiometers' voltages in the new stylus. A resolution of 0.08° can be achieved with the 12-bit ADC conversion. In the following experiments and application demonstrations, the CombOmni device was controlled by a customized C++ program with APIs from the Open HapticTM libraries.
Comprehensive Experimental Characterizations
To verify the efficacy of the proposed CombOmni design, comprehensive experimental characterizations were carried out.
The CombOmni is a 6-in-5-out haptic device. It shall be able to sense the position and orientation of the stylus handle, as well as generate force and torque outputs.
The accuracy of the sensed positions and orientations are quantified as in the "Characterization of the Positioning Accuracy" subsections and the "Characterization of the Dexterous Workspace" subsection respectively. The static and dynamic outputs of the force and the torque are quantified as in the "Verification of Static Force and Torque Outputs" subsection and the "Verifications of Dynamic Force and Torque Outputs" subsection. The CombOmni's stiffness is examined as in the subsection "Stiffness Quantification" subsection. Three application scenarios of the CombOmni are described in the "Application Case I: Virtual Sphere and Cube" subsections to the "Application Case III: Surgical Teleoperation" subsection respectively. A comparison between the CombOmni and a few commercially available haptic devices is presented in the "Comparison with Commercial Haptic Devices" subsection to indicate the value of the CombOmni device.
Characterization of the positioning accuracy
A Denso manipulator was used to hold and move the CombOmni's new stylus inside its workspace, as shown in Figure 8. Using the Denso manipulator provides accurate, repeatable and known inputs to the CombOmni for quantifying its position and orientation sensing capabilities.
To confirm the movement of the Denso manipulator, two markers were attached to the base and the stylus handle of the CombOmni. The markers were tracked using an optical tracker (Micron Tracker SX60 from Claron Technology Inc.). The reference marker represents a frame with a known translation with respect to the world coordinate defined as in Figure 3a. The reference marker was not directly attached at the world coordinate because the stylus might block the view of the optical tracker. On the other hand, the stylus marker also has a known transformation with respect to the handle coordinate. The positions of the stylus handle in were obtained from the tracked positions of the stylus marker with respect to the reference marker and plotted in Figure 9a.
With the position and orientation readings of the Omni devices and the readings of the potentiometer, positions and orientations of the stylus handle can be calculated using Equation (7). The errors between the tracked positions and the calculated positions (a.k.a. the position reading errors of the CombOmni) are plotted in Figure 9b. The reading errors are mostly under 0.15 mm. The result is reasonable given the position resolution of the Omni device of 0.05 mm as listed in the datasheet and the limited precision of the 3D printed parts (usually around 0.1 mm). Similar results were obtained throughout the CombOmni's rated workspace.
The results can be acceptable for the CombOmni's intended applications of teleoperation. The inputs may be proportionally scaled, if higher motion input accuracy is needed at a local area (e.g., a 1.0-mm input is sent as 0.10-mm motion command for teleoperation, while suturing a thin vessel).
Characterization of the dexterous workspace
Dexterous workspace of the CombOmni inside its rated workspace greatly affects the usefulness of the haptic device, because an operator needs to orient the stylus handle as free as possible to interact with a virtual or remote environment.
As shown in Figure 6, the CombOmni has a rated workspace of 100 × 200 × 200 mm3 with the center located at [0 mm 0 mm 150 mm]T in . Then the Denso manipulator was utilized to orient the CombOmni's new stylus at different positions (Points A to L in Figure 10).
The reachable orientation of the CombOmni at each point is quantified by solid angle (in steradian) that can be reached by the handle axis of the CombOmni (The axis in Figure 3). The handle's rotation about the axis can always be sensed by the potentiometers installed inside the CombOmni's stylus elbow. The quantification of the aforementioned solid angle is as follows.
The Denso manipulator was programed to maneuver the CombOmni to orient the CombOmni's handle axis to different directions while maintaining the CombOmni's handle position at Points A to L as indicated in Figure 10. A six-dimensional force sensor (Nano17 from ATI Industrial Automation Inc.) was used as in Figure 11. The direction was considered reachable when both the following two conditions were satisfied: i) The position of the CombOmni's handle obtained from the readings of the two Omni devices was at the designated point; ii) The forces and the torques detected by the Nano17 force sensor were smaller than 1 N and 100 mNm. When the dexterous workspace was quantified, the CombOmni's handle moved passively with the Denso manipulator. If the Nano17 force sensor detected large forces or torques, structural interference or joint motion limits must have occurred.
The desired goal directions were parameterized by a yaw angle and a pitch angle in increments of 1°. And the reachable directions of the CombOmni's handle are visualized as red patches in Figure 10. Since the two Omni devices are placed symmetrically with respect to the rated workspace, the twelve selected Points (A to L) representatively show the CombOmni's dexterous workspace.
The dexterous workspace is best at the center of the rated workspace (10.78 sr). The original dexterous workspace of an Omni device is 7.07 sr (shown as the red patch in Figure 4). The CombOmni's dexterous workspace is even bigger because the introduced stylus elbow (in Figure 8) utilizes the θi6 joint of the Omni device to enhance the orienting capability of the new stylus. On the contrary, only the θi4 and θi5 joints are used for orienting the stylus in the original Omni device.
Verification of static force and torque outputs
To verify the CombOmni's force and torque outputs performance, the experimental setup as in Figure 11 was used. A 6-dimensional force sensor (Nano17 from ATI Industrial Automation Inc.) was held by the Denso manipulator. Then the CombOmni was commanded for a desired wrench output. As the Denso manipulator held the CombOmni's new stylus still, the readings from the Nano17 force sensor were used to quantify the CombOmni's outputs.
It was noted that the Omni devices do not generate the exact output force according to the command it received via its OpenHaptics Toolkit API. It is hence necessary to calibrate the Omni devices, using a similar setup to Figure 11.
The calibrations for the output forces were performed in the XYZ directions in . The commanded force for the Omni devices increased from 0 N to 3 N in increments of 0.2 N and the actual output force was recorded using the Nano17 sensor. The measured forces are plotted in Figure 12. It is clear that the output force is proportionally lower than the commanded values. Then the output force compensation of the Omni devices can be formulated as follows.
Where K1 = diag (1.19, 1.28, 1.16) and K2 = diag (1.25, 1.19, 1.17) are the compensation matrices such that the compensated force shall be used in the OpenHaptics Toolkit API to generate the desired force . All the coefficients were obtained by linear regression of the measurement results.
After the Omni devices were calibrated, the force/torque outputs of the CombOmni were verified. As in Figure 11, the Denso manipulator moved the CombOmni's new stylus to four representative points (Points A, B, D and H in Figure 10). The CombOmni's force outputs were verified in the XYZ directions in , while the torque outputs were verified in the YZ directions.
The measured force and torque outputs are plotted in Figure 13. It can be seen that the CombOmni has relatively accurate force and torque outputs with the force output range of ± 5 N in the YZ directions and ± 3.5 N in the X direction at Points A and B. As the Points D and H actually lie outside the recommended workspace of the Omni device, the accuracy of the force and torque outputs deteriorates. The Omni device only has a recommended workspace of 70 × 160 × 120 mm3.
Verifications of dynamic force and torque outputs
Performance of the dynamic force and torque outputs was quantified by tracking independent force and torque profiles.
The CombOmni's new stylus was held by the Denso manipulator at Point A. The CombOmni was then commanded to follow a sinusoidal force trajectory with an amplification of 3 N and a period of 4 s in the XYZ directions respectively. Then the CombOmni was commanded to follow a sinusoidal torque trajectory with an amplification of 100 mNm and a period of 4 seconds in the YZ directions.
The force and torque tracking performances of the CombOmni are plotted in Figure 14. While tracking dynamic force and torque trajectories, the force and torque output errors can be up to 0.55 N and 35 mNm respectively. The results are slightly worse than those in the "Verification of Static Force and Torque Outputs" subsection.
The CombOmni can track both independent and combined force and torque profiles. To verify this performance, the CombOmni's new stylus was again held by the Denso manipulator at Point A. The CombOmni was then commanded to follow a trajectory in a wrench subspace. The trajectory begins at [0 N 0 mNm 0 mNm]T for the wrench subspace and expands inside a force and torque cube with the size of 3 N × 150 mNm × 150 mNm as in Figure 15.
The trajectory-tracking results show that the CombOmni is capable of generating combined force and torque outputs.
Stiffness is an important specification of a haptic device. The stiffness of the CombOmni is defined as the output force per unit length under an external disturbance at the new stylus and it was quantified here.
Using the experimental setup in Figure 11, the CombOmni's new stylus was held by the Denso manipulator to four representative points (Points A, B, D and H in Figure 10). The Denso manipulator then disturbed the CombOmni's new stylus to move in the XYZ directions in , respectively. This disturbance on the CombOmni would generate position deviations at the gimbal centers of the two Omni devices. The Omni devices can thus generate force outputs according to the position deviations.
As stated in the datasheet, the Omni device has stiffness of 1.26 N/mm, 1.02 N/mm and 2.31N/mm in the XYZ direction in . Using these stiffness settings, each Omni device can be programmed to generate a force according to a position deviation. Then the total force output of CombOmni was recorded by the Nano17 sensor, as the Denso manipulator disturbed the new stylus.
The measurement results are plotted in Figure 16. Lines were fitted to the measured forces with respect to the given disturbances. The slopes of the fitted lines indicate the CombOmni's stiffness in different directions. The stiffness in the XYZ directions is 2.22 N/mm, 1.96 N/mm, and 4.26 N/mm at Point A, respectively. The stiffness is slightly less than twice those of the Omni devices, possibly due to imperfect loading conditions in the experiments.
The torsional stiffness about and in under the pose shown in Figure 3a was also quantified, involving the distance between the two gimbal center of 80 mm. The torsional stiffness at these four points is as follows: Point A: 6.82 Nm/rad about and 3.14 Nm/rad about ; Point B: 6.75 Nm/rad about and 2.91 Nm/rad about ; Point D: 6.48 Nm/rad about and 2.77 Nm/rad about ; and Point H: 6.22 Nm/rad about and 2.38 Nm/rad about . Please note that the CombOmni device cannot generate an active torque about in , since it is a 6-in-5-out device.
Application case I: Virtual sphere and cube
In Application Case I, the CombOmni was programmed to interact with a virtual sphere and a virtual cube.
A virtual sphere with a radius of rs = 50 mm and the center at (Point A) in was set up to evaluate the haptic response of the CombOmni. The stylus position was programmed to be restricted inside the virtual sphere. When the moved inside the virtual sphere, the CombOmni would not generate any force outputs to the user. When the was moved by the user to approach the surface of the virtual sphere, the CombOmni would produce a force output pointing to the center of the virtual sphere to prevent the stylus from leaving the sphere. The force follows a spring model with stiffness of k = 1.5 N/mm as in Equation (13).
Where is the distance between and
Similarly, a virtual cube with the center at and a side length of lc = 100 mm was set up. When the approached a cube surface, the CombOmni would generate the force pointing inside the cube surface. The force again follows a spring model with stiffness of k = 1.5 N/mm as in Equation (14).
Where is the offset between the and the cube center ; and represent the X, Y or Z component of the vector and D.
As shown in Figure 17, the trajectory of the CombOmni's new stylus clearly shows the stylus was effectively restricted by the virtual sphere and the virtual cube.
The working time of the CombOmni can be set for an arbitrary period of time, as long as the user input does not stall and overheat the motors inside the CombOmni device.
Application case II: Virtual pin insertion
To demonstrate the realized torque outputs, Application Case II was conducted for a virtual pin insertion task.
The virtual environment was constructed with a free space, a pin, and a wall with a deep hole inside the wall. The virtual pin with a length of 80 mm was attached to the stylus.
When the pin was manipulated in the free space, the CombOmni did not apply any wrench to the user (namely, the virtual pin was not subjected to any restrictions).
When the pin approached the wall, the wall prevented the stylus from penetrating unless the pin and the hole were closely co-axial (allowed position and angular deviations of 1 mm and 1° respectively). When the pin was inserted, the force and the torque followed an elastic model with linear stiffness of k = 1.5 N/mm and torsional stiffness of c = 80 mNm/degree. In other words, when the stylus moved away from the hole axis, every 1 mm translation generated 1.5 N restraining force; When the stylus rotationally deviated from the hole axis, every 1 degree inclination generated 80 mNm restraining torque. After insertion, the allowed motions were translation and rotation along and about the hole axis.
The pin insertion process is shown in Figure 18 as well as in the multimedia extension. The working time of the CombOmni can be set for an arbitrary period of time, as long as the user input does not stall and overheat the motors inside the CombOmni device.
Application case III: Surgical teleoperation
Application Case III demonstrates the CombOmni's usefulness in facilitating teleoperation, as shown in Figure 19.
The CombOmni firstly teleoperated the camera arm for adjusting the FoV (Field of View). Then the CombOmni teleoperated the grasper arm for a threading task. The descriptions of the continuum camera arm and the grasper arm can be found in .
As shown in Figure 19a, the tip motions of the camera arm and the grasper arm are represented by a camera coordinate and a grasper coordinate , respectively, described in the base coordinate In teleoperation, the CombOmni's world coordinate is registered to and the motions of the CombOmni's stylus coordinate are used to teleoperate the camera arm or the grasper arm. The CombOmni's stylus coordinate is at the middle between the two handle coordinates and .
After the CombOmni finished teleoperating the camera arm for adjusting the FoV, it was about to be switched to the grasper arm. Torque outputs then actively changed the stylus's orientation to match that of the grasper such that teleoperation of the grasper arm for a threading task can be conveniently initiated. The angle between the two handles coordinates and was used to control the opening angle of the grasper. Please also refer to the video clip in the multimedia extension.
What's more, the CombOmni's force and torque outputs are also highly preferred for generating damping wrenches if the operator moves or turns the stylus too fast, or for preventing the stylus from entering cautioned regions, possibly leading to a safer teleoperation.
The working time of switching the CombOmni's handle to match another arm's pose is set to five seconds. A switching motion too fast might introduce unnecessary vibrations. Before and after the switching motion, the working time of teleoperation using the CombOmni device can be as long as the user input does not stall and overheat the motors.
Comparison with commercial haptic devices
It is expected that the CombOmni can become an affordable and conveniently accessible haptic device for researchers who prefer a 6-in-n-out device. The CombOmni's specifications were hence compared to a few commercially available devices including the Omni device, the Delta.6 and Sigma.7 devices, in order to highlight the CombOmni's cost performance.
According to Table 2, the CombOmni has a translational workspace close to that of the Sigma.7 device. The listed translational workspace of the Omni device from its datasheet is quite small, but its actual workspace is larger. The delta.6 device has a much bigger translational workspace but the dexterous workspace is quite small. The CombOmni has the biggest dexterous workspace, resulting from the introduced stylus elbow. This makes it an ideal candidate for tasks of surgical teleoperation as one may want to orient a surgical tool freely.
The force and torque output capabilities of the CombOmni are lower compared with the Delta.6 and the Sigma.7 devices, but double those of the Omni device.
This paper presents the design, kinematics-based optimization, system integration and comprehensive experimental characterization of the CombOmni device, an affordable haptic device with force and torque outputs.
The main motivation is to provide an affordable and convenient solution for researchers who do not have sufficient access to haptic devices with both force and torque outputs: The commercial products are usually high-priced and the state-of-the-art research prototypes are often inaccessible.
The key innovation is the introduction of a stylus elbow which optimally uses the available joint ranges to facilitate the connection between two Omni devices. With the two Omni devices carefully arranged, the CombOmni fully utilizes the capabilities of the Omni devices to realize a 6-in-n-out haptic device. A few existing investigation fails to optimize for the modification and integration parameters and only generated subpar results.
The performances of the CombOmni were gauged via a comprehensive set of experiments. It has a rated workspace of 100 × 200 × 200 mm3 with a position sensing accuracy of 0.15 mm. The dexterous workspace is up to 10.78 sr. With the two Omni devices calibrated for their force outputs, the CombOmni generated a force output up to [4.1 N 5.2 N 5.8 N]T and a torque output up to [0 mNm 150 mNm 150 mNm]T in (it cannot generate torque about the stylus axis). It has stiffness of [2.22 N/mm 1.96 N/mm 4.26 N/mm]T. At the points on the boundaries of the rated workspace, the force and torque output capabilities deteriorate due to the limited performances of the Omni devices.
Compared to the current commercially available 6-in-n-out devices that are much more expensive, the CombOmni possesses similar performance characteristics. This would allow it a proper choice in many applications. Besides, the CombOmni was also successfully used in three application cases. With these demonstrations, the CombOmni can be a viable option for a haptic device with both force and torque outputs in a cost-effective way.
Future works mainly concern two aspects. Firstly, the CombOmni would be upgraded with better fabrication techniques of the new stylus and the robust assembly of the whole structure for higher reliability. Secondly, a miniature motor can be integrated in the stylus to generate the missing torque component for a 6-in-6-output haptic device.
This work was supported in part by National Key R&D Program of China (Grant No. 2017YFC0110800), and in part by the National Natural Science Foundation of China (Grant No. 51722507).
The authors would like to thank Shu'an Zhang for the help of setting up the teleoperation experiment.