Imitating human soft tissue on basis of a dual-material 3D print using a support-filled metamaterial to provide bimanual haptic for a hand surgery training system
Three-dimensional (3D) printing is a rapid-prototyping and cost-effective method to generate real physical objects with high geometrical complexity (1-3). Accurately planned, computer-based 3D renderings are transmitted via polygon surface mesh object files for example in STL format (triangular mesh format for STereoLithography) or STEP format (STandard for Exchange of Product model data) to a 3D printer (4). Surface meshes consist of vertices, edges and faces to model the 3D surface shape of an object. The enclosed volume can be printed with a layer-by-layer fabrication method, which was originally developed in non-medical fields (3). Over the last decades 3D printing also gains attention in the health care sector (1-3). Studies showed that 3D-printed models provide a good anatomical representation, have significant potential for new medical applications and can provide a better understanding of the human anatomy (5). The surface models are based on additive manufacturing and are constructed from a segmented computer tomography (CT) and/or magnetic resonance imaging (MRI) scan (1,2). Current medical application fields are for example implant generation, diagnostics, orthopedics, spinal surgery, cardiac surgery, etc. (3,5). These anatomical models are mostly only geometrically accurate and do not imitate mechanical characteristics like soft tissue. In haptic assisted training systems, a realistic haptic feeling is crucial. Thus, tissue-imitating phantoms are especially developed for medical education purpose (1).
Kirschner-Wires (K-wires, Figure 1A) are long, sharp, stainless steel pins, which are drilled minimally invasive through a small stab incision into an injured bone to stabilize fractures of the human hand (6,7). To ensure trouble-free healing, K-wires are removed within a time period of around 4 to 6 weeks. During this sophisticated operation, surgeons can only check the K-wire position with an X-ray device (6). Because of the restricted view during the drilling process it is hard to find the best position and bore angle to avoid sensitive nerves or vessels. Surgeons rely on their anatomical knowledge to pinpoint bones and risk structures by palpating specific bony protrusions on the human hand (8). Hence, haptic feeling is essential for a successful operation.
Currently, surgical training is restricted to drilling on synthetical or defrosted cadaver bones and surgical guidance from an expert surgeon in the operating room (9,10). While the first case can lead to ethical problems and to limited reality (11), additional training in the operating room is very time consuming for the expert surgeon and can endanger the patient (9,10). Virtual reality (VR) based training systems are more and more attracting attention for medical education and are already used in some training simulators for laparoscopic, endoscopic or dental cases (11-13). Computer-based training does not only train the hand-eye coordination of the surgeon (14), it can be used for surgical planning or debriefings, but it also gives young surgeons additional training possibility to improve their surgical skills (3,5). Especially hand-eye coordination is important in the operating room, because surgeons need to process multiple impressions simultaneously. Several studies showed (11,12,15), that training on medical simulators increases surgical performance of surgeons in the operating room, decreasing the number of errors and reduces the duration of the surgery. Within the HaptiVisT project we developed a VR-based and visual assisted training system for minimally invasive hand surgery (16,17). For this, a driller is mounted at the tip of a haptic device (Virtuose 6D Desktop, Haption GmbH, Aachen, Germany, www.haption.com) with 6 degrees of freedom to firstly track the position in a 3D space and secondly to transmit real-time computed force components from the computer to the haptic device. Thus, virtual objects become tactile for the user. In contrast to the drill holder, the K-wire is placed within the drill chuck only in the visualization. An autostereoscopic 3D monitor (SeeFront GmbH, Hamburg, Germany, www.seefront.com) uses an eye-tracking bar and thus no glasses are required for a depth effect. Figure 1B provides the basic setup of our system. As an additional haptic component, a 3D-printed haptic hand phantom will be arranged between monitor and user. A marker is mounted at the rear section of the haptic hand phantom (Figure 1C,D) to track the 3D orientation by a real-time tracking camera (MicronTracker, ClaroNav, Toronto, Canada, www.claronav.com). This allows an intuitive adjustment of the hand visualization on the 3D monitor in rotation and position. By combining the haptic device with the haptic hand phantom, the bimanual haptic scenario of the operating room is transferred to the training system.
The haptic of the human hand during a minimally invasive surgery is essential. Therefore, in a training system this haptic component should be as realistic as possible including skin, soft tissue and bones. To ensure this, the following requirements have to be met:
- Hand bones should be printed with hard material and be palpable through a rubber-like skin and soft tissue;
- Under pressure the elasticity behavior of the 3D print must be similar to human soft tissue;
- The hand shape must not alter after repeated dispersing.
The flexion and extension of the fingers are prohibited, because the training system currently does not support the bending of bones, what would lead to system calibration problems.
This paper introduces our approach to imitate bony and soft tissue of the human hand by using dual-material restricted 3D printing and metamaterials. The design and haptic properties of the sample prints are specified, discussed in the results section and evaluated by an expert surgeon case study.
Equipment and print material
For creating test-samples we used the Connex3 3D printer system with build tray Connex500® from Stratasys Ltd. (Minnesota, USA, www.stratasys.com). With its PolyJet technology it is possible to load up to three different materials to print from rigid plastic to rubber-like multi-materials (18). It is possible to blend both materials in a certain ratio into each other to generate material properties with a different color or elasticity (1,5,19). Two of the three loaded materials can be chosen individually, but the third one always has to be support material (SUP). During the layer-by-layer 3D construction, overhanging features of complex objects must be supported (5), otherwise they would collapse. Empty space within the 3D structure is also automatically padded with SUP, because the 3D printer cannot print into air. After 3D printing, SUP is usually removed. Depending on the composition of the SUP, it can be removed by hand, by heating, or by flushing using water or solvent (5). With the Connex500, however, washable SUPs cannot be selected as they are not supported. For printing a hand phantom, one hard material is needed to print the bones and one rubber-like material to imitate skin and soft tissue. For each material we experimented with a clear and an opaque variant from Stratasys, respectively. TangoPlus (TP, FLX930) and TangoBlackPlus (TBP, FLX980) were used as most elastic available rubber-like material with a shore hardness from 18-22 scale A. VeroClear (VC, RGD810) and VeroBlackPlus (VBP, RGD875) were used as the stiffest printable materials from Stratasys with 83-86 scale D (20). These materials were also suggested by Wang et al. for imitating tissue phantoms (1). As SUP we used the polymerized Object SUP705 from Stratasys as well. The SUP is always mixed with a harder material as additional support. The mixing ratio can be chosen between light, medium and heavy. Because of the skin coverage, the SUP is left inside the hand phantom.
Although TP is the most elastic material, it is still too firm to imitate soft tissue according to its material properties. Thus, we developed a metamaterial design to imitate the soft tissue behavior. Metamaterial was first introduced in electromagnetic or physics applications and are novel or artificial engineered structures to obtain unusual electromagnetic properties (21,22). This approach was recently extended to other material properties of a body to also obtain a novel structure in terms of mechanics (22). It shows, that a body behavior does not depend only on the material it is made of but also on the structure it is composed of (19,22). Metamaterial design in 3D prints is achieved by assemblies of small-scaled structures (23). Both, the scales of the structure interfaces and the composed material are defining the mechanical properties of a metamaterial (22). In our new metamaterial-based approach, we used the porous structure of SUP to soften TP material. Our SUP is a blowy material and decomposes under pressure. The space between skin and bone is filled with SUP but pillowed by a rubbery TP or TBP lattice structure to create support netting. This allows us to generate a variable material softness through structure thickness, while the lattice structure ensures, that the hand phantom does not lose its original shape.
To create 3D lattice models in metamaterial design we used the engineering software nTopology’s Element Pro V1.22.0 (nTopology, New York, USA, www.ntopology.com) made for advanced manufacturing. Firstly, based on the origin coordinate system a wireframe is created, which consists of repeated arrangement of unit cells. The tessellation can be customized as cube, hexagonal prism, tetrahedron, octahedron, etc. A wireframe can be trimmed to surface meshes to spare out inner bone structure, for example. To create a 3D lattice model for a 3D print, the wireframe is thickened with a specific diameter and then meshed. Based on wireframes we generate a metamaterial structure using predefined rules from the Element software. For our sample prints we chose rules in simple cube edge (CE) and self-designed cube face (CF) design. The CE design consists of a simple cube-shaped unit cell with unclosed sides, whereas the sides are closed within CF. The cube design allows us to pack as much SUP as possible into the interior volume. Figure 2 shows a wireframe preview composed of both unit cell apposition designs CE and CF. First, samples were printed in a cuboid shape without internal bones to test only the material behavior and flexibility. Figure 3 shows the construction of the cuboid outer case and of the lattice structure in CE and CF design.
As an initial point for printing a hand phantom we use manually segmented 3D surfaces of the skin and the bones, which are obtained from a male patients CT and MRI scan. Skin and bone models are available as separate surface meshes. For our print samples we use only the middle finger as representative part of the hand. Our overall lattice construction workflow with pre-processing, lattice generation and post-processing is illustrated in Figure 4. The resulting individual production step in the Element software can be seen in Figure 5. Table 1 includes all production parameters. The workflow is structured as follows. Primarily the skin surface mesh of the finger (Figures 4A,5A) is shrunk by a negative offset (offsetskin) to the inside direction (Figures 4B,5B). After turning over the face normals we get a thickened skin model (thickskin) with an outside and inside wall. If desired, the surface meshes of middle, distal and intermediate phalanges (Figure 4C) are expanded outwards (offsetbone) with a positive offset (Figure 4D). This results in a thin layer that surrounds the bones at a fixed distance (thickbone) like a periosteal layer (Figure 5C,D). After offset appliance, we remesh shrunken or expanded surface meshes of the skin and periosteal layer to get a regular surface mesh with equally shaped edges (Figure 4E,F). This allows a performance increase during the trimming process. In the next step the wireframe is generated (Figures 4G,5E). The size of the unit cells (rule size) must be chosen carefully, as it will ultimately affect the entire haptic of the print-sample. The wireframe is trimmed firstly to the inner wall of the skin (Figures 4H,5F) and then, if desired, secondly to the generated periosteal layer (Figures 4I,5G). After that, the wireframe perfectly fills the cavity between periosteal layer and skin. The wireframe is thickened (thicklatt) with a well-defined diameter (Figures 4J,5H,5I) and afterwards meshed to get a surface lattice as result (Figures 4K,5J). A final surface mesh simplification reduces the file size by reducing the number of vertices (Figure 4L). The negative and positive offsets for shrinking the skin (offsetskin) and expanding the bones (offsetbone) led to a wireframe, that extends half into the skin as well as the bone or periosteal layer (if desired) after thickening. If the lattice should extend directly into the bone, a negative offset (offsetbone) must be applied. Since the bending of the bones should not be possible, two cylinders are placed between bones. These connect and stiffen the bones, as they protrude into the interior of both bones and are filled with the same hard material as the bones. The skin, the periosteal layer and the lattice structure (Figure 4M) are then existent in separate STL files and ready to print.
Results (Figures 6-8)
Table 1 lists all relevant cuboid and finger shaped test prints with material, thickness diameter, unit cell design, offset size and lattice dimensions. Cuboid formed test prints always have outer dimensions of 30 mm × 30 mm × 50 mm with a constant wall thickness of 2 mm regardless of the unit cell type used. The lattice always extends (CE and CF) 1 mm into the skin wall. As reference information for cuboids with lattice structure, both a cuboid filled with neat rubber material (C01) and a cuboid filled completely with SUP (C07) are printed. Three cuboids in CE and four cuboids in CF design were designed and printed with different unit cell dimensions and thickening diameter. The CE cuboids have a rule size of 5 mm (C06) and 7 mm (C05, C11). The side length of the volume filled with SUP within a unit cell is calculated by rule size minus thickening (rule size corrected). A cuboid with rule size of 7 mm and integrated bone (C11) was printed to test the bony haptic. The four CF designed cuboids differ only by dimensions of the internal lattice. The rule size varies from 5 mm (C12) to 1.5 mm (C15) and the lattice thickening from 2 mm (C12) to 0.5 mm (C14, C15). Fingers are prepared only in CF design. In addition to the thickening diameter and the unit cell dimension we tested samples with and without rubber-like material filled periosteal layer, with or without lattice intrusion and various skin thicknesses (F04–F18). The inner lattice can extend into the bone, periosteal layer and skin, depending on the thickening. For testing the rigid bone connection with two cylinders, a sample with only two bones of the middle finger was printed (B01). During the whole printing procedure, we decided to only use SUP with light mixing. The average ratio of TP or TBP in SUP is about 0.5%. Middle and hard mixing ratios would complicate the disaggregation of SUP within the lattice structure.
The cuboid shaped sample print consisting of pure TP (C01) material shows that the most elastic available material is effectively too hard to imitate soft tissue anyway (Figure 6C). A cuboid filled completely with SUP and a TP skin wall of 2 mm (C07) results in a softer behavior as the SUP is completely decomposed after repeated dispersing. However, the SUP can be shifted from one end of the volume to the other and vice versa. Figure 6A shows cuboid C07, that can be deformed without retrieving its original shape.
Sample prints C05 and C06 show great elasticity and are initially dimensionally stable. Inside bones are held in place as well (C11). However, after several compressions, SUP decomposes and can be freely moved inside the volume, since the unit cell sides are open. The lattice structure could not fully keep up the original cuboid shape (Figure 6B) and begins to break, so that bones can be freely moved inside the volume.
Thanks to the CF design, the SUP is held in place inside every unit cell and cannot be moved through the volume. Even after repeated kneading, the fabric acquires its original shape due to the lattice structure and shows great elasticity (Figure 6D,E,F). While reducing the rubber-like material and increasing the amount of SUP by choosing smaller thickening diameters (C12–C14), the softness of the cuboid grows steadily. A lattice design with rule size of 1.5 mm by 0.5 mm (C15) thickening shows a great elasticity as well and the SUP crumbles only partial after great effort. The best haptic impression is given from cuboids C13 and C15.
Few 3D-printed fingers with various lattice structures (F14–F18) are shown in Figure 7A. According to our requirement analysis the bone connectors (B01, Figure 7B) achieve a stiffening of the bones. Thus, no bending of the bones or breaking of the TP material is possible. A lattice intrusion ensures that the lattice structure is firmly attached to the hand and bones and do not get anchorless after dispersing. However, this intrusion resulted in an overlap between hard bone and soft material (Figure 7C). This mixture softens up the hard surface of the bone, so that the bones get hard to palpate. A surrounding periosteal layer not only provides better anchoring between bones and lattice. The bones are also kept better in their place and can be felt easily. However, the additional periosteal layer reduces the amount of SUP. Thus, we reduced the skin thickness to 1.5 mm. Test prints with thicker lattice walls and smaller unit cells contain less SUP (Table 1) and appear firmer and vice versa.
Expert surgeon evaluation
For an expert interview six cuboids (C01, C06, C12–C15) and five fingers (F14–F18) were preselected with previously experimentally determined parameters. The fingers only differ in CF rule size to find the appropriate one for imitating realistic soft tissue. Surgeons were asked about their opinion on the sample prints compared to the haptic, elasticity and softness behavior of real human tissue. In addition to the cuboids, the sensitivity of the bones and the relocatability of the skin were queried on the fingers. Eleven interviews were included in our analysis, six subjects were male (55.5%) and five female (45.5%). The average age was 32.4, with the youngest participant 27 and the oldest 38. On average, the participants have practical experience of 4.9 years in trauma surgery, hand surgery or other medical fields. Nine out of 11 surgeons had no experience on 3D-printed phantoms previously. Figure 8 shows a bar chart with the evaluation results separated into positive (mean of very positive, positive and rather positive) and negative (mean of very negative, negative and rather negative) feedback. The subjects confirmed, that pure TP material (C01) and cuboids in CE design (C06) cannot imitate realistic soft tissue due to bad haptic (C01: 63.6%, C06: 72.7%), elasticity (C01: 54.5%, C06: 81.8%) and softness (C01: 81.8%, C06: 72.7%). However, C13 and C15 in CF design convinced the subjects with their good haptic experience (C13: 81.8%, C15: 81.8%), their elasticity (C13: 90.9%, C15: 90.9%) and their softness (C13: 81.8%, C15: 72.7%). Finger samples F14, F17 and F18 received positive feedback in each category queried. The haptic showed best behavior at F17 with 90.9%, the bone sensitivity at F14 and F18 with 72.7%, the elasticity at F14 and F17 with 81.8%, the softness at F18 with 72.7% and the relocatability at F17 and F18 with 63.6%.
In this work we presented a new method to imitate soft tissue of the human hand by using only a dual-material 3D printer and metamaterial design filled with blowy SUP. Especially the CF design convinced with its elasticity and fabric softness and met our desired requirements. The CE design showed deficiencies in maintaining the original object shape and, therefore, is not suitable for imitating tissue. Our imitation of human soft tissue can be transferred to different body parts and thus offers great potential for future training applications that require a realistic haptic for effective surgical training. In addition to training simulators, 3D-printed realistic haptic models can be used in surgical planning, teaching, diagnostics or orthopedics.
Since our used unit cells in cubic shape are not rotationally symmetric and depend on the original located coordinate system, further lattice constructions must be tested in the future. For example, a unit cell rule in sphere design can provide a rotation-independent property but compared to a cube its SUP ratio and, hence, its softness is reduced. Since natural structures are usually highly mechanically anisotropic (22), phantoms with anisotropic design could extend the haptic properties of the hand in future. Further, segmentation not only between bones and skin, but also between fat and muscle tissue could affect the tissue haptic. An additional layer of pure SUP around the bone may improve the relocatability of soft tissue. The use of a 3D printer capable of loading more than two materials would allow the use of multiple (hard and soft) rubber-like materials, which can be blended together to create lattice structures of different hardness. In our future work we will print a whole hand phantom in CF design to complete the HaptiVisT training system. Therefore, the entire hand needs to be cut into several parts to not exceed the maximum permissible STL file size of 400 MB, specified by the Connex500. The tracking marker is then inserted into a provided cavity at the rear tip of the hand phantom. If in the future movement of phalanges should be made possible in the simulation, the bone connectors can be filled with soft material instead of hard material to imitate ligaments and tendons.
The authors want to thank Florian Olbrich and Thomas Mühlenfeld from the Sensor Technology Application Center (SappZ, Regensburg, Germany, www.sappz.de) for their technical support, for providing the 3D printer Connex500 and printing the samples. We thank Martina Simon from Fraunhofer Center for Applied Research on Supply Chain Services (SCS) for her help designing the expert surgeon evaluation. Thanks to the Trauma Surgery & Plastic, Reconstructive and Hand Surgery Department from the University Hospital Regensburg for their support in performing an expert evaluation.
Funding: This work was supported by the German Federal Ministry of Education and Research with grant 16SV7560 and by the Bavarian Academic Forum (BayWISS) – Doctoral Consortium “Health Research” (funded by the Bavarian State Ministry of Science and the Arts).
Conflicts of Interest: The authors have no conflicts of interest to declare.
- Wang K, Wu C, Qian Z, Zhang C, Wang B, Vannan MA. Dual-material 3d printed metamaterials with tunable mechanical properties for patient-specific tissue-mimicking phantoms. Addit Manuf 2016;12:31-7. [Crossref]
- Pati F, Ha DH, Jang J, Han HH, Rhie JW, Cho DW. Biomimetic 3d tissue printing for soft tissue regeneration. Biomaterials 2015;62:164-75. [Crossref] [PubMed]
- Tack P, Victor J, Gemmel P, Annemans L. 3d-printing techniques in a medical setting: a systematic literature review. Biomed Eng Online 2016;15:115. [Crossref] [PubMed]
- Bernard A, Fischer A. New trends in rapid product development. CIRP Annals 2002;51:635-52. [Crossref]
- Rengier F, Mehndiratta A, von Tengg-Kobligk H, Zechmann CM, Unterhinninghofen R, Kauczor HU, Giesel FL. 3d printing based on imaging data: review of medical applications. Int J Comput Assist Radiol Surg 2010;5:335-41. [Crossref] [PubMed]
- Boeckstyns MEH, Richter M. Fractures of the Hand and Carpus: FESSH 2018 Instructional Course Book. Stuttgart, Germany: Thieme, 2018.
- Franssen BB, Schuurman AH, Van der Molen AM, Kon M. One century of kirschner wires and kirschner wire insertion techniques: a historical review. Acta Orthop Belg 2010;76:1-6. [PubMed]
- Haughton D, Jordan D, Malahias M, Hindocha S, Khan W. Principles of hand fracture management. Open Orthop J 2012;6:43-53. [Crossref] [PubMed]
- Gould DA, Chalmers N, Johnson SJ, Kilkenny C, White MD, Bech B, Lonn L, Bello F. Simulation: moving from technology challenge to human factors success. Cardiovasc Intervent Radiol 2012;35:445-53. [Crossref] [PubMed]
- Tsai MD, Hsieh MS, Tsai CH. Bone drilling haptic interaction for orthopedic surgical simulator. Comput Biol Med 2007;37:1709-18. [Crossref] [PubMed]
- Badash I, Burtt K, Solorzano CA, Carey JN. Innovations in surgery simulation: a review of past, current and future techniques. Ann Transl Med 2016;4:453. [Crossref] [PubMed]
- McCloy R, Stone R. Virtual reality in surgery. BMJ 2001;323:912-5. [Crossref] [PubMed]
- Wu J, Yu G, Wang D, Zhang Y, Wang CC. Voxel-based interactive haptic simulation of dental drilling. 29th Computers and Information in Engineering Conference, Parts A and B. 2009;2:39-48.
- Griﬃth JL, Voloschin P, Gibb GD, Bailey JR. Diﬀerences in eye-hand motor coordination of video-game users and non-users. Percept Mot Skills 1983;57:155-8. [Crossref] [PubMed]
- Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458-63; discussion 463-4. [Crossref] [PubMed]
- Maier J, Haug S, Wang D, Huber M, Katzky U, Neumann S, Perret J, Prinzen M, Scorna U, Weber K, Wittenberg T, Wöhl R, Palm C. Development of a haptic and visual assisted training simulation concept for complex bone drilling in minimally invasive hand surgery. Proceedings of the 31th International Congress and Exhibition of Computer Assisted Radiology and Surgery (CARS). 2017:135-6.
- Maier J, Huber M, Katzky U, Perret J, Wittenberg T, Palm C. Force-feedback-assisted bone drilling simulation based on CT data. In: Maier A, Deserno T, Handels H, Maier-Hein K, Palm C, Tolxdorff T. editors. Bildverarbeitung für die Medizin 2018. Informatik aktuell. Berlin, Heidelberg: Springer Vieweg, 2018:291-6.
- Stratasys. Connex3: the versatility standard in 3D printing. Accessed 07 September 2018. Available online: http://www.stratasys.com/-/media/files/white-papers-new/wp_pj_connex3_0517a-web.pdf
- Wang K, Chang YH, Chen Y, Zhang C, Wang B. Designable dual-material auxetic metamaterials using three-dimensional printing. Mater Des 2015;67:159-64. [Crossref]
- Stratasys. PolyJet materials data sheet. Accessed 07 September 2018. Available online: http://usglobalimages.stratasys.com/Main/Files/Material_Spec_Sheets/MSS_PJ_PJMaterialsDataSheet.pdf?v=635785205440671440
- Dong Y, Itoh T. Metamaterial-based antennas. Proceedings of the IEEE. 2012;100:2271-85. [Crossref]
- Lee JH, Singer JP, Thomas EL. Micro-/nanostructured mechanical metamaterials. Adv Mater 2012;24:4782-810. [Crossref] [PubMed]
- Schumacher C, Bickel B, Rys J, Marschner S, Daraio C, Gross M. Microstructures to control elasticity in 3d printing. ACM Trans Graph 2015;34:1-13. [Crossref]