Posted on

Learning Jaw Surgery with Virtual Reality

Learning Jaw Surgery with Virtual Reality

Jaw surgery can be complex and there are many factors that contribute to how a procedure is done. From routine corrective surgery to reconstructive surgery, the traditional means of teaching these scenarios has been unchanged for years. In an age populated with computers and the growing popularity of virtual reality, students still find themselves moving paper cut-outs of their patients around on a table top to explore different surgical methods.

Dr. Hera Kim-Berman was inspired to change this. Working with the Duderstadt Center’s 3D artist and programmers, a more immersive and comprehensive learning experience was achieved. Hera was able to provide the Duderstadt Center with patient Dicom data. These data sets were originally comprised of a series of two-dimensional MRI images, which were converted into 3D models and then segmented just as they would be during a surgical procedure. These were then joined to a model of the patient’s skin, allowing the movement of the various bones to influence real-time changes to a person’s facial structure, now visible from any angle.

This was done for several common practice scenarios (such as correcting an extreme over or under bite, or a jaw misalignment) and then imported into the Oculus Rift, where hand tracking controls were developed to allow students to “grab” the bones for adjusting in 3D.

Before re-positioning the jaw segments, the jaw has a shallow profile.
After re-positioning of the jaw segments, the jaw is more pronounced.

As a result, students are now able to gain a more thorough understanding of the spatial movement of bones and more complex scenarios, such as extensive reconstructive surgery, could be practiced well in advance of seeing a patient for a scheduled surgery.

Posted on

Steel Structures – Collaborative Learning with Oculus Rift

Steel Structures – Collaborative Learning with Oculus Rift

Civil & Environmental Engineering: Design of Metal Structures (CEE413) uses a cluster of Oculus Rift head-mounted displays to visualize buckling metal columns in virtual reality. The cluster is configured in the Duderstadt Center’s Jugular software so that the instructor leads a guided tour using a joystick while three students follow his navigation. This configuration allows the instructor to control movement around the virtual object while students are only able to look around.

Developed in a collaboration with the Visualization Studio, using the Duderstadt Center’s Jugular software this simulation can run on both an Oculus Rift or within the MIDEN.

Posted on

Art Students Model With Photogrammetry

Art Students Model With Photogrammetry

The Stamps School of Art and Design features a fabrication class called Bits and Atoms. This course is taught by Sophia Brueckner and it focuses on detailed and accurate modeling for 3D digital fabrication and manufacturing.

Sophia brought her students into the Duderstadt Center to use our new Photogrammetry rig. This rig features 3 cameras that take multiple photos of a subject placed on a rotating platform. Each photograph captures a different angle of the subject. When these photos are imported into a computer program, the result is a 3D model of the subject. The program tracks the movement of reference points in each photo in order to construct this model. This process is called photogrammetry.

The art students created digital models of themselves by sitting on the rotating platform. Their 3D models were then manipulated using Rhino and Meshmixer.

Posted on

Robert Alexander’s “Audification Explained” Featured on BBC World Service

Robert Alexander’s “Audification Explained” Featured on BBC World Service

Sonification is the conversion of data sets to audio files. Robert Alexander II is a Sonification Specialist working with NASA, who uses satellite recordings of the sun’s emissions to discover new solar phenomena. The Duderstadt Center worked with Robert to produce a short video explaining the concept of data audification.

Recently Robert was featured in a BBC World Service clip along with his video about making music from the sun: http://www.bbc.co.uk/programmes/p03crzsv

Posted on

Lia Min: RAW, April 7 – 8

Lia Min: RAW

Research fellow Lia Min will be exhibiting “RAW”  in the 3D lab’s MIDEN April 7 & 8th from 4 – 6pm. All are welcome to attend. Lia Min’s exhibit is an intersection of art and science, assembled through her training as a neuroscientist. Her data set, commonly referred to as a “Brainbow“,  focuses on a lobe of a fruit fly brain at the base of an antenna. This visualization scales microns to centimeters to enlarge the specimen with an overall visual volume of about 1.8 x 1.8 x 0.4 meters.

Posted on

An Application for Greek Transcription

An Application for Greek Transcription

Practice is the only way to learn a new language. However, when learning ancient languages, such as Greek, it can be difficult to get immediate, reliable feedback on practice work. This is why Professor Pablo Alvarez in Papyrology is working with Duderstadt Center student programmer Edward Wijaya to create an app for students to practice transcribing ancient Greek manuscripts into digital writing.

The app is divided into three modes: Professor/curator mode, student mode, and discovery mode. The professor mode allows the curator to upload a picture of the manuscript and post a line by line digital transcription of the document. These are the “answers” to the document. In student mode, these manuscript are transcribed by the students. When they click the check button, the student is given a line by line comparison to the curator’s answers. Furthermore, the discovery mode allows individuals with no Greek training to learn about the letters and read descriptions in the notations used.

A wide variety of fragile manuscripts which are often inaccessible to students are available on the app allowing the students to  gain experience with diverse handwriting and histories

Posted on

Surgical Planning for Dentistry: Digital Manipulation of the Jaw

Surgical Planning for Dentistry: Digital Manipulation of the Jaw

CT data was brought into Zbrush & Topogun to be segmented and re-topologized. Influence was then added to the skin mesh allowing it to deform as the bones were manipulated.

Hera Kim-Berman is a Clinical Assistant Professor with the University of Michigan School of Dentistry. She recently approached the Duderstadt Center with an idea that would allow surgeons to prototype jaw surgery specific to patient data extracted from CT scans. Hera’s concept involved the ability to digitally manipulate portions of the skull in virtual reality, just as surgeons would when physically working with a patient, allowing them to preview different scenarios and evaluate how effective a procedure might be prior to engaging in surgery.

Before re-positioning the jaw segments, the jaw has a shallow profile.

After providing the Duderstadt Center with CT scan data, Shawn O’Grady was able to extract 3D meshes of the patient’s skull and skin using Magics. From there, Stephanie O’Malley worked with the models to make them interactive and suitable for real-time platforms. This involved bringing the skull into a software like Zbrush and creating slices in the mesh to correspond to areas identified by Hera as places where the skull would potentially be segmented during surgery. The mesh was then also optimized to perform at a higher frame rate when incorporated into real-time platforms. The skin mesh was also altered, undergoing a process called “re-topologizing” which allowed it to be more smoothly deformed.  From there, the segmented pieces of the skull were re-assembled, and then assigned influence over areas of the skin in a process called “rigging”. This allowed for areas of the skin to move with selected bones as they were separated and shifted by a surgeon in 3D space.

After re-positioning of the jaw segments, the jaw is more pronounced.

Once a working model was achieved, it was passed off to Ted Hall and student programmer Zachary Kiekover, to be implemented into the Duderstadt Center’s Jugular Engine, allowing the demo to run at large scale and in stereoscopic 3D from within the virtual reality MIDEN but also on smaller head mounted displays like the Oculus Rift. Additionally, more intuitive user controls were added which allowed for easier selection of the various bones using a game controller or motion tracked hand gestures via the Leap Motion. This meant surgeons could not only view the procedure from all angles in stereoscopic 3D, but they could also physically grab the bones they wanted to manipulate and transpose them in 3D space.

Zachary demonstrates the ability to manipulate the model using the Leap Motion.

Posted on

Tour the Michigan Ion Beam Laboratory in 3D

Tour the Michigan Ion Beam Laboratory in 3D

3D Model of the Michigan Ion Beam Laboratory

The Michigan Ion Beam Laboratory (MIBL) was established in 1986 as part of the Department of Nuclear Engineering and Radiological Sciences in the College of Engineering. Located on the University of Michigan’s North Campus, the MIBL serves to provide unique and extensive facilities to support research and development. Recently, Professor Gary Was, Director of the MIBL reached out to the Duderstadt Center for assistance with developing content for the MIBL website to better introduce users to the capabilities of their lab as construction on a new particle accelerator reached completion.

Gary’s group was able to provide the Duderstadt Center with a scale model of the Ion Beam Laboratory generated in Inventor and a detailed synopsis of the various components and executable experiments. From there, the Stephanie O’Malley of the Duderstadt Center optimized and beautified the provided model, adding corresponding materials, labels and lighting. A series of fly-throughs, zoom-ins, and experiment animations were generated from this model that would serve to introduce visitors to the various capabilities of the lab.

These interactive animations were then integrated into the MIBL’s wordpress platform by student programmer, Yun-Tzu Chang. Visitors to the MIBL website are now able to compare the simplified digital replica of the space with actual photos of the equipment as well as run various experiments to better understand how each component functions.  To learn more about the Michigan Ion Beam Laboratory and to explore the space yourself, visit their website at  mibl.engin.umich.edu.

Posted on

Breaking Ground at Taubman College

Breaking Ground at Taubman College

The Taubman College of Architecture at the University of Michigan is adding another wing to the college.  Located on North Campus, the College of Architecture currently shares the 240,000 square foot space with the Penny Stamps School of Art and Design.  The architecture studios itself, located on the third floor of the building, occupy a space of around 30,000 square feet, making it the largest academic studio space in the world.

After a recent gift of $12.5 million made by Alfred Taubman, the college plans on building a new addition which will be called the A. Alfred Taubman wing.  The completed wing will be 36,000 square feet and will have new studios, new offices for faculty and new classrooms.

On April 25, 2015 the University of Michigan’s President Mark Schlissel along with Taubman College’s Dean Monica Ponce De Leon with donor A. Alfred Taubman present, broke ground at the site of the new wing.  The ceremonial shoveling was preformed by Taubman College’s Kuka Robot, a robot designed for architectural fabrication research but today was modified to assist in the ceremony.

The Duderstadt Center helped program the robot for the ceremony by filming a human shoveling in the lab.  The motion was captured in the lab by motion capture cameras and a program was developed for the robot to mimic the motion.

Integrated Design Solutions, a firm based in Troy, Michigan, along with architect Preston Scott Cohen are in charge of the design for the new college.  The building is scheduled to be completed in 2017.

Click here to read an article about the ceremony released on Taubman College’s website.

Posted on

User Story: Rachael Miller and Carlos Garcia

User Story: Rachael Miller and Carlos Garcia 

Rachael Miller and Carlos Garcia discuss how their individual experiences with the Digital Media Commons (DMC) shaped their projects and ambitions. Rachael, an undergraduate in computer science, was able to expand her horizons by working in the Duderstadt Center on projects which dealt with virtual reality. She gained vital knowledge about motion capture by working in the MIDEN with the Kinect, and continues to apply her new skills to projects and internships today.

Carolos Garcia worked to combine technology and art in the form of projection mapping for his senior thesis Out of the Box. To approach the project, he began by searching for resources and found DMC to be the perfect fit. By establishing connections to staff in the 3D Lab, Groundworks, the Video Studio and many others, he was able to complete his project and go on to teach others the process as well. For a more behind the scenes look at both Carlos Garcia and Racheael Miller’s projects and process, please watch the video above!