Developed by a Microsoft Research team, KinÊtre is a 3D graphic animation technology that allows even novice users to animate any object with their body movements. Rather than using expensive and sophisticated professional software, KinÊtre uses the Kinect motion sensor and its Windows-based software. Microsoft Research answered Futura-Sciences' questions.
Slipping into the skin of an inanimate 3D object to bring it to life with the movements of one's own body, this is a performance that was until now reserved for graphic animation specialists equipped with powerful computers and software. It may soon be accessible to the general public, including children.
To make a chair or a table work, a cuddly toy, to play football with a cupboard, to bring toys to life, here are some of the things you can do KinÊtre. This project, the fruit of a team from Microsoft Research in Cambridge, England, is based on the use of the Kinect sensor for Windows, which associates the virtual skeleton of a person with the geometry of a 3D digitized object. As Rachel Howard, spokesperson for Microsoft Research, explained to Futura-Sciences, the KinÊtre system consists of two phases.
To begin, the user will scan a physical object by walking around it using the Kinect sensor for Windows. The creation of this 3D model is based on another technology developed by Microsoft, named KinectFusion, which will separate the digitized object from its environment to make it autonomous. Then, a specially developed algorithm will transform the 3D mesh into a deformable structure that will react to body movements.
Once the first phase has been completed, the user stands in the space where the object he has digitised was, in front of the Kinect sensor connected to a television set. The sensor scans the person and produces a virtual skeleton that appears on the screen behind the 3D object. He then has to move to adjust the alignment between his virtual skeleton and the object to create points of contact between the two meshes. When the alignment is correct, the user says the English word "possess" to create the animation. He can then bring the 3D object to life by moving his body. "We use a numerical optimization algorithm to make the deformation processing as smooth as possible and reduce stretching," says Howard.
Why not enjoy unlimited reading of UP'? Subscribe from €1.90 per week.
In the first part of this video presenting KinÊtre, a chair performs contortions by following the movements of a person that we see incrusted. The red and blue parts correspond to the members of the virtual skeleton that has merged with the 3D chair. In detail, the operator starts by scanning the real chair with the Kinect sensor, then scans himself and comes to adjust his virtual skeleton on the 3D chair before pronouncing the word "possess" to "own" the chair by creating contact points with the 3D mesh. He can then incarnate the object and move it as he sees fit. © Microsoft Research/YouTube
With KinÊtre, 3D animation accessible to everyone
KinÊtre can therefore give life to any object in everyday life. This is enough to unleash the imagination of creators of animated films, cartoons, video games or simply of amateurs who will have access to a technology that has until now been complex and expensive. "We believe that KINÊtre can be used in a wide range of fields. Our initial goal was to make animation accessible to everyone.
We want children to use our system to tell their own stories. We also believe that KINÊtre can give birth to a new category of games that incorporate your personal objects. And there are still other applications for this technology in areas such as teleconferencing and virtual environments where studies have shown the benefits of avatars for collaboration," continues the Microsoft Research spokesperson.
KinÊtre, still at the prototype stage
In addition to its simplicity of implementation, the interest of KinÊtre is that it does not require powerful hardware to operate. "KinÊtre can run on a standard desktop PC or a relatively powerful laptop. Real-time 3D scanning with KinectFusion requires a recent graphics processor such as a NVidia GeForce GTX 580," says Rachel Howard.
This innovation was presented last week at the 39th Siggraph exhibition held in Los Angeles. However, Microsoft was very vague about its plans for KinÊtre and its eventual market introduction. "KinÊtre is still a research prototype that we continue to test. At this stage, we have no information to communicate about its availability," we were told.