Researchers at Columbia Engineering announced today that they have introduced a new type of robotic finger with a sense of touch. Their finger can localize touch with very high precision—<1mm—over a large, multicurved surface, much like its human counterpart.
“There has long been a gap between stand-alone tactile sensors and fully integrated tactile fingers–tactile sensing is still far from ubiquitous in robotic manipulation,” says Matei Ciocarlie, associate professor in the departments of mechanical engineering and computer science, who led this work in collaboration with Electrical Engineering Professor Ioannis (John) Kymissis. “In this paper, we have demonstrated a multicurved robotic finger with accurate touch localization and normal force detection over complex 3D surfaces.”
Current methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or difficulty fitting into small fingertips, thus preventing use in dexterous hands. The Columbia Engineering team took a new approach: the novel use of overlapping signals from light emitters and receivers embedded in a transparent waveguide layer that covers the functional areas of the finger.
By measuring light transport between every emitter and receiver, they showed that they can obtain a very rich signal data set that changes in response to deformation of the finger due to touch. They then demonstrated that purely data-driven deep learning methods can extract useful information from the data, including contact location and applied normal force, without the need for analytical models. Their final result is a fully integrated, sensorised robot finger, with a low wire count, built using accessible manufacturing methods and designed for easy integration into dexterous hands.
The study, published online in IEEE/ASME Transactions on Mechatronics, demonstrates the two aspects of the underlying technology that combine to enable the new results. Firstly, in this project, the researchers use light to sense touch. Under the “skin,” their finger has a layer made of transparent silicone, into which they shined light from more than 30 LEDs. The finger also has more than 30 photodiodes that measure how the light bounces around. Whenever the finger touches something, its skin deforms, so light shifts around in the transparent layer underneath. Measuring how much light goes from every LED to every diode, the researchers end up with close to 1,000 signals that each contains some information about the contact that was made. Since light can also bounce around in a curved space, these signals can cover a complex 3D shape such as a fingertip.
“The human finger provides incredibly rich contact information–more than 400 tiny touch sensors in every square centimetre of skin,” says Ciocarlie. “That was the model that pushed us to try and get as much data as possible from our finger. It was critical to be sure all contacts on all sides of the finger were covered–we essentially built a tactile robot finger with no blind spots.”
Secondly, the team designed this data to be processed by machine learning algorithms. Because there are so many signals, all of them partially overlapping with each other, the data is too complex to be interpreted by humans. Fortunately, current machine learning techniques can learn to extract the information that researchers care about: where the finger is being touched, what it is touching the finger, how much force is being applied, etc.
“Our results show that a deep neural network can extract this information with very high accuracy,” says Kymissis. “Our device is truly a tactile finger designed from the very beginning to be used in conjunction with AI algorithms.”
In addition, the team built the finger so it, and others, can be put onto robotic hands. Integrating the system onto a hand is easy: thanks to this new technology, the finger collects almost 1,000 signals, but only needs a 14-wire cable connecting it to the hand, and it needs no complex off-board electronics. The researchers already have two dexterous hands (capable of grasping and manipulating objects) in their lab being outfitted with these fingers–one hand has three fingers, and the other one four. In the next months, the team will be using these hands to try and demonstrate dexterous manipulation abilities, based on tactile and proprioceptive data.
“Dexterous robotic manipulation is needed now in fields such as manufacturing and logistics, and is one of the technologies that, in the longer term, are needed to enable personal robotic assistance in other areas, such as healthcare or service domains,” Ciocarlie says.
New measures as video production comes to a halt
MultiChoice has announced a range of measures to safeguard the incomes of cast, crew, and creatives, as well as the sustainability of production houses.
MutliChoice Group has confirmed that production of video entertainment has come to a complete halt in South Africa, as the industry adheres to the national lockdown.
Many sectors across the African continent are adjusting and finding ways to deal with challenges brought about by the current coronavirus pandemic. Among these, MultiChoice Group is one of the best-known brands across Africa, due to its dominance of the video entertainment sector. As a result, it says, it is acutely aware of the challenges its partners in the industry are facing.
“For many years MultiChoice has been a leader in the sector and are a leading investor in local content production,” it said in a statement on Friday. “The group has deep rooted partnerships and long-standing commitments that have grown the industry for the benefit of many, both in front and behind the camera.”
For this reason, says MultiChoice, it has decided to implement several measures aimed at safeguarding the incomes of cast, crew, and creatives as well as the sustainability of production houses.
“With these measures the group can hopefully steer the industry through this tumultuous time.”
MultiChoice announced the following measures:
- “MultiChoice has set aside R80 million to ensure that current productions are able to pay full salaries of cast, crew, and creatives for the months of March and April, by when hopefully we will have the worst of the disruptions behind us. The need to secure salaries of our creatives goes a long way in creating income stability for them and their families. We believe this to be critical for the industry and in our view simply the right thing to do.
- “Through the MultiChoice Talent Factory we will be launching an online learning portal that will support over 40 000 members of the industry to gain access to courses and online master classes, so they can continue to hone their craft whilst adhering to the public health measures of social distancing and isolation.
- “Furthermore, we have committed to guarantee the incomes of freelancers in our SuperSport Productions, who are currently unable to work due to the suspension of sport and the national lockdown. This extends to guaranteeing the income of freelancers in our broadcast technology environment.”
MultiChoice Group CEO Calvo Mawela said: “Our main concern is to ensure as much as possible that we secure the incomes of creatives, cast and crew over this period. We want to ensure that they and their families are not negatively impacted as work has come to a standstill.”
Nintendo South Africa launches online gameathon
Nintendo’s South African distributor has set up live events for virtual participation during lockdown.
Being stuck inside doesn’t mean not being with the game. Players now have the chance to meet up virtually to play Nintendo Switch Online games like Ring Fit Adventure, Animal Crossing: New Horizons, and Pokémon Sword and Shield.
A daily schedule is available for each game. For example, Ring Fit Adventure has a kids challenge at 9am from Monday to Friday, while adults can join in at 2pm. The challenges are co-ordinated by a dedicated Ring Fit coach.
The challenge instructions will be available via the Discord channel. Join the channel via the invite here.
Players of all skill levels are encouraged to join. Plenty tips and tricks will be shared about the games.