Category Archives: News

How UConn Researchers are Teaching Robots to Think Like Humans

There’s a great scene in the movie “Iron Man” where Robert Downey Jr.’s character Tony Stark (aka Iron Man) is crawling across his lab, desperately trying to reach the small arc reactor he needs to keep his heart beating and stay alive.

Weakened by a run-in with arch villain Obadiah Stane, Stark can’t reach the gizmo where it sits on a tabletop. Defeated, he rolls onto his back, exhausted and pondering his inevitable doom.

But the very moment that we think our intrepid hero’s a goner, a metallic hand appears at Stark’s shoulder, holding the lifesaving device. “Good boy,” Stark says weakly as he takes the device from his robot assistant, Dum-E.

And just like that, our hero is saved.

From the dutiful shuffling of C-3PO to the terrorizing menace of The Terminator, Hollywood has made millions tantalizing audiences with far-out robot technology. Scenes like the one in “Iron Man” make for good entertainment, but they also are based, to some degree, in reality.

Dum-E’s interaction with Stark is called collaborative robotics, where robots with advanced artificial intelligence, or A.I., not only work alongside us humans but also are able to anticipate our actions and even grasp what we need.

Collaborative robotics represents the frontier of robotics and A.I. research today. And it’s happening at UConn.

Three thousand miles away from the klieg lights of Hollywood, Ashwin Dani, director of UConn’s Robotics and Controls Lab, or RCL, stands in the stark fluorescent light of his Storrs office staring at a whiteboard covered in hastily scrawled diagrams and mathematical equations.

Here, in the seemingly unintelligible mishmash of numbers and figures, are the underlying mathematical processes that are the lifeblood of collaborative robotics.

If robots are going to interact safely and appropriately with humans in homes and factories across the country, they need to learn how to adapt to the constantly changing world around them, says Dani, a member of UConn’s electrical and computer engineering faculty.

“We’re trying to move toward human intelligence. We’re still far from where we want to be, but we’re definitely making robots smarter,” he explains.

All of the subconscious observations and moves we humans take for granted when we interact with others and travel through the world have to be taught to a robotic machine.

When you think about it, simply getting a robot to pick up a cup of water (without crushing it) and move it to another location (without spilling its contents or knocking things over) is an extraordinarily complex task. It requires visual acuity, a knowledge of physics, fine motor skills, and a basic understanding of what a cup looks like and how it is used.

“We’re teaching robots concepts about very specific situations,” says Harish Ravichandar, the senior Ph.D. student in Dani’s lab and a specialist in human-robot collaboration. “Say you’re teaching a robot to move a cup. Moving it once is easy. But what if the cup is shifted, say, 12 inches to the left? If you ask the robot to pick up the cup and the robot simply repeats its initial movement, the cup is no longer there.”

Repetitive programs that work so well for assembly-line robots are old school. A collaborative robot has to be able to constantly process new information coming in through its sensors and quickly determine what it needs to do to safely and efficiently complete a task. If that robot is part of an assembly line, the line has to shut down and the robot has to be reprogrammed to account for the change, an inefficient process that costs manufacturers money. Hence the thinking robot this team is trying to create.

While the internet is filled with mesmerizing videos of robots doing backflips, jumping over obstacles, and even making paper airplanes, the UConn team’s effort at controlling robots through advanced artificial intelligence is far less flashy but potentially far more important.

Every move the UConn team wants its test robot to make starts here, says Dani, with control theory, engineering, whiteboards, and math.

“We’re writing algorithms and applying different aspects of control theory to take robot intelligence to a higher level,” says Ravichandar. “Rather than programming the robot to make one single movement, we are teaching the robot that it has an objective — reaching for and grabbing the cup. If we succeed, the robot should be able to make whatever movements are necessary to complete that task no matter where the cup is. When it can do that, now the robot has learned the task of picking something up and moving it somewhere else. That’s a very big step.”

While most of us are familiar with the robots of science fiction, actual robots have existed for centuries. Leonardo da Vinci wowed friends at a Milan pageant in 1495 when he unveiled a robotic knight that could sit, stand, lift its visor, and move its arms. It was a marvel of advanced engineering, using an elaborate pulley and cable system and a controller in its chest to manipulate and power its movements.

But it wasn’t until Connecticut’s own Joseph Engelberger introduced the first industrial robotic arm, the 2,700-pound Unimate #001, in 1961 that robots became a staple in modern manufacturing.

Unimates were first called into service in the automobile industry, and today, automobile manufacturers like BMW continue to be progressive leaders using robots on the factory floor. At a BMW plant in Spartanburg, South Carolina, for example, collaborative robots help glue down insulation and water barriers on vehicle doors while their human counterparts hold the material in place.

The advent of high-end sensors, better microprocessors, and cheaper and easily programmable industrial robots is transforming industry today, with many mid-size and smaller companies considering automation and the use of collaborative robots.

Worldwide use of industrial robots is expected to increase from about 1.8 million units at the end of 2016 to 3 million units by 2020, according to the International Federation of Robotics. China, South Korea, and Japan use the most industrial robots, followed by the United States and Germany.

Anticipating further growth in industrial robotics, the Obama administration created the national Advanced Robotics Manufacturing Institute, bringing together the resources of private industry, academia, and government to spark innovations and new technologies in the fields of robotics and artificial intelligence. UConn’s Robotics and Controls Lab is a member of that initiative, along with the United Technologies Research Center, UTC Aerospace Systems, and ABB US Corporate Research in Connecticut.

Manufacturers see real value in integrating collaborative robots into their production lines. The biggest concern, clearly, is safety.

There have been 39 incidents of robot-related injuries or deaths in the U.S. since 1984, according to the federal Occupational Safety and Health Administration. To be fair, none of those incidents involved collaborative robots and all of them were later attributed to human error or engineering issues.

The first human known to have been killed by a robot was Robert Williams in 1979. Williams died when he got tired of waiting for a part and climbed into a robot’s work zone in a storage area in a Ford Motor plant in Flat Rock, Michigan. He was struck on the head by the robot’s arm and died instantly. The most recent incident happened in January 2017, when an employee at a California plastics plant entered a robot’s workspace to tighten a loose hose and had his sternum fractured when the robot’s arm suddenly swung into action.

“When you have a human and a robot trying to do a joint task, the first thing you need to think about of course is safety,” says Dani. “In our lab, we use sensors that, along with our algorithms, not only allow the robot to figure out where the human is but also allow it to predict where the human might be a few seconds later.”

One way to do that is to teach robots the same assembly steps taught to their human counterparts. If the robot knows the order of the assembly process, it can anticipate its human partners’ next moves, thereby reducing the possibility of an incident, Dani says. Knowing the process would also allow robots to help humans assemble things more quickly if they can anticipate an upcoming step and prepare a part for assembly, thus improving factory efficiency.

“Humans are constantly observing and predicting each other’s movements. We do it subconsciously,” says Ravichandar. “The idea is to have robots do the same thing. If the robot sees its human partner performing one step in an assembly process, it will automatically move on to prepare for the next step.”

Which brings us back to the whiteboards. And the math.

Failure is always an option. But when the math finally works, Ravichandar says, the success is exhilarating.

“Once you have the math figured out, it’s the best feeling because you know what you want the robot to do is going to work,” Ravichandar says with an excited smile.

“Implementing it is a whole other challenge,” he adds quickly, his passion for his work undiminished. “Things never work the first time. You have to constantly debug the code. But when you finally see the robot move, it is great because you know you have translated this abstract mathematical model into reality and actually made a machine move. It doesn’t get any better than that.”

With an eye on developing collaborative robotics that will assist with manufacturing, Dani and his team spent part of the past year teaching their lab’s test robot to identify tools laid out on a table so it can differentiate between a screwdriver, for example, and a crescent wrench, even when the tools’ initial positions are rearranged. Ultimately, they hope to craft algorithms that will help the robot work closely with a human counterpart on basic assembly tasks.

Another member of the team, Ph.D. candidate Gang Yao, is developing programs that help a robot track objects it sees with its visual sensors. Again, things we humans take for granted, such as being able to tell the difference between a bird and a drone flying above the trees, a robot has to learn.

Building advanced artificial intelligence doesn’t happen overnight. Ravichandar has been working on his projects for more than three years. It is, as they say, a process. Yet the team has learned to appreciate even the smallest of advances, and late last year, he flew to California to present some of the lab’s work to an interested team at Google.

“C-3PO is a protocol droid with general artificial intelligence,” says Ravichandar. “What we are working on is known as narrow artificial intelligence. We are developing skills for the robot one task at a time and designing algorithms that guarantee that whatever obstacles or challenges the robot encounters, it will always try to figure out a safe way to complete its given task as efficiently as it can. With generalized intelligence, a robot brings many levels of specific intelligence together and can access those skills quickly on demand. We’re not at that point yet. But we are at a point where we can teach a robot a lot of small things.”

Inevitably, as robots gain more and more human characteristics, people tend to start worrying about how much influence robots may have on our future.

Robots certainly aren’t going away. Saudi Arabia recently granted a robot named Sophia citizenship. Tesla’s Elon Musk and Deep Mind’s Mustafa Suleyman are currently leading a group of scientists calling for a ban on autonomous weapons, out of concern for the eventual development of robots designed primarily to kill.

Although it doesn’t apply directly to their current research, Dani and Ravichandar say they are well aware of the ethical concerns surrounding robots with advanced artificial intelligence.

Ravichandar says the problem is known in the field as “value alignment,” where developers try to make sure the robot’s core values are aligned with those of humans. One way of doing that, Ravichandar says, is to create a safety mechanism, such as making sure the robot always understands that the best solution it can come up with for a problem might not always be the best answer.

“The time is coming when we will need to have consensus on how to regulate this,” says Ravichandar. “Like any technology, you need to have regulations. But I think it’s absolutely visionary to inject humility into robots, and that’s happening now.”

That’s good news for the rest of us, because killer robots certainly are not the droids we’re looking for.


originally written by Colin Poitras

Zhang and His Students Look To Advance Power Systems Into The Future

Peng with the members of his Power and Energy Systems Lab. (UConn Photo/Christopher Larosa)

Over 350 million people in the United States depend on the reliability and consistency of the 450,000 miles of high-voltage lines that form the U.S. power grid to do important daily tasks. With stronger weather events and an increasing number of cyber-attacks, reliable safeguards and technologies are needed to protect this very important utility. The Power and Energy Systems Laboratory, run by Dr. Peng Zhang, F.L. Castleman Associate Professor of Electrical and Computer Engineering and his graduate students, aims at tackling these important issues.

The lab, which focuses on smart grid technology, microgrids, and sustainable energy, has worked on several crucial projects over the past several years, including a dedicated approach to networking the grid system, determining risk assessment models for unintentional islanding of power generators, using ocean waves to generate a sustainable power source, and many other related areas of research.

The lab’s most recent research, which focuses on national infrastructure, is looking to enhance the connectedness of the fractured U.S. grid system:  

“The goal of our research is to make our nation’s energy infrastructure resilient, reliable, secure, and sustainable,” Zhang said. “One of our main areas of focus now is large systems power stability, which is important, because there are no tools available to really assess and predict the status of the system. This tool is highly needed, especially for connected systems.”

That tool, which is being developed from a $1.05 million Department of Energy grant, is being worked on by Zhang and Ph.D. student Yan Li. The idea for the grant was inspired by the learnings Zhang and his students gathered from studying the tools used by local utility company, Eversource Energy:

“The whole U.S. and Canadian grid are connected together, and it’s a huge system, so it’s  therefore very difficult to monitor, assess and control its stability ,” Zhang said. “If you look at Connecticut, companies like Eversource, for the most part, use off-line tools which run many scenarios and only look at snapshots of the system, but that kind of work is not great for analysis, it needs to be monitored and assessed in real-time.”

In particular, Li and Zhang will be forming a formal theory with mathematical rigor, which will be established for computing the bounds of all possible trajectories and estimating the stability margin for the entire system, including the integrated transmission and distribution network.

Furthermore, a new open-source tool via reachable set computations will be developed for real-time dynamic analysis and stability margin calculations. It will be applicable for not only forecasting and monitoring grid performance, but also formally verifying various resiliency enhancement strategies, such as new schemes for system integrity protection and automation to adapt to this evolution of electric networks. 

Zhang and his students are also making significant contributions to knowledge advancement in the state of Connecticut and the region, with their work at the Eversource Energy Center at UConn, which conducts research related to advancing energy technology, as well as performs significant consulting work with Eversource Energy.

Zhang and his students have specifically performed work on research related to unintentional islanding, which is a phenomenon in which a distributed generator continues to be electrified and running, even when the electrical grid surrounding it is no longer active. Traditional methods of detection can often be fooled to think that grid conditions are normal, especially when multiple power generation devices are connected to the same line. 

(UConn Photo/Christopher Larosa)

This kind of scenario is very dangerous to field workers, as the lack of knowledge could cause them to be electrocuted. Zhang said that the research that’s currently being done by himself and his team addresses the creation a risk assessment model to safely avoid danger:

“This kind of research is using machine learning to predict risk in the non-detection zone,” Zhang said. “By non-detection we mean that when an island occurs, there are certain scenarios where the utility company, but with this new risk assessment model, we’ll be able to accurately predict when this scenario is likely to occur.”

Most importantly though, Zhang is happy that the graduate students that he is mentoring, and providing hands-on opportunities to, are getting the necessary experience needed to launch their careers in research and academia.

Ph.D. student Taofeek Orekan, one of the members of the lab, is one of the students that will be using that experience, as he looks for post-grad opportunities in the next few months:

“This is a great lab to start off in,” Orekan said. “I know that any lab that I launch during my career in academia will absolutely be an extension of this lab.”

For more information on the lab, visit 

Remembering Charles Knapp, an Engineering Icon

Professor Emeritus Charles Harris Knapp, 86, passed away at

Robert (left) and Charles ’53, ’62 Knapp. (Photo courtesy of The UConn Foundation)

home on Thanksgiving Day

surrounded by his family and loving wife of 62 years, Charleen Gaudet Knapp. Over the course of his 40-year career at UConn, Dr. Knapp enriched and shaped the

lives of innumerable students and inspired an imperishable legacy, The Charles H. Knapp Associate Professorship in Electrical Engineering.

Charlie (or “Red” or “Harris,” as he was variously known) was so eminent in his field, that a handwritten note recommending a student to a graduate program at another well-regarded university signed only “Charlie” was sufficient; no letterhead needed. Colleagues and former students remembering Charlie reiterate his traits as an educator, mentor, and researcher calling him “a true gentleman,” showing people “how to treat others,” “the best teacher,” and never seeing “him lose his cool.”

A lifelong Yankees fan, Charlie was born in New York City, but moved with his family to Coventry, CT at an early age. As an Electrical Engineering undergraduate at UConn, he was the first University Scholar from the School of Engineering and member of Tau Beta Phi, Eta Kappa Nu, and Sigma Xi engineering honor societies. Upon graduation in 1953, Charlie served two years of active duty in the U.S. Air Force before earning his Master’s degree from Yale University and working for RCA and IBM. He returned to UConn in 1958 and became the first candidate awarded a Ph.D. in Electrical Engineering.   

Charlie had many interests outside of academia. He began running in his late 20s, was an avid gardener, active member of the Storrs Congregational Church serving in many capacities including his favorite, singing in the choir for more than 40 years, and spending time with his four children and their families.

In 2012, the Knapp children honored their father with a generous donation establishing The Charles H. Knapp Associate Professorship in Electrical Engineering. Explaining the decision to create an associate professorship, typically given to newer, upcoming faculty, Charlie said, “The younger professors have the freshest education and ideas and are looking for places where they can grow. This professorship will give its holder an edge and will be very helpful in retaining the best associate professors. It is important for the university and the college if we can keep them here, and it’s good for local industry, as well.”

During his lifetime, Professor Knapp enriched the lives of countless students and colleagues, family and friends. Through their lives and his eponymous professorship, his legacy and memory will endure for generations.


Donations in his memory may be made to: The Charles H. Knapp Associate Professorship in Electrical Engineering, c/o The UConn Foundation, 2390 Alumni Drive., Unit 3206, Storrs, CT, 06269-3206

Portable Microscope Makes Field Diagnosis Possible

Siddharth Rawat, left, a Ph.D. student, and Bahram Javidi, Board of Trustees Distinguished Professor of Electrical and Computer Engineering, operate a prototype device to examine blood samples for diseases. The portable holographic field microscope offers medical professionals a fast and reliable tool for the identification of diseased cells. (Peter Morenus/UConn Photo)

A portable holographic field microscope developed by UConn optical engineers could provide medical professionals with a fast and reliable new tool for the identification of diseased cells and other biological specimens.

The device, featured in a recent paper published by Applied Optics, uses the latest in digital camera sensor technology, advanced optical engineering, computational algorithms, and statistical analysis to provide rapid automated identification of diseased cells.

One potential field application for the microscope is helping medical workers identify patients with malaria in remote areas of Africa and Asia where the disease is endemic.

Quick and accurate detection of malaria is critical when it comes to treating patients and preventing outbreaks of the mosquito-borne disease, which infected more than 200 million people worldwide in 2015, according to the Centers for Disease Control. Laboratory analysis of a blood sample remains the gold standard for confirming a malaria diagnosis.  Yet access to trained technicians and necessary equipment can be difficult and unreliable in those regions.

The microscope’s potential applications go far beyond the field diagnosis of malaria. The detailed holograms generated by the instrument also can be used in hospitals and other clinical settings for rapid analysis of cell morphology and cell physiology associated with cancer, hepatitis, HIV, sickle cell disease, heart disease, and other illnesses, the developers say.

In checking for the presence of disease, most hospitals currently rely on dedicated laboratories that conduct various tests for cell analysis and identification. But that approach is time consuming, expensive, and labor intensive. It also has to be done by skilled technicians working with the right equipment.

“Our optical instrument cuts down the time it takes to process this information from days to minutes,” says Bahram Javidi, Board of Trustees Distinguished Professor in the Department of Electrical and Computer Engineering and the microscope’s senior developer. “And people running the tests don’t have to be experts, because the algorithms will determine if a result is positive or negative.”

The research team consulted with hematologists, and the algorithms used with the instrument are able to compare a sample against the known features of healthy cells and the known features of diseased cells in order to make proper identification. “It’s all done very quickly,” Javidi says.

How the Device Works

When it comes to identifying patients with malaria, here’s how the device works: A thin smear from a patient’s blood sample is placed on a glass side, which is put under the microscope for analysis. The sample is exposed to a monochromatic light beam generated by a laser diode or other light source. Special components and optical technologies inside the microscope split the light beam into two beams in order to record a digital hologram of the red blood cells in the sample. An image sensor, such as a digital webcam or cell phone camera, connected to the 3-D microscope captures the hologram.  From there, the captured data can be transferred to a laptop computer or offsite laboratory database via the internet. Loaded with dedicated algorithms, the computer or mobile device hardware reconstructs a 3-D profile of the cell and measures the interaction of light with the cell under inspection. Any diseased cells are identified using computer pattern recognition software and statistical analysis.

Quantitative phase profiles of healthy red blood cells (top row) and malaria infected cells (bottom row). (Holographic microscope image courtesy of Bahram Javidi)

Red blood cells infected with the malaria-causing Plasmodium parasite exhibit different properties than healthy blood cells when light passes through them, Javidi says.

“Light behaves differently when it passes through a healthy cell compared to when it passes through a diseased cell,” Javidi says. “Today’s advanced sensors can detect those subtle differences, and it is those nanoscale variations that we are able to measure with this microscope.”

Conventional light microscopes only record the projected image intensity of an object, and have limited capability for visualizing the detailed quantitative characterizations of cells. The digital holograms acquired by UConn’s 3-D microscope, on the other hand, capture unique micro and nanoscale structural features of individual cells with great detail and clarity. Those enhanced images allow medical professionals and researchers to measure an individual cell’s thickness, volume, surface, and dry mass, as well as other structural and physiological changes in a cell or groups of cells over time – all of which can assist in disease identification, treatment, and research. For instance, the device could help researchers see whether new drugs impact cells positively or negatively during clinical trials.

The techniques associated with the holographic microscope also are non-invasive, highlighting its potential use for long-term quantitative analysis of living cells.

Conventional methods of testing blood samples for disease frequently involve labeling, which means the sample is treated with a chemical agent to assist with identification. In the case of malaria, red blood cells are usually treated with a Giemsa stain that reacts to proteins produced by malaria-carrying parasites and thus identifies them. But introducing a chemical into a live cell can change its behavior or damage it.

“If you’re doing an in vitro inspection of stem cells, for instance, and you introduce a chemical agent, you risk damaging those cells. And you can’t do that, because you may want to introduce those cells into the human body at some point,” Javidi says. “Our instrument doesn’t rely on labeling, and therefore avoids that problem.” 

Ph.D. students Tim O’Connor ’17 (ENG), left, Siddharth Rawat, and Adam Markman ’11 (ENG) operate a prototype device to examine blood samples for diseases at the Javidi lab in the Information Technologies Engineering Building. (Peter Morenus/UConn Photo)

The holographic microscope was developed in UConn’s new Multidimensional Optical Sensing & Imaging Systems or MOSIS lab, where Javidi serves as director. The MOSIS lab integrates optics, photonics, and computational algorithms and systems to advance the science and engineering of imaging from nano to macro scales.

A comprehensive report on the MOSIS lab’s work with 3-D optical imaging for medical diagnostics was published last year in Proccedings of the IEEE, the top-ranked journal for electrical and electronics engineering. Joining Javidi in this research are graduate students Adam Markman, Siddharth Rawat, Satoru Komatsu, and Tim O’Connor from UConn; and Arun Anand, an applied optics specialist with Maharaja Sayajirao University of Baroda in Vadodara, India.

The microscope research is supported by Nikon and the National Science Foundation (ECCS 1545687). Students are supported by the U.S. Department of Education, GE, and Canon fellowships. Other sponsors that have supported Javidi’s broader research work and the MOSIS lab over the years include the Defense Advanced Research Projects Agency or DARPA, the U.S. Airforce Research Lab, the U.S. Army, the Office of Naval Research, Samsung, Honeywell, and Lockheed Martin. He has collaborated with colleagues from numerous universities and industries around the world during his time at UConn, including research facilities in Japan, Korea, China, India, Germany, England, Italy, Switzerland, and Spain, among other countries.

Javidi is working with colleagues at UConn Health, including medical oncology and hematology specialist Dr. Biree Andemariam and her staff, for other medical applications. UConn’s tech commercialization office has been involved in discussing potential marketing opportunities for the portable digital microscope. A prototype of the microscope used for initial tests was assembled using 3-D printing technologies, lowering its production costs.


Original from UConn Today, Colin Poitras.

Navy Using New UConn Software to Improve Navigation

The Navy is using new software developed by UConn engineering professor Krishna Pattipati to vastly improve the ability to route ships through unpredictable situations.

Major research discoveries generate news headlines. But a research undertaking by one University of Connecticut engineering lab seeks to forestall some headlines of a different kind.

The loss of life because of weather events, as happened on Oct. 1, 2015 when cargo ship El Faro sank with its 33-member crew in Hurricane Joaquin, is one example. Transcripts released by the National Transportation Safety Board showed an increasingly anxious and panicked crew as the 790-foot vessel sailed into the raging storm two years ago.

Software developed by Krishna Pattipati, UTC Professor in Systems Engineering at UConn and his research team, in collaboration with the U.S. Naval Research Laboratory-Monterey, may go a long way toward avoiding such tragedies.

The prototype, named TMPLAR (Tool for Multi-objective Planning and Asset Routing), is now being used by the Navy to vastly improve the ability of ships to reroute through unpredictable weather. It is the type of technology transition that the new National Institute for Undersea Vehicle Technology based at UConn Avery Point, is now able to foster.

Screenshot of a requested ship transit from San Diego, California, toward Alaska. The black line is the suggested route the Navy navigator is given to accept or reject and send on as directions to a ship’s captain. The numbered red circles are ‘waypoints’ along the route, with the starting point labeled ‘0’. These waypoints divide up a possibly long voyage and keep the ship’s path in check.

Created by Pattipati and electrical and computer engineering graduate students David Sidoti, Vinod Avvari, Adam Bienkowski, and Lingyi Zhang, and undergraduate students Matthew Macesker and Michelle Voong, TMPLAR is still in development, but it has already been fully integrated with the Navy’s meteorology and oceanographic weather forecasts.

Members of the UConn team meet weekly with Navy officials, via teleconference, to discuss project updates and receive  feedback.

“Their progress is fast,” says Sidoti. “Frankly, it’s kept us on our toes as we try to manage both our academic responsibilities here at UConn while enhancing and updating the software.”

TMPLAR is like a much more complex version of Google Maps, because it will be applied to ships and submarines, where there is no underlying network of roadways to navigate.

In Google Maps, a user typically seeks to maximize the average speed of travel between start and end locations to get to a destination in the shortest amount of time, hence the route may favor highways instead of back roads.

Pattipati’s team is now approaching problems with upwards of 17 or more objectives, which may change depending on the vehicle and the conditions.

The algorithms take into account obstacles such as ocean depth, undersea pipelines, cables, oil rigs, for example. And they factor in multiple user objectives, whether to traverse to an area to minimize travel time, maximize fuel efficiency given the predicted weather, accomplish training objectives, or maximize operational endurance.

“The tool guarantees safe travel from any point in the ocean, above, on, or below its surface, while making choices en route that optimize fuel consumption and cater to any set of objectives of the operator,” says Sidoti. “Using special clustering techniques, the tool’s algorithms have even been applied to finding low-risk routes that avoid storms or hurricanes.”

The next step for TMPLAR is programming the tool for use by aircraft, such as drones.

Last month, Pattipati and Sidoti traveled to San Diego to demonstrate the capabilities of the software to the Space and Naval Warfare Systems Center Pacific. Their algorithim is now going to be integrated with a tool for aircraft carrier strike group planning.

The lab first published details about the software last year in the journal IEEE, the world’s largest professional organization for the advancement of technology. Avvari, one of the graduate students, will detail some of the enhancements that have been made since then at an upcoming professional conference.

And, as the software transitions to operational settings, the team is looking to speed up the capabilities to output smart weather-informed route recommendations in less than a second. Adding neural network modules to TMPLAR is another new horizon; artificial intelligence would help condense solutions so it is less overwhelming to a user, says Sidoti.

When he reviewed the factors faced by the crew of El Faro using TMPLAR software, Sidoti was able to find safe routes for the ship that involved waiting at waypoints and varying the ship’s speed in order to avoid unsafe environmental conditions, while also reducing costs of the route.

The Coast Guard’s report on the tragedy – released just a month ago – said the captain misjudged the strength of Hurricane Joaquin and should have changed the El Faro’s course.

Sidoti found up to eight possible safe routes using TMPLAR. That’s the sort of information he hopes other captains will have.

Recently, the team received notification that the software was demo’ed to onboard ship navigators who were interested to the point that they requested the ability to use it in order to plan and test it on a real-world deployment.

Funding for this research is supported by the U.S. Office of Naval Research under contracts #N00014-16-1-2036 and #N00014-12-1-0238; by the Naval Research Laboratory under contract #N00173-16-1-G905; and by the Department of Defense High Performance Computing Modernization Program under subproject contract #HPCM034125HQU.


Original from UConn Today, Kristen Cole.

Award-winning Paper Questions ECG As Secure Biometric

A paper from UConn fourth year PhD student Nima Karimian has won the best student paper award at the recent IJCB 2017 conference in Denver.

The Conference

The International Joint Conference on Biometrics (IJCB 2017) combines two major biometrics research annual conferences, the Biometrics Theory, Applications and Systems (BTAS) conference and the International Conference on Biometrics (ICB). The blending of these two conferences in 2017 is through special agreement between the IEEE Biometrics Council and the IAPR TC-4, and presents an exciting event for the entire worldwide biometrics research community.

The Paper

The paper, “On the Vulnerability of ECG Verification to Online Presentation Attacks,” examined the use of Electrocardiogram (ECG) as a secure biometric modality. ECG has long been regarded as a biometric modality which is impractical to copy, clone, or spoof. However, it was recently shown that an ECG signal can be replayed from arbitrary waveform generators, computer sound cards, or off-the-shelf audio players. The award-winning paper is one of the first in the field to seriously question the security of ECG verification, and goes a long way towards debunking the assumption of its security.

The paper developed a novel presentation attack where a short template of the victim’s ECG is captured by an attacker and used to map the attacker’s ECG into the victim’s, which can then be provided to the sensor using one of the above sources. The authors’ approach involved exploiting ECG models, characterizing the differences between ECG signals, and developing mapping functions that transform any ECG into one that closely matches an authentic user’s ECG. Their proposed approach, which can operate online or on-the-fly, is compared with a more ideal offline scenario where the attacker has more time and resources. In the experiments, the offline approach achieved average success rates of 97.43% and 94.17% for non-fiducial and fiducial based ECG authentication. In the online scenario, the performance is degraded by 5.65% for non-fiducial based authentication, but is nearly unaffected for fiducial authentication.

The work was supported by US Army Research Office (ARO) under award number W911NF16-1-0321.

Teaching Robots to Think

Original Author: Colin Poitras – UConn Communications – September 13, 2017

Ashwin Dani, assistant professor of electrical and computer engineering, demonstrates how the robot can be given a simple task which can be repeated. Sept. 7, 2017. (Sean Flynn/UConn Photo)

In a research building in the heart of UConn’s Storrs campus, assistant professor Ashwin Dani is teaching a life-size industrial robot how to think.

Here, on a recent day inside the University’s Robotics and Controls Lab, Dani and a small team of graduate students are showing the humanoid bot how to assemble a simple desk drawer.

The “eyes” on the robot’s face screen look on as two students build the wooden drawer, reaching for different tools on a tabletop as they work together to complete the task.

The robot may not appear intently engaged. But it isn’t missing a thing – or at least that’s what the scientists hope. For inside the robot’s circuitry, its processors are capturing and cataloging all of the humans’ movements through an advanced camera lens and motion sensors embedded into his metallic frame.

Ashwin Dani, assistant professor of electrical and computer engineering, is developing algorithms and software for robotic manipulation, to improve robots’ interaction with humans. (Sean Flynn/UConn Photo)

Ultimately, the UConn scientists hope to develop software that will teach industrial robots how to use their sensory inputs to quickly “learn” the various steps for a manufacturing task – such as assembling a drawer or a circuit board – simply by watching their human counterparts do it first.

“We’re trying to move toward human intelligence,” says Dani, the lab’s director and a faculty member in the School of Engineering. “We’re still far from what we want to achieve, but we’re definitely making robots smarter.”

To further enhance robotic intelligence, the UConn team is also working on a series of complex algorithms that will serve as an artificial neural network for the machines, helping robots apply what they see and learn so they can one day assist humans at their jobs, such as assembling pieces of furniture or installing parts on a factory floor. If the process works as intended, these bots, in time, will know an assembly sequence so well, they will be able to anticipate their human partner’s needs and pick up the right tools without being asked – even if the tools are not in the same location as they were when the robots were trained.

This kind of futuristic human-robot interaction – called collaborative robotics – is transforming manufacturing. Industrial robots like the one in Dani’s lab already exist. Although currently, engineers must write intricate computer code for all of the robot’s individual movements or manually adjust the robot’s limbs at each step in a process to program it to perform. Teaching industrial robots to learn manufacturing techniques simply by observing could reduce to minutes a process that currently can take engineers days.

From left back row, Ph.D. students Iman Salehi, Harish Ravichandar, Kyle Hunte, Gang Yao, and seated, Ashwin Dani, assistant professor of electrical and computer engineering. (Sean Flynn/UConn Photo)

“Here at UConn, we’re developing algorithms that are designed to make robot programming easier and more adaptable,” says Dani. “We are essentially building software that allows a robot to watch these different steps and, through the algorithms we’ve developed, predict what will happen next. If the robot sees the first two or three steps, it can tell us what the next 10 steps are. At that point, it’s basically thinking on its own.”

In recognition of this transformative research, UConn’s Robotics and Controls Lab was recently chosen as one of 40 academic or academic-affiliated research labs supporting the U.S. government’s newly created Advanced Robotics for Manufacturing Institute or ARM. One of the collaborative’s primary goals is to advance robotics and artificial intelligence to maintain American manufacturing competitiveness in the global economy.

“There is a huge need for collaborative robotics in industry,” says Dani. “With advances in artificial intelligence, lots of major companies like United Technologies, Boeing, BMW, and many small and mid-size manufacturers, are moving in this direction.”

The United Technologies Research CenterUTC Aerospace Systems, and ABB US Corporate Research – a leading international supplier of industrial robots and robot software – are also representing Connecticut as part of the new ARM Institute. The institute is led by American Robotics Inc., a nonprofit associated with Carnegie Mellon University.

Connecticut’s and UConn’s contribution to the initiative will be targeted toward advancing robotics in the aerospace and shipbuilding industries, where intelligent, adaptable robots are more in demand because of the industries’ specialized needs.

Joining Dani on the ARM project are UConn Board of Trustees Distinguished Professor Krishna Pattipati, the University’s UTC Professor in Systems Engineering and an expert in smart manufacturing; and assistant professor Liang Zhang, an expert in production systems engineering.

“Robotics, with wide-ranging applications in manufacturing and defense, is a relatively new thrust area for the Department of Electrical and Computer Engineering,” says Rajeev Bansal, professor and head of UConn’s electrical and computer engineering department. “Interestingly, our first two faculty hires in the field received their doctorates in mechanical engineering, reflecting the interdisciplinary nature of robotics. With the establishment of the new national Advanced Robotics Manufacturing Institute, both UConn and the ECE department are poised to play a leadership role in this exciting field.”

The aerospace, automotive, and electronics industries are expected to represent 75 percent of all robots used in the country by 2025. One of the goals of the ARM initiative is to increase small manufacturers’ use of robots by 500 percent.

Industrial robots have come a long way since they were first introduced, says Dani, who has worked with some of the country’s leading researchers in learning and adoptive control, and robotics at the University of Florida (Warren Dixon) and the University of Illinois at Urbana-Champaign (Seth Hutchinson and Soon-Jo Chung). Many of the first factory robots were blind, rudimentary machines that were kept in cages and considered a potential danger to workers as their powerful hydraulic arms whipped back and forth on the assembly line.

Today’s advanced industrial robots are designed to be human-friendly. High-end cameras and elaborate motion sensors allow these robots to “see” and “sense” movement in their environment. Some manufacturers, like Boeing and BMW, already have robots and humans working side-by-side.

Of course, one of the biggest concerns within collaborative robotics is safety.

In response to those concerns, Dani’s team is developing algorithms that will allow industrial robots to quickly process what they see and adjust their movements accordingly when unexpected obstacles – like a human hand – get in their way.

“Traditional robots were very heavy, moved very fast, and were very dangerous,” says Dani. “They were made to do a very specific task, like pick up an object and move it from here to there. But with recent advances in artificial intelligence, machine learning, and improvements in cameras and sensors, working in close proximity with robots is becoming more and more possible.”

Dani acknowledges the obstacles in his field are formidable. Even with advanced optics, smart industrial robots need to be taught how to distinguish a metal rod from a flexible piece of wiring, and to understand the different physics inherent in each.

Movements that humans take for granted are huge engineering challenges in Dani’s lab. For instance: Inserting a metal rod into a pre-drilled hole is relatively easy. Knowing how to pick up a flexible cable and plug it into a receptacle is another challenge altogether. If the robot grabs the cable too far away from the plug, it will likely flex and bend. Even if the robot grabs the cable properly, it must not only bring the plug to the receptacle but also make sure the plug is oriented properly so it matches the receptacle precisely.

“Perception is always a challenging problem in robotics,” says Dani. “In artificial intelligence, we are essentially teaching the robot to process the different physical phenomena it observes, make sense out of what it sees, and then make the appropriate response.”

Research in UConn’s Robotics and Controls Lab is supported by funding from the U.S. Department of Defense and the UTC Institute of Advanced Systems Engineering. More detailed information about this research being conducted at UConn, including peer-reviewed article citations documenting the research, can be found here. Dani and graduate student Harish Ravichandar also have two patents pending on aspects of this research: “Early Prediction of an Intention of a User’s Actions,” Serial #15/659,827, and “Skill Transfer From a Person to a Robot,” Serial #15/659,881.

Embedded System Competition Award


A UConn team of students competed in a MITRE-sponsored embedded systems security capture the flag competition this semester and got first place. The team was led by UG ECE students Brian Marquis and Patrick Dunham with grad student Chenglu Jin and two CSE UG students.

UConn Chapter of HKN wins the Outstanding Chapter Award (2015-2016)

The IEEE-HKN Board of Governors has conferred on the UConn Chapter of HKN (Eta Kappa Nu: the electrical engineering honor society) the 2015-2016 IEEE-HKN Outstanding Chapter Award. This award is presented to IEEE-HKN chapters in recognition of excellence in their chapter administration and programs. Recipients are selected on the basis of their annual chapter report. Winning chapter reports not only showcase their chapter’s activities in an individualized manner, they provided multiple views and instances of their work, which really brought their chapter’s activities to life. Of critical concern to the Outstanding Chapter Awards evaluation committee in judging a chapter are activities to: improve professional development; raise instructional and institutional standards; encourage scholarship and creativity; provide a public service, and generally further the established 

goals of IEEE-HKN.


The UConn Chapter is one of 21 chapters selected for their outstanding performance and the value they bring to their members, peers, and university.

UConn Named to Advanced Robotics Manufacturing Institute

The new national initiative aims to increase small manufacturers’ use of robots by 500 percent. Researchers at UConn will focus on the aerospace and shipbuilding industries. (Getty Images)

The University of Connecticut is part of a new national institute designed to advance robotics manufacturing and maintain America’s global competitiveness in that arena. UConn researchers will help develop new sensing, software, artificial intelligence, and other technologies to improve the use of robotics in manufacturing for the aerospace and shipbuilding industries.

The institute, called the Advanced Robotics Manufacturing Institute (ARM), was announced earlier this month and will include several Connecticut businesses and academic institutions. The Connecticut portion of the proposal was led by UConn, the United Technologies Research Center, UTC Aerospace Systems, and ABB US Corporate Research. The institute will be led by American Robotics Inc., a nonprofit associated with Carnegie Mellon University in Pittsburgh, Penn.

The ARM institute is the 14th and final national institute created under President Obama’s Manufacturing USA initiative, according to Michael Accorsi, senior associate dean of engineering.

“The focus on robotics makes it a great fit for Connecticut, with our strong ties to the aerospace and shipbuilding industries – industries that can really benefit from the next generation of robotic innovation,” he said.

The new institute is supported by a total of $253 million in funding. Federal funding represents $80 million of that, with the remaining money coming from 123 industrial partners, 40 academic and academically affiliated partners, and 64 government and nonprofit partners.

At UConn, Ashwin P. Dani’s Robotics and Controls Lab is already performing research on interactions between robots and humans. Dani and his graduate students are creating algorithms so that industrial robots can learn what action a person will likely take in a given situation. By understanding where a person will move, a robot can work alongside a human and avoid injuring them.

“The new institute is designed to create an ecosystem of robotics,” said Dani, assistant professor of electrical and computer engineering. “That ecosystem will involve creating collaborative robotics that can do flexible, highly variable jobs efficiently and create advancements in artificial intelligence, particularly human-robotic interactions. That’s an area we already focus on here at UConn.”

The U.S. Department of Defense’s Manufacturing USA initiative is designed to encourage private industry, academia, and government collaboration to revitalize and enhance U.S. competitiveness in key areas. As a part of ARM, UConn will create a new, advanced robotics facility within the new UConn Tech Park, which will expand on UConn’s existing robotic capabilities.

The aerospace, automotive, and electronics industries will represent 75 percent of all robots used in the country by 2025. UConn and other Connecticut partners are focusing on the aerospace and ship building industries, which have been slower to adopt robotic technologies than the automotive industry. Dani said that because these industries create a smaller volume of products than the automotive industry, they need robots that can do a variety of tasks.

“The automotive industry makes millions of cars every year, so each robot can be highly specialized. The aerospace industry creates far fewer individual products, so each robot needs to be able to quickly learn and perform multiple tasks,” Dani said. “UConn and ARM will make the innovations necessary to create agile, dexterous, and collaborative robotics.”

The new institute aims to increase small manufacturers’ use of robots by 500 percent. UConn will work with community colleges around the state to provide training in robotic jobs within existing STEM programs, to meet the increasing demands for the robotic manufacturing industry.