Trends in HMI technology

(Photo: DLG)

The application of connectivity and automation in the off-highway equipment sector will enhance the interaction between man and machine. The ideal human-machine interface (HMI) system supports the operator by providing important information at the right time. Camera and radar systems facilitate all-round visibility and can contribute to increasing safety. Augmented reality (XR) is also finding its way into assistance systems and HMI solutions for mobile machinery.

“Mobile machinery must be able to work reliably in a harsh and constantly changing environment. The machinery performs tasks that demands the operator’s undivided attention,” said Petra Kaiser,  organizer of Systems & Components, DLG (German Agricultural Society). “Autonomous functions are therefore enhanced support for operators. These functions reduce workload, while at the same time the machine operator retains control.”

HMI specialists will be part of the Systems & Components portion of the Agritechnica trade show, which takes place November 9 -15 at the exhibition grounds in Hanover, Germany. The B2B event is organized by the DLG. Systems & Components was created to be a meeting place for the international supplier industry for the off-highway sector. This year’s theme is “Touch Smart Efficiency.” 

The wide variety of machines in the agricultural and construction industries leads to the need for flexible solutions. “This means that new functions for the autonomous operation of a machine can be added piece by piece. This development is also reflected at the exhibition grounds in Hanover,” said Kaiser.

Reliable under harsh conditions

In the future, system architecture in mobile machines will be required to process more data from an increasing number of sensors. Ideally, this information will be presented to the operator in an intuitive way, and smart HMI technology is essential for that to occur. It keeps the driver informed of the current situation during operation.

“An optimal communication between man and machine is required, in which the vehicle also increasingly takes on an active role,” said Kaiser.

The goal is for the vehicle to help the driver understand a given situation while enhancing safety and providing the necessary information - without overloading the operator. Throughout the Systems & Components hall, a variety of HMI solutions are expected to focus on continuous feedback of operating data such as speed, fuel consumption, crop yield and machine status. This makes it possible for the operator to make informed decisions in the cab and work more efficiently.

Mobile machines are increasingly being controlled remotely. In these cases, it is a challenge for human operators to maintain an overview of the machine and its surroundings. The latest developments use extended reality (XR) technologies. The digital extension of human senses is intended to take interaction with the machine to a new level.

For Christiana Seethaler, vice president, Product Development at TTControl, the integration of XR technologies into HMI products is the “next step towards creating smarter, more reliable off-highway machines.” A provider of functional safety systems, TTControl is a joint-venture company of TTTech and Hydac International with locations in Vienna, Austria and Brixen, Italy. 

XR technologies show the machine status in the driver’s immediate field of vision and haptic feedback can send tangible signals to joystick, for example. Acoustic signals can also draw attention to potential danger.

Sensor fusion

Solutions for accident prevention are part of Systems & Components, as well. In combination with external sensors, XR tools can help to identify obstacles which can enhance safety and increase efficiency.

“The conceivable scenarios and the added value of such information are key to increasing productivity for various areas of application in the agricultural and construction industries,” said Kaiser. “Autonomous technologies and assistance systems score points through improved processes, more precise work and contribution to increased safety. Work processes in the off-highway environment can be precisely planned and coordinated.”

Technology suppliers for off-highway machines are confronted with different environmental conditions. Poorly lighted environments pose challenges for camera systems, for example, and that is why several sensor systems such as LiDAR, ultrasound or radar are used in combination with RGB or infrared cameras.

The process of combining sensor data coming from different sources is known as sensor fusion. The application increases the reliability of the resulting commands of the assistance systems, said DLG, which can allow work cycles to be carried more effectively and more safely, which ultimately saves time and costs.

Haptic collision warning is an example of how modular assistance systems can contribute to safety on a farm or jobsite. While this type of information is still predominantly provided by the cameras and sensors that are mounted on the machine it, in the future data generated by the environment, for example by other vehicles, robots and drones, could be combined into an intuitive picture for the operator.