Autonomous Intelligence: About SafeAI and the Autonomous Operation Systems
Hitachi haul truck using SafeAI system at the Karlawinda mine, Western Australia.
SafeAI is a California-based technology company which has developed a retrofit autonomous driving system intended for vehicles used in mining and construction operations. The system is OEM and machine agnostic, meaning it can be fitted to any surface vehicle - as yet, the company does not apply the autonomous driving capability to underground machines.
“If you visit a mining site, they might have hundreds of vehicles, haul trucks, loaders, bulldozers, but they have a series of machine suppliers,” explains Sudhanshu Singh, senior vice president, Global Operations. Based on that mix, he says customer companies have highlighted the need for the autonomous systems to work across multiple brands and machine types, taking into account differences in design and function.
“I was involved in the design of these vehicles for 30 years before joining SafeAI,” says Singh. “They can operate in excess of 100,000 miles per year, possibly over a 15- or 20-year lifespan. The haul trucks carry perhaps three times their own weight and travel at 30 to 40 miles per hour, bouncing over holes in the ground. So the system we devised has to be as robust as the machines to which it is fitted.”
After taking the harsh operating environment into account, Singh points out that industry sectors such as mining offer some positives. “There are significantly fewer people coming into the path of these trucks, a lot less objects which the system has to analyse and avoid.” He adds that you can also predict a lot of features at mine sites, such as the vehicle routes, together with the shovel, load and dump areas. This detail helps with planning the automated guidance systems.
LEVERAGED TECH
The CEO of SafeAI, Bibhrajit Halder, has a background in development of autonomous driving systems for various passenger vehicle OEMs and ride share companies (Uber, Waymo, etc.). Singh says that the overarching plan is to leverage the tech developed for these companies by applying it in the off-highway machine sector. But beyond that, it’s basic computational power which supports such systems.
Sudhanshu Singh SVP of Operations at SafeAINone of this would be happening without the latest processor chips. Over the last five years, GPS, Lidar, camera [resolution], they have significantly improved and that’s what supports the system. We have leveraged that power, while also planning for updates every two years or so.
All of these guidance systems – and more – are fitted to the machines to support autonomous operation. Within just the Lidar function, there are a low- and high-range lasers being used to analyse vehicle surroundings, with onboard AI interpreting that information and making decisions on vehicle movement. “We have every type of sensing equipment on the machines and the data they provide is processed by a series of IMUs (inertial measurement units) to make decisions. This is the artificial intelligence (AI) in the SafeAI name,” he explains.
According to Singh, where autonomy for passenger cars is measured on a scale of 1 to 5, where Level 5 autonomy involves no human interaction with machine operation, in off-highway applications there is just Autonomy 1.0 and 2.0. He says that 1.0 was introduced about 15 years ago, but that was limited due to processor speeds. Autonomy 2.0 takes advantage of improvements in hardware, software and CPU power, while also leveraging neural networking to deliver the required AI. This means that machines are not wholly dependent on related networks, but can also learn on their own.
“We train the AI to understand what it will likely see, but there are some things that we cannot predict,” says Singh. He points to the ongoing trial of a machine at a mine in Australia, where there is the potential for the vehicles to encounter a kangaroo. “We can train the AI for this, but these are animals and in the end [the system] has to make its own decisions,” he adds.
AUTONOMY SOFTWARE
The underlying operating system used by SafeAI to power autonomous machine functionality is known as SafeAI Autonomous Framework (SAF). SAF is an operating system which provides easeof- use and functionality for development and deployment of autonomous operation applications.
Supported by SAF, the autonomous software is loaded onboard the given vehicle and it is this which controls the drive function, while also making AI-based decisions.
On top of that, SafeAI also produces an autonomous management system known as Zeno. This cloud-based software dispatches commands remotely to the vehicles, while also monitoring their status. While independent from SAF and the onboard autonomous software, Zeno works in synergy with the two systems to create a dynamic control system.
Route mapping for SafeAI systems.
DATA OVERLOAD
The amount of data produced by autonomous systems is impressively large. The A350, the latest airliner from Airbus, is reported to produce about 800 megabytes (Mb) of data per flight. The older A380 (the largest airliner in the world) produces half of this total. By comparison, an autonomous driving system in a passenger car can produce about 1.4 terabytes (Tb) of data per hour; the A350 airliner produces just 0.05% of that total.
Singh says that managing the data produced by the SafeAI system required special attention, simply due to the quantity of information. “The network was getting bogged down by the amount of data being produced,” he explains. “We had to reduce the load on the system. That’s not such a big problem with mining applications; it’s a feature of the AI to understand what needs to be actioned and what can be ignored.”
In addition to the hardware delivering the actionable data and software to manage data analysis, SafeAI also provides “autonomy kits” which include drive-by-wire systems. These are operated based on data received from the machine’s engine control unit (ECU) and the related CAN-bus system.
“Most machines use the CAN network. We don’t produce valves or other specific hardware, we have a series of suppliers which deliver those parts to support retrofitting of the trucks,” reports Singh.
He says that the physical retrofitting of a machine, which can be diesel or electrically-powered, can be completed “in just a couple of weeks”. While these are clearly complex systems, it’s understood that these machines are capital investments which must remain in operation. As such, the systems have been designed with ease of fitment in mind.
In early 2022, SafeAI partnered with MACA, a mining contractor group based in Australia, to retrofit a fleet of 100 mixed vehicles. The deal is reported to have created one of the world’s largest autonomous heavy equipment fleets.
Once installed, final validation takes advantage of the modular system design, as Singh puts forward: “We validate the final machines, but a lot of individual elements of the system have already been validated, tested in various ways. There’s not really anything new, other than mapping of the site.”
Should a new route be added on site, there are various ways to update the system. Singh says that the onboard AI could recognise the new addition and assimilate that into the existing mapping. Otherwise, GPS coordinates can be added, a light vehicle can map the route, or a drone can take aerial scans of the new route for upload to the mapping data.
While Singh is not at liberty to disclose pricing of the autonomous operation system, he says that it is “typically less than the cost of a new OEM-based autonomous vehicle”. In addition, ROI timeframes are expected to decrease due to the efficiencies unlocked by the autonomous vehicle functionality.
