Remote operations.. the next step toward AI

In reply to Keith Tanner :

DBEs are just a method for politicians to score points with their uneducated constituents by saying "We are hiring minority owned businesses" (which sounds an awful lot like "We are giving opportunities to minorities", but it's very different).

In reply to SV reX :

Yes, got it, thank you!

In reply to SV reX :

Here's our parking lot right now. Our building and parking spaces are on the left. On the right, is a new set of apartment buildings that are being constructed. There was another building in that space before that got demolished. Behind and to the left is another site being torn down in preparation for more new construction.

As you're pointing out - different use cases will use different approaches.

I agree with you that there is room for automation. Your use case is a perfect example.

I disagree with the people saying that this is going to get rid of all the trained operators or have them all be remote out of the country. There is more than enough construction/demolition like what's going on next to me that will necessitate human operators.

I think the biggest limiting factor is not going to be what the technology is capable of. I think the biggest limiting factor will be legal/bureaucratic. When something goes wrong, who is responsible? Who is liable for damages or injuries?

If I had to put money on what I think will happen: I think government and insurance will require that there be a human operator responsible for every machine. I think that in situations like your's, they will allow a single operator to oversee a fleet of automated machines. In situations like mine, they will require an operator for every active machine, but a single operator might have a remote station that allows them to quickly switch between machines. In both cases, I think they will require the operator to be in the state where the construction is taking place to ensure that they are subject to all the laws and regulations of that state.

The operator won't necessarily have to be manipulating controls to control each movement of their machine. They will often just push a button to run an automated function. But in situations needing a dedicated operator, they will be expected to be there to push the big red "STOP" button at the first sign of trouble.

In reply to Beer Baron :

I agree. 
 

That's mostly covered now with "line of sight" OSHA rules which prohibit operators from leaving a running machine unattended.  Those laws aren't detailed enough to prevent remote operators. 
 

For example... Does "line of sight" mean a human has to physically be standing there, or can a human be watching through a camera?

I think there is a much higher ability to move toward automation for heavy equipment on construction sites than there is for automobiles in the "real" world.  Fewer variables. Fewer people around to get hurt. 

Caterpillar sells to a worldwide market. 
 

This is a chart of the number of earth moving "great" infrastructure projects worldwide underway in 2022. (Projects that exceed $25 million US).

Most of you know how huge the project is that I am working on. There are over 7000 such projects worldwide.
 

I don't think Caterpillar cares a lot about how many jobs they CAN'T do remotely!

Beer Baron said:

Not going to lead to control centers moving overseas. Lag and latency are still a thing. The speed of light is finite.

Ever played an online shooter? You are at a disadvantage playing against people on a server on the other side of America. If you try to play on a server across an OCEAN, routing through multiple nodes and such, it's dang near impossible. Keep in mind that every input has to cross that ocean TWICE before you find out what happened.

Imagine the damage that can be done if there is a 1/10 - 1/4 second of lag between operator and action. I'd have trouble operating a forklift like that, I can't imagine what an excavator would be like.

That could be mitigated just by moving the machine more slowly. An excavator is already moving at a snail's pace compared to a Q3A character or today's Doomslayer. For videoconferencing up to a 1/3rd second lag is considered acceptable, although that's more of a one-way issue.

Or the problem could be further reduced by switching to a less laggy LEO satellite connection like Starlink.

GameboyRMH said:

Beer Baron said:

Not going to lead to control centers moving overseas. Lag and latency are still a thing. The speed of light is finite.

Ever played an online shooter? You are at a disadvantage playing against people on a server on the other side of America. If you try to play on a server across an OCEAN, routing through multiple nodes and such, it's dang near impossible. Keep in mind that every input has to cross that ocean TWICE before you find out what happened.

Imagine the damage that can be done if there is a 1/10 - 1/4 second of lag between operator and action. I'd have trouble operating a forklift like that, I can't imagine what an excavator would be like.

That could be mitigated just by moving the machine more slowly. An excavator is already moving at a snail's pace compared to a Q3A character or today's Doomslayer. For videoconferencing up to a 1/3rd second lag is considered acceptable, although that's more of a one-way issue.

See SVRex's comment about how an extra 15 seconds/load works out to an additional $4,600/day. You'll spend way more in lost time than you will save by offshoring the workforce.

The lag/latency isn't an issue with the machine rolling across the ground. It's the fine control of moving arms and buckets by inches.

In the shooter analogy, it's how difficult it will be to aim a gun if there is a 1/10 second lag between an input and your reticle moving.

Inches matter. 1/3 second can easily be the difference between rupturing a power/water/sewer/gas line or not.

Agreed, latency is an issue overseas.  If round trip through all the servers from operator to machine is 24000 miles and there is ZERO processing time, thats still 0.12 seconds, which is fairly significant.

If you've ever used a mouse with a display with a TV that has some processing latency... its extremely difficult to do.

 

Locally I dont see much of an issue.  Even if the guy operating is on-site in an air conditioned trailer, that means 1 set of controls, etc. per "cluster" of machines you would find on a worksite, and there are huge savings & operating improvements available.  

In reply to Beer Baron :

I agree with you about the lag issue, but the $4600 cost per day would not come close to the savings of offshoring the workforce.

I'm running a crew of 100 men.  At $20 per hour that's $16,000 per day in straight time, plus $12,000 per day in overtime, plus about $6000 in taxes and WC insurance. That's $34,000 per day. 
 

Let assume 70 operators could control the same equipment, but had to slow down by 25% because of the lag. We'd need 88 operators to do the same work per day. If we paid them $12 per hour (I'll bet it would be less) the same 12 hour day would cost $144 per man. No overtime, no WC insurance. Let's assume 10% taxes and costs. So the cost per man day would be $159.  88 operators would cost $13,992.  That's a savings per day of over $20,000, even at a 25% reduction in speed. 
 

The offshore labor rates will be very hard to ignore at some point. 

Thread resurrection: An idea related to this randomly popped into my head that could eliminate the lag issue of outsourcing, by greatly reducing the distance. It is extremely cyberpunk.

So let's say things are at the stage where you have an Indian company doing outsourced equipment operations, but they have to work more slowly because of lag due to the distance between India and the US.

But now let's say this Indian company buys an old cruise ship and packs it to the gills with equipment operator stations and their operators and docks it in the US. The ship gets connected to shore facilities including, say, a massive fiber-optic link, but everyone on the ship stays there, legally they have not entered the US, just as if they were working from India.

Now the company works with low latency as if they're located just a few states over, bringing lag down to levels that could even work for a fast-paced FPS match, but legally it's the same as if they were working from India. They have to pay for this cruise ship but the speed difference can be eliminated.

I imagine the company would have more than one of these ships operating on months-long rotating shifts, and would probably not sail them from India but rather somewhere in the Caribbean that was just a few days away to minimize transit time and costs. Plenty of impoverished countries there would be eager to work out a travel visa arrangement that would make it easy for workers to fly in and board the ships while also benefiting from aircraft landing fees, ship docking & registration fees and local transit costs.

In fact because of the distance involved there might be an advantage to Caribbean or African companies doing this instead, although they'd have to contend with the disadvantage of having a higher cost-of-living than India that requires higher wages.

Also I think I traced back how this idea popped into my head. Started from the default state of sulking about the E36 M3tiness of the present caused in large part by the current job market. Started thinking about things I might like to do. Then I started thinking about being an equipment operator again, because I seem to be pretty good at it from messing around with my uncle's backhoe and who doesn't want to play with Tonkas at full-scale? Then started thinking about potential threats to such a career. Remembered the remote operation option and the lag issue. Started thinking about how that could be overcome or worked around, and here we are.

One thing that's important to learn from my failure to foresee the earliest threats to an IT career is that you don't only have to consider threats from automation, but also centralization, which can happen much sooner.

I don't even think outsourcing that far is needed.  Most of the gain can be had on-site.  Move 1 or 2 drivers into an air conditioned trailer and give them the ability to take control of the vehicles with minimal autonomous capabilities and suddenly they are playing a Co-op RTS and more efficient than a small army of drivers without any of the human support/interface needs on any of those vehicles.

On site can also be highly isolated from all sorts of cloud bullE36 M3. 
 

In reply to prodarwin :

True, but that's not what Caterpillar is selling. 
 

(They are also not selling remote operators in overseas locations... yet)

Potash mines out here in Sask are already moving towards remote operation.

I'd like to see how the feedback controls give you the feeling of having your machine hit a hump in the grass when you're operating on the side of a hill.

I doubt any simulation will provide that sort of pucker factor.

SV reX said:

In reply to prodarwin :

True, but that's not what Caterpillar is selling. 
 

(They are also not selling remote operators in overseas locations... yet)

Nope, but what you show in the original post is a step in this direction.  Adding quick-swap abilities to that would be another (smaller) step.  Enhanced autonomous capabilities would be yet another.

Even what they show on page 1 interests me.  With an array of screens it allows for many more camera-fed viewpoints for the operator, enabling (?) much more precision in some of their activities.  Lots of heavy machinery operations necessitate a spotter.  Imagine if you could see through the eyes of your spotters.  And through the eyes of the excavator arm, from the top of the vehicle, from the load on a crane, etc. 

Plus, one thing that I've always found insane is that heavy machinery uses so many different levers to operate, and each one is different.  Computer interface can make them all the same, using the same controllers.  I'm picturing something like a spacemouse to operate an excavator arm.  Or heck, even a nintendo powerglove like interface.

UI/UX improvements in the world of really complex machinery could potentially be huge advancements.  

SV reX said:

I think the remote operations would wear out people very quickly. The kind of people who operate equipment would die quickly of production fatigue if they were sitting in an office.  They enjoy the outdoors, some level of independence, smoking, drinking, and breaking stuff.  They are not office environment people. 

I would agree with this, but I could also see the productivity aspects of it making payscales somewhat equal - the guy in cab gets his feet muddy and might have to fix things, but he also gets paid for a tremendous amount of idle/downtime, while the woman in the seat at the office across the country has less down time and makes the company more money.

But, alternatively, this could open up options to women, disabled or older folks who may not want or cannot be in the cab.

I see advantages here for mining, road building, anything where you can set up communications equipment for months or years at a time. 

As you've said, there will still be plenty of high fidelity, one off jobs for traditional operators. 

I work for a megacorp that is very risk averse for employees.  It would not be long from the time this tech became available to the time it was mandated for safety and accessibility reasons. 

In reply to prodarwin :

It's already available. 

A few years back I drew cad remotely from a distance of about 300 miles for about a year. My end was connected to copper telephone wires and I have no idea how the other end was hooked up. The lag between moving the cursor with my mouse, waiting for it to reach the remote computer and then return to my screen was time consuming and aggravating. I can't imagine drawing like that long term. My house has since been hooked up to fiber optic cable so I would hope it is much improved now.

Edit: I was using TeamViewer remote. I think. Also, the graphics requirements for AutoCAD isn't all that big, even in 3D.

In reply to VolvoHeretic :

The same problem still exists with remote desktop connections even at today's speeds, although it's probably not nearly as bad and isn't a big issue for most tasks. Often remote desktop software has an option for a "local cursor" which moves instantaneously and shows where the remote cursor will be next, that helps a lot.