Heads should roll at Boeing

This is the polite off topic forum. If you’re looking to talk smack and spew nonsense, keep moving along.

Moderators: mgil, chromoly

Post Reply
User avatar
Hanley
Strength Nerd
Posts: 8777
Joined: Fri Sep 15, 2017 6:35 pm
Age: 46

Re: Heads should roll at Boeing

#81

Post by Hanley » Tue Apr 23, 2019 8:01 am

Mattjd wrote: Tue Apr 23, 2019 7:47 am
Hanley wrote: Tue Apr 23, 2019 7:45 am
Mattjd wrote: Tue Apr 23, 2019 7:13 ampeople that havent the slightest bit of knowledge in that domain.
Do you really think a math PhD and embedded programmer with many years of experience are clueless about control theory?

What's your real-world experience with integrated systems?
I'm talking specifically to you

I'm not getting the impression that anyone but you is horrified that someone would apply a software solution to this
Well...then talk to me and not "people".

Yeah, I haven't taken a control theory class.

My perspective is that of a user/operator who was part of a very, very big team who was responsible for designing weapons systems for security boats. From my perspective, you are completely missing my "global" concern of the enormous complexity of the design process and bad design. From this perspective I totally sympathize with the point of the article (ie sloppy mechanical engineering and fixes from software dudes with little domain knowledge that results in a clusterfuck of a non-intuitive experience for the end-user/operator).
Last edited by Hanley on Tue Apr 23, 2019 8:14 am, edited 1 time in total.

JonA
Registered User
Posts: 2138
Joined: Fri Sep 29, 2017 7:00 am
Age: 48

Re: Heads should roll at Boeing

#82

Post by JonA » Tue Apr 23, 2019 8:02 am

Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?

User avatar
Savs
Dream Weaver
Posts: 1210
Joined: Sat Sep 16, 2017 2:50 pm
Age: 60

Re: Heads should roll at Boeing

#83

Post by Savs » Tue Apr 23, 2019 8:08 am

Mattjd wrote: Tue Apr 23, 2019 7:47 am I'm not getting the impression that anyone but you is horrified that someone would apply a software solution to this
Please allow me to introduce myself. I'm a man of wealth and taste.

Please add me to the list of the horrified.

User avatar
Root
Grillmaster
Posts: 1997
Joined: Fri Sep 15, 2017 8:28 am
Location: Western Upper Lower
Age: 44

Re: Heads should roll at Boeing

#84

Post by Root » Tue Apr 23, 2019 8:10 am

JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?

User avatar
Hanley
Strength Nerd
Posts: 8777
Joined: Fri Sep 15, 2017 6:35 pm
Age: 46

Re: Heads should roll at Boeing

#85

Post by Hanley » Tue Apr 23, 2019 8:11 am

Mattjd wrote: Tue Apr 23, 2019 7:58 amAll I'm saying is that if you are so shocked they fixed this with a software solution
Are you talking to me again? 'Cause I'm pretty sure the embedded programmer has a sense of the scope/scale of software solutions in the real-world.

You're welcome to go back and read everything I've posted. I'm not shocked. Not horrified.

User avatar
Les
Kitten
Posts: 1037
Joined: Mon Sep 18, 2017 6:23 am
Location: West Bend, WI
Age: 45

Re: Heads should roll at Boeing

#86

Post by Les » Tue Apr 23, 2019 8:12 am

Root wrote: Tue Apr 23, 2019 7:49 am
Mattjd wrote: Tue Apr 23, 2019 7:13 am I have a feeling my points appear non sequitur bc I'm trying to explain myself (and beknowest to me very poorly) in technical terms to people that havent the slightest bit of knowledge in that domain.

John, is your issue with Boeing the fact that they used a software solution to fix a structural one?
I just designed and implemented a collision avoidance system for an unmanned drone. Not just the controller. The system. That's my side gig. And your "points" are non sequitur.

No one is saying that the MACS system, or any controller in general, is incapable of fixing this problem. You don't need to have taken a class in control theory to know that. Also, the software wasn't buggy* or poorly implemented. It was extremely poorly designed. This is an important distinction.

The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk. Everything comes with risk in real engineering. The most thouroughly tested software can still have bugs. Redundant sensors can still fail. CAN busses can fail. Cables can break. Etc. So when you try to fix a fundamental problem by adding complexity, you are adding points of failure, and that is almost always bad engineering practice. In this case, they made bad engineering decisions to save money. And then whoever designed the avionics/sensor/controller/software really screwed the fucking pooch. So, the point of failure that they added failed and killed a lot of people. If they had designed the MACS system correctly, this might not have happened. But that doesn't make it a good idea.

*Unless there is evidence for that "they tried to turn it off, but it re-enabled itself" statement
This is a great point. Adding complexity can have it's drawbacks. I had a total electrical and battery failure once in a Cessna 172. Thankfully it was an older model, so a lot of the instruments still work, and the engine stays running. Had it been something like the DA-42, I would have been SOL.

JonA
Registered User
Posts: 2138
Joined: Fri Sep 29, 2017 7:00 am
Age: 48

Re: Heads should roll at Boeing

#87

Post by JonA » Tue Apr 23, 2019 8:13 am

Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.

User avatar
Hanley
Strength Nerd
Posts: 8777
Joined: Fri Sep 15, 2017 6:35 pm
Age: 46

Re: Heads should roll at Boeing

#88

Post by Hanley » Tue Apr 23, 2019 8:16 am

JonA wrote: Tue Apr 23, 2019 8:13 am Because it's an example of fixing a fundamental mechanical problem with software.
Satan. Doing his work.

User avatar
damufunman
Registered User
Posts: 2974
Joined: Tue Sep 19, 2017 6:14 pm
Age: 36

Re: Heads should roll at Boeing

#89

Post by damufunman » Tue Apr 23, 2019 8:17 am

Mattjd wrote: Mon Apr 22, 2019 7:39 am I'm not an aerospace engineer, I dont see a problem with utilizing software to fix another issue. The entire field of digital signal processing it devoted that. Instead of making electronic controllers and filters out of operational amplifiers (which require a circuit designer, manufacturing etc. and redesigning if there is an issue) you can instead digitally sample the signal and apply DSP techniques to get the same results. It works. To me, what they did seems reasonable. By what they did of course I mean fixing a hardware (structural in this case) with software. THIS LITERALLY SOUNDS JUST LIKE APPLYING A DIGITAL CONTROLLER. They have problems with the aircraft responding a certain way due to its build so they made a controller to offset that. That is literally the entire premise of control theory. You have a system (the plane) you want a desired response, it doesn't do that, so you throw on a controller in front, which receives the error (desired vs actual... i.e. the sensor in this case) to precondition the signal such that the output is what you want. Makes sense, to me.

The bad part is how they had 1 sensor (wut?), informed the customer that no new training was needed (this is arguably my biggest issue), buggy software that couldn't be manually overridden (people aren't perfect, but I would bet it was due to corporate wanting to get the shit out fast and being loose on the testing). Actually my biggest issue is how they forced this shit out for use due to the competing company having a new plane and corporate NEEDING something new and fresh to keep up.
Didn't see the next 3 pages yet, so maybe addressed below (above now...), but an unstable system, in a safety-critical function seems to be a terrible idea, control system notwithstanding.

Ah I see
Savs wrote: Tue Apr 23, 2019 8:08 am
Mattjd wrote: Tue Apr 23, 2019 7:47 am I'm not getting the impression that anyone but you is horrified that someone would apply a software solution to this
Please allow me to introduce myself. I'm a man of wealth and taste.

Please add me to the list of the horrified.

User avatar
Root
Grillmaster
Posts: 1997
Joined: Fri Sep 15, 2017 8:28 am
Location: Western Upper Lower
Age: 44

Re: Heads should roll at Boeing

#90

Post by Root » Tue Apr 23, 2019 8:18 am

JonA wrote: Tue Apr 23, 2019 8:13 am
Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.
It is.

If there was a way to fix it without adding complexity, that probably would have been a better solution. But I don't think there's a (ETA: "less complex") way to make 4 dumb electric motors level themselves without a controller of some kind.
Last edited by Root on Tue Apr 23, 2019 8:21 am, edited 1 time in total.

User avatar
Mattjd
Registered User
Posts: 860
Joined: Mon Sep 18, 2017 8:52 pm
Age: 32

Re: Heads should roll at Boeing

#91

Post by Mattjd » Tue Apr 23, 2019 8:20 am

Why is my shit not fucking posting

User avatar
Les
Kitten
Posts: 1037
Joined: Mon Sep 18, 2017 6:23 am
Location: West Bend, WI
Age: 45

Re: Heads should roll at Boeing

#92

Post by Les » Tue Apr 23, 2019 8:20 am

JonA wrote: Tue Apr 23, 2019 8:13 am
Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.
Since it's a drone, you will have to have some kind of programming I would imagine. But the question would be, does the computer set RPM for each rotor, or does the computer regulate the pitch angle on the rotor? In a human environment, you set the RPM (this is airplanes, I don't fly helos), and a mechanical system changes the pitch angle to maintain RPM. So if the drone is doing the job of a human by setting RPM, I guess you would need that programming. But having a program control the pitch when I mechanical system can already do that job might be an example of unnecessary programming? Again, I don't know drones, but I was thinking of an example of a fixed wing drone. The same concept may apply.

User avatar
Mattjd
Registered User
Posts: 860
Joined: Mon Sep 18, 2017 8:52 pm
Age: 32

Re: Heads should roll at Boeing

#93

Post by Mattjd » Tue Apr 23, 2019 8:22 am

https://www.google.com/amp/s/www.nytime ... s.amp.html

Interpret that as you will.

To my understanding the entire tragedy started with a faulty sensor. The pilots disabled mcas, as well as some other systems but for whatever reason (software or just physics) they couldn't level the plane and started back up some of the electronics unfortunately the that started back up the mcas with a faulty sensor.

Edit : reading it again it appears the system overcompensated that manual recovery was difficult if not impossible. I don't know what to think, fucked system, faulty sensor, all the above.

JonA
Registered User
Posts: 2138
Joined: Fri Sep 29, 2017 7:00 am
Age: 48

Re: Heads should roll at Boeing

#94

Post by JonA » Tue Apr 23, 2019 8:26 am

Hanley wrote: Tue Apr 23, 2019 8:16 am
JonA wrote: Tue Apr 23, 2019 8:13 am Because it's an example of fixing a fundamental mechanical problem with software.
Satan. Doing his work.
Now you're just jealous because you didn't think of adding any remote exploits to the control software in those security gunboats.

User avatar
damufunman
Registered User
Posts: 2974
Joined: Tue Sep 19, 2017 6:14 pm
Age: 36

Re: Heads should roll at Boeing

#95

Post by damufunman » Tue Apr 23, 2019 8:27 am

JonA wrote: Tue Apr 23, 2019 8:13 am
Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.
These drones are typically are not carrying passengers. Other than with pretty extreme requirements (fighter, acrobatics aircraft) where instability is helpful for the purpose of these planes, I suspect most would agree that stability and some propensity to move AWAY from dangerous conditions passively is a helpful feature for safety.

User avatar
Les
Kitten
Posts: 1037
Joined: Mon Sep 18, 2017 6:23 am
Location: West Bend, WI
Age: 45

Re: Heads should roll at Boeing

#96

Post by Les » Tue Apr 23, 2019 8:28 am

Oh, and if you guys wanted to see something truly shocking, you should have been at FlightSafety LaGuardia watching me ninja crawl/waddle into the Beech 1900 D flight sim. :-)

User avatar
Mattjd
Registered User
Posts: 860
Joined: Mon Sep 18, 2017 8:52 pm
Age: 32

Re: Heads should roll at Boeing

#97

Post by Mattjd » Tue Apr 23, 2019 8:29 am

Les wrote: Tue Apr 23, 2019 8:20 am
JonA wrote: Tue Apr 23, 2019 8:13 am
Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.
Since it's a drone, you will have to have some kind of programming I would imagine. But the question would be, does the computer set RPM for each rotor, or does the computer regulate the pitch angle on the rotor? In a human environment, you set the RPM (this is airplanes, I don't fly helos), and a mechanical system changes the pitch angle to maintain RPM. So if the drone is doing the job of a human by setting RPM, I guess you would need that programming. But having a program control the pitch when I mechanical system can already do that job might be an example of unnecessary programming? Again, I don't know drones, but I was thinking of an example of a fixed wing drone. The same concept may apply.
The controller will adjust the rpm of all the motors to get the desired movement. The flight controller will take data from a handful of sensors plus the preprogrammed flight plan and send the commands to the electronic speed controllers (whose P I D you have to manually tune per craft but is set completely with software)

User avatar
Les
Kitten
Posts: 1037
Joined: Mon Sep 18, 2017 6:23 am
Location: West Bend, WI
Age: 45

Re: Heads should roll at Boeing

#98

Post by Les » Tue Apr 23, 2019 8:30 am

damufunman wrote: Tue Apr 23, 2019 8:27 am
JonA wrote: Tue Apr 23, 2019 8:13 am
Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.
These drones are typically are not carrying passengers. Other than with pretty extreme requirements (fighter, acrobatics aircraft) where instability is helpful for the purpose of these planes, I suspect most would agree that stability and some propensity to move AWAY from dangerous conditions passively is a helpful feature for safety.
AFT CG flight characteristics are also appealing to some aircraft manufacturers for higher airspeeds/better fuel efficiency. See my MD11 post in this thread. I agree that there are trade-offs and maybe it isn't the best idea.

JonA
Registered User
Posts: 2138
Joined: Fri Sep 29, 2017 7:00 am
Age: 48

Re: Heads should roll at Boeing

#99

Post by JonA » Tue Apr 23, 2019 8:33 am

Les wrote: Tue Apr 23, 2019 8:20 am Since it's a drone, you will have to have some kind of programming I would imagine. But the question would be, does the computer set RPM for each rotor, or does the computer regulate the pitch angle on the rotor? In a human environment, you set the RPM (this is airplanes, I don't fly helos), and a mechanical system changes the pitch angle to maintain RPM. So if the drone is doing the job of a human by setting RPM, I guess you would need that programming. But having a program control the pitch when I mechanical system can already do that job might be an example of unnecessary programming? Again, I don't know drones, but I was thinking of an example of a fixed wing drone. The same concept may apply.
I don't know about the more expensive ones, but the cheap ones are completely controlled by engine speed. You can see this in a sharp bank, you can actually see one rotor completely stop turning.

User avatar
damufunman
Registered User
Posts: 2974
Joined: Tue Sep 19, 2017 6:14 pm
Age: 36

Re: Heads should roll at Boeing

#100

Post by damufunman » Tue Apr 23, 2019 8:36 am

Les wrote: Tue Apr 23, 2019 8:20 am
JonA wrote: Tue Apr 23, 2019 8:13 am
Root wrote: Tue Apr 23, 2019 8:10 am
JonA wrote: Tue Apr 23, 2019 8:02 am
Root wrote: Tue Apr 23, 2019 7:49 am The problem with fixing a fundamental mechanical problem with software is that you are adding potential points of failure, and therefore adding risk.
How mechanically stable is a multi rotor drone without the software controlling engine speeds?
I don't know, probably not very stable. Why do you ask?
Because it's an example of fixing a fundamental mechanical problem with software.
Since it's a drone, you will have to have some kind of programming I would imagine. But the question would be, does the computer set RPM for each rotor, or does the computer regulate the pitch angle on the rotor? In a human environment, you set the RPM (this is airplanes, I don't fly helos), and a mechanical system changes the pitch angle to maintain RPM. So if the drone is doing the job of a human by setting RPM, I guess you would need that programming. But having a program control the pitch when I mechanical system can already do that job might be an example of unnecessary programming? Again, I don't know drones, but I was thinking of an example of a fixed wing drone. The same concept may apply.
Yeah, to echo JonA, I would guess for cost reasons they use fixed blades, and electric motors that size can change speed quickly enough it shouldn't be a problem at all to regulate speed pretty closely.

Post Reply