GPT Builder Assistant

[TedGarcia]

I have been working with ChatGPT for a few years. My career is in Cloud Computing, I designed (well, my competent team designed) the first GPU infrastructure that ran OpenAI which got me interested in the possibilities.

I built a custom GPT that I call ShopHawk. The GPT is trained on AC 43.13, all available Garmin PDFs for my avionics, all available BearHawk build manuals, and, soon, I am hoping to scrape the forums and have a JSON script keep the GPT in synch with new content. I may do a markdown file of my builder log as well, so that the completed GPT will know how the plane was built.

The latest GPT interface has a voice prompt. I have an app on my phone that allows me to ask "what rivets should I use to join two pieces of 0.032 aluminum," and it references the AC43.13, pulls the info, and even draws a diagram with proper edge spacing and pitch.

The app is in test mode right now. I am doing a 100% cross-check to ensure that GPT does not hallucinate. Once that is done, I will share more.

My goal is to have a POH searchable by voice-activated natural language questions that support all written documentation. And I will likely use it as a maintenance tracker.


Is anyone else leveraging AI for their builds? I am not a coder - would love to connect with folks who are skilled in scripting to help this process along.

-ccc

[AKKen07]

That’s amazing Ted ! Could other builders assist by providing our own build logs or the like for data? I’m no computer whiz but I have a build log in excel.

[Westward_Flyer]

Very interesting, as I plan my build I’ve noticed that there are many many good sources of information but no single complete source. However, if we were able to gather all the excellent information from those many sources and the content in this forum (and maybe Facebook group) and synthesize it accurately that would be a tremendous asset to the community, and ultimately, I believe make it safer. Very much looking forward to how this goes, will be starting my build in about three years.

[jaredyates]

My answer is, I haven't found it to be good enough yet.
I have been dabbling with a similar project but have not yet been able to get it to work work enough yet to make it a forum or Beartracks archive feature. If you are Home Depot or Target or someone of that size, you can invest in a system and spend the ongoing money to keep it going. For something the size of our community, we are too small to afford the pricing plans. And setting aside the financial cost, I'm worried about quality. Even with very specific instructions, in my daily personal use, LLMs are routinely disregarding my sourcing instructions and providing bad info. Our consequences are high, and not everybody is ready to invest in learning what it takes to use an LLM as a thought partner and not a thought leader.
I'm not saying it can't be done, but rather, the current level of performance, for the tools available to me, are not ready.

[Bcone1381]

My answer is, I haven't found it to be good enough yet.
I have been dabbling with a similar project but have not yet been able to get it to work work enough yet to make it a forum or Beartracks archive feature. If you are Home Depot or Target or someone of that size, you can invest in a system and spend the ongoing money to keep it going. For something the size of our community, we are too small to afford the pricing plans. And setting aside the financial cost, I'm worried about quality. Even with very specific instructions, in my daily personal use, LLMs are routinely disregarding my sourcing instructions and providing bad info. Our consequences are high, and not everybody is ready to invest in learning what it takes to use an LLM as a thought partner and not a thought leader.
I'm not saying it can't be done, but rather, the current level of performance, for the tools available to me, are not ready.

I'm curious if Ted's ShopHawk has skills and capabilities that exceeded your LLM's ability. That would destroy numerous common barriers both starting the project and to the Bearhawk completion rate.

[jaredyates]

Yes for sure, my experience relates largely to my lack of expertise. Everything I know about how to do it came from me asking the model how to do it. Information management is a huge part of the build and with the right human mindset, LLMs are the most promising solution to my years-long quest to make Bearhawk information easier to find. It's just that with the tools I have, I couldn't make it good enough yet.

[gregc]

If playing with machine learning is fun for you, go for it. I would disagree however with the notion that it has any real role in experimental aircraft construction. Despite the extraordinary (and often false) claims made by vendors, LLMs cannot reason and simply generate convincing mimicry. Because of that, they can never be completely accurate. Even if they were, I don´t see the point. Let's imagine I have the digital equivalent of Bob Barrows, Colin Campbell and an FAA regulatory lawyer in my phone. I no longer any need to have to think, investigate, research, struggle, learn, create something new, make and correct mistakes, etc. I just ask the expert. It seems to me that I have removed the single most valuable part of building an airplane. This is a case grasshopper, where it really is the journey, not the destination, that matters.

[jaredyates]

If playing with machine learning is fun for you, go for it. I would disagree however with the notion that it has any real role in experimental aircraft construction. Despite the extraordinary (and often false) claims made by vendors, LLMs cannot reason and simply generate convincing mimicry. Because of that, they can never be completely accurate. Even if they were, I don´t see the point. Let's imagine I have the digital equivalent of Bob Barrows, Colin Campbell and an FAA regulatory lawyer in my phone. I no longer any need to have to think, investigate, research, struggle, learn, create something new, make and correct mistakes, etc. I just ask the expert. It seems to me that I have removed the single most valuable part of building an airplane. This is a case grasshopper, where it really is the journey, not the destination, that matters.

This is a very interesting philosophical question, and it spans all aspects of the build. You have some folks who build more for the journey and some who build more for the destination, and everything in between. We see this manifest in kit vs scratch building, and a hundred other ways.

There was an audio interview I heard recently with Temple Grandin that was mostly about how much variety there can be in the functioning of our various brains. Her example was that some folks who could not pass an algebra class can be highly effective makers and designers of things. But some folks need an algebra class in order to learn how to think. She breaks it up into verbal and visual thinking, and was promoting her book which I'm sure goes into much more detail, but here is the audio I was listening to:

https://youarenotsosmart.com/2022/12/12/yanss-249-temple-grandins-vision-for-a-world-better-suited-to-different-styles-of-thinking/

Before hearing that discussion, I had not really though of how much the world and education system can end up being set up to favor the type of learners who match the type of the ones doing the setting up. If we don't allow for diversity of learning types, we miss out on the works of folks who get screened out unfairly by the system.

This also applies to the forum in the sense that certain highly productive builders might not get the kind of benefit from forum participation that I'd like to think I do. This could very well just be because our brains work differently.

In the past, I had always just assumed that some folks didn't want to allocate the time to a word-based discussion about a very tangible process, but there might be more to it than that.

It is unfortunate when it occasionally happens that someone makes a mistake that we've already learned about before. How can we shape the Bearhawk corner of the internet to cast as wide of a welcome as possible, yet also make the good stuff that is known, by at least someone, also available to anyone, when they need it? AI poses some real opportunities in this ongoing challenge.

[fairchild1934]

I think all you have to do is listen to AI on any subject that you all ready know a lot about---- it just rambles in circles and as often as not says completely
incorrect crap. I fail to understand why we have been brainwashed into believing that a sack full of hard drives and IO interfaces is better than a text book.
And for FYI--- I dont believe a 1000$ smart phone is better than a 1960's rotary phone. so how about that.......

[SpruceForest]

Much of the discussion on enthusiast fora is regurgitation of known fact or community sourced best practice - both of which seem near ideal fits for the current SOTA for AI, which ranges from dimwitted moron to idiot-semisavant. As with other examples of the death of expertise and democratization of knowledge creation and management, your mileage may vary in terms of usefulness.

Re: rotary phone versus smart phone... if you have a rotary phone handy, try transitioning back to that and wood pulp-based media for a week. I know a dead tech place than can even get you one of those little pig-tail car phone antennas for your AMC Pacer or Datsun pickup if you want to go newer-school... Here - let me drop a pin for you... oh... never mind.

[819f15h]

I am not a coder - would love to connect with folks who are skilled in scripting to help this process along.

Perhaps something like Replit could help? You don't need a lick of coding experience; you just prompt the agent w/ what you want, and then continue to give feedback to refine the software it writes until you are happy with it.

[gregc]

How can we shape the Bearhawk corner of the internet to cast as wide of a welcome as possible, yet also make the good stuff that is known, by at least someone, also available to anyone, when they need it? AI poses some real opportunities in this ongoing challenge.

Thinking about the goals for the forum going forward, this seems like an important one.

[Frank]

"Open the pod bay doors, HAL!"
But seriously, an expansive AI explanatory index of all available Bearhawk-related information is a laudable enterprise.
Can AI go in and pick Bob's brain for us?

[OhioLSA]

[QUOTE=fairchild1934;n97225]I think all you have to do is listen to AI on any subject that you all ready know a lot about

I agree. Any time I read AI-generated material on a subject that I already know pretty well, the errors are plentiful and immediately obvious. This makes me really reluctant to trust it when I'm delving into a subject that I don't know much about. Maybe it has its place, and probably it will improve to the point that it's really reliable, but I don't think it's there yet.