If you already know what you want but want to save time writing it

IME, going to ChatGPT for code usually meant losing time, cause I’d go back and forth trying to get a usable snippet and it would just keep refactoring the same slop that didn’t work in its first attempt

They’re pretty reasonable for consensus-based programming prompts as well like “Compare and contrast popular libraries for {use case} in {language}” or “I want to achieve {goal/feature} in {summary of project technologies}, what are some ways I could structure this?”

Of course you still shouldn’t treat any of the output as factual without verifying it. But at least in the former case, I’ve found it more useful than traditional search engines to generate leads to look into, even if I discard some or all of the specific information it asserts

Edit: Which is largely due to traditional search engines getting worse and worse in recent years, sadly

This is the best article I’ve seen yet on the topic. It does mention the “how” in brief, but this analogy really explains the “why” Gonna bookmark this in case I ever need to try to save another friend or family member from drinking the Flavor-Aid

So, they’ve basically accidentally (or intentionally) made Eliza with extra steps (and many orders of magnitude more energy consumption).

I enjoyed reading this, thank you.

I think this is still fundamentally misunderstood widely.

The fact that it’s being sold as artificial intelligence instead of autocomplete doesn’t help.

Or Google and Microsoft trying to sell it as a replacement for search engines.

It’s malicious misinformation all the way down.

Not all AIs, since many AIs (maybe even most) are not LLMs. But for LLMs, you’re right. Minor nitpick.

It’s literally just Markov chains with extra steps

It does remind me of that recent Joe Scott video about the split brain. One part of the brain would do something and the other part of the brain that didn’t get the info because of the split just makes up some semi-plausible answer. It’s like one part of the brain does work at least partially like an LLM.

It’s more like our brain is like a corporation, with a spokesperson, a president and vice president and a number of departments that with semi-independently. Having an LLM is like having only the spokesperson and not the rest of the work force in that building that makes up an AGI.

based on weighted averages of ‘what people are saying’ with a little randomization to spice things up

That is massively oversimplified and not really how neural networks work. Training a neural network is not just calculating averages. It adjusts a very complex network of nodes in such a way that certain input generates certain output. It is entirely possible that during that training process, abstract mechanisms like logic get trained into the system as well, because a good NN can produce meaningful output even on input that is unlike anything it has ever seen before. Arguably that is the case with ChatGPT as well. It has been proven to be able to solve maths/calculating tasks it has never seen before in its training data. Give it a poem that you wrote yourself and have it write an analysis and interpretation - it will do it and it will probably be very good. I really don’t subscribe to this “statistical parrot” narrative that many people seem to believe. Just because it’s not good at the same tasks that humans are good at doesn’t mean it’s not intelligent. Of course it is different from a human brain, so differences in capabilities are to be expected. It has no idea of the physical world, it is not trained to tell truth from lies. Of course it’s not good at these things. That doesn’t mean it’s crap or “not intelligent”. You don’t call a person “not intelligent” just because they’re bad at specific tasks or don’t know some facts. There’s certainly room for improvement with these LLMs, but they’ve only been around in a really usable state for like 2 years or so. Have some patience and in the meantime use it for all the wonderful stuff it’s capable of.

Yes!!! It doesn’t know Trump has been convicted and told me that even when I give it sources, it won’t upload to a central database for privacy reasons. 🤷‍♀️

You can use the RAG tactic to make it more useful. That involves starting with reputable sources as input, which creates an AI character that’s essentially supposed to be an expert in a certain topic.

The normal AI system is a scammer who tries to convince others to act like them… just like me and other internet trolls or crazy people. It needs some snark to act like a real person does, but pure snark is quite useless.

Essentially: nonsense in, nonsense out Or science books and journals in, sci fi speculation out

LLMs don’t “feel”, “know”, or “understand” anything. They spit out statistically most significant answer from it’s data-set, that is all they do.

It’s trained on internet discussions and people on the internet rarely say, “I don’t know”.

they have to provide an answer

Indeed. That’s the G in chatGPT. It stands for generative. It looks at all the previous words and “predicts” the most likely next word. You could see this very clearly with chatGPT-2. It just generated good looking nonsense based on a few words.

Then you have the P in chatGPT, pre-trained. If it happens to have received training data on what you’re asking, that data is shown. It it’s not trained on that data, it just uses what is more likely to appear and generates something that looks good enough for the prompt. It appears to hallucinate, lie, make stuff up.

It’s just how the thing works. There is serious research to fix this and a recent paper claimed to have a solution so the LLM knows it doesn’t know.

I’ve had a similar experience. Except in my case I used lyrics for a really obscure song where I knew the writer. I asked Chat GPT, and it gave me completely the wrong artist. When I corrected it, it apologized profusely and agreed with exactly what I had said. Of course, it didn’t remember that correct answer, because it can’t add to it update its data source.

It all depends on the training data and preprompt. With the right combination of those, it will admit when it doesn’t know an answer most of the time.

The issue is: What is right and what is wrong?

"mondegreen"s are so ubiquitous that there are multiple websites dedicated to it. Is it “wrong” to tell someone that the song where Jimi Hendrix talked about kissing a guy is Purple Haze? And even pointing out where in the song that happens has value.

In general, I would prefer it if all AI Search Engines provided references. Even a top two or three pages. But that gets messy when said reference is telling someone they misunderstood a movie plot or whatever. “The movie where Anthony Hopkins pays Brad Pitt for eternal life using his daughter is Meet Joe Black. Also you completely missed the point of that movie” is a surefired way to make customers incredibly angry because we live in bubbles where everything we do or say (or what influencers do or say and we pretend we agree with…) is reinforced, truth or not.

And while it deeply annoys me when I am trying to figure out how to do something in Gitlab CI or whatever and get complete nonsense based on a single feature proposal from five years ago? That… isn’t much better than asking for help in a message board where people are going to just ignore the prompt and say whatever they Believe.

In a lot of ways, the backlash against the LLMs reminds me a lot of when people get angry at self checkout lines. People have this memory of a time that never was where cashiers were amazingly quick baggers and NEVER had to ask for help to figure out if something was an Anaheim or Poblano pepper or have trouble scanning something or so forth. Same with this idea of when search (for anything non-trivial) was super duper easy and perfect and how everyone always got exactly the answer they wanted when they posted on a message board rather than complete nonsense (if they weren’t outright berated for not searching for a post from ten years ago that is irrelevant).

They seem to give the average answer, not the correct answer. If you can bound your prompt to the range of the correct answer, great

If you can’t bind the prompt it’s worse than useless, it’s misleading.

What? You don’t have a set of cutting hammers in the kitchen?

Well I asked some questions yesterday about classes of DAoC game to help me choose a starter class. It totally failed there attributing skills to wrong class. When poking it with this error it said : you are right, class x don’t do Mezz, it’s the speciality of class Z.

But class Z don’t do Mezz either… I wanted to gain some time. Finally I had to do the job myself because I could not trust anything it said.

Why? It actually answered the question properly, just not to the OP’s satisfaction.

Best use I’ve had for them (data engineer here) is things that don’t have a specific answer. Need a cover letter? Perfect. Script for a presentation? Gets 95% of the work done. I never ask for information since it has no capability to retain a fact.

It struggles to make more than 3 different bedtime stories in a row for my son, and they are always badly written, especially the conclusion that is almost always the same. But at least their sillyness (especially Gemini) is funny.

Yes. I’ve experimented with this too. This is the perfect use case for LLMs - there are no wrong answers, the LLM should just make something up, which is what it does.