Why is ChatGPT output believable?

[Samuel Tai]

Recently, I've seen forum members ask ChatGPT questions on "How to do X in TrueNAS?" Why would you expect to get a useful and correct answer from a clever bot that seems to "intelligently" cut & paste text from across the web together into a "reasonably" lucid pastiche? This is, at bottom, just another case of GIGO.

 

[Jailer]

ChatGPT terrifies me. Seriously.

 

[Whattteva]

Honestly, after seeing some of the responses. I would tend to trust ChatGPT more than random forum/reddit posters to be honest with you. Some people are highly misinformed and fight for it tooth and nail. I had a guy recently call my arguments "meaningless" while providing nothing himself. When I finally referenced first-party sources, he shut up and deleted all his past Reddit posts in the thread and disappeared. But, he was so convinced that he was right up until that point.

 

[John Doe]

we can see that trend all over the various forums.
had an issue with my nextcloud sending every 5 mins an email.

raised this in the forum and some other user ChatGPT'd the answer. it would have fixed the issue by limiting cron jobs execution to once a day.
but that comes with other issues.

however i think if you feed it properly the output is not too bad.

 

[Samuel Tai]

Thing is, ChatGPT will make shit up, but make it sound convincing. See https://insidemedicine.substack.com...ting?r=5p3cr&utm_campaign=post&utm_medium=web. When pressed for a clinical literature reference defending its response to a question, it merrily cut & pasted together random words into the form of a reference, but no such clinical paper existed.

We already have enough problems with forum members blindly following Youtube videos, Reddit postings, and God knows what else, to try to solve a problem without understanding their issue, and blowing up their systems or data as a result. Now ChatGPT will eventually recommend they run zfs destroy to fix a minor permissions issue.

 

[Whattteva]

Thing is, ChatGPT will make shit up, but make it sound convincing. See https://insidemedicine.substack.com...ting?r=5p3cr&utm_campaign=post&utm_medium=web. When pressed for a clinical literature reference defending its response to a question, it merrily cut & pasted together random words into the form of a reference, but no such clinical paper existed.

Yes, but who in their right mind would make any kind of academic paper/research referencing ChatGPT? I think of its use more like Wikipedia for casual users but in a different format.

 

[Jailer]

however i think if you feed it properly the output is not too bad.

Thing is, ChatGPT will make shit up, but make it sound convincing.

I'm sorry, my responses are limited. You must ask the right questions.

 

[sretalla]

I guess people ask the forum because it is resourced with people who have "done the stuff" or "read the docs" and they don't want to need to do that to get their answer.

Since ChatGPT has "read the docs", it may be an interesting source of shortcut to an answer...

Unfortunately, "read the docs" and "understood the docs" isn't the same thing, so ChatGPT will limit it's context to the requested topic, but it will use many sources and apply a machine learning algorithm to figure out which word usually follows each subsequent word in a sentence... hence the "made up reference to a non-existent paper", which was simply taking examples of how many different people said things similar to that in their papers, so it made a best guess as to how a reference should look based on the algorithm using all the references it had seen... no understanding, just algorithm.

It can be a very useful resource, but just like the Internet before it (and the Internet is one of its sources), it's full of crap (at least in part).

 

[Ericloewe]

I mean, poor judgement is as common as humans are, no big surprise there.

What I want to know is what sort of infrastructure was backing this monstrosity - and who the hell was footing the bill until Microsoft figured they should swoop in and buy themselves street cred.

Seriously, this thing tries to answer a crazy range of topics and does so with far more credibility than you can get from a tiny deployment in some random cloud. There's some serious hardware backing it, but it flew under the radar and popped up without huge fanfare from x, y and z backers. So what's the agenda? Just a mad gambit to sell to a big company?

 

[Dice]

I'm sorry, my responses are limited. You must ask the right questions.

This brings me right back to a colorful memory from my early days on the forums;

You need to change your problem so that lines up with my solution.

( https://www.truenas.com/community/threads/incoming-files-storage-configuration.40756/post-258231 )

 

[Redcoat]

This brings me right back to a colorful memory from my early days on the forums;

I clearly remember that one, too. Put @anodos on my watch-list...