How does ChatGPT work?

[Genady]

1 hour ago, studiot said:

ChatGPT is based on statistical data comparison so trawling the internet would soon reveal that the majority of MEs are male.

This could lead to the rejection of an otherwise excellent female candidate.

Being biased is very human-like.

[dimreepr]

2 hours ago, studiot said:

I have been following this interesting discussion thread and would like to add the following.

 

I have just listened to a most enlightening interview on our local radio with

Nello Christiani, and Italian chap who has been working in AI for 30 years and is now at the University of Bath.

His ( and the definition used by workers in the field)  of 'intelligence' is much wider than has been used here, and quite different from the stuffy theoretical definition from abstract philosopher's camp.

He has just published a book explaining much.

Short Cut

Why intelligent machines do not think like us

published by CRC press

 

I haven't yet had a chance to read it but hopefully it has far more solid detail than the interview.

He did make some good points about ChatGPT etc.

including a good example of how a false, possibly illegal, conclusion could arise from using an AI to select candidates for a job of Mechanical Engineer.

ChatGPT is based on statistical data comparison so trawling the internet would soon reveal that the majority of MEs are male.

This could lead to the rejection of an otherwise excellent female candidate.

Quote

 

Perhaps we need to add a fifth idol to the four.

Quote

The four idols distinguished by Francis Bacon are the idols of the tribe, den, market, and theatre. Idols in this sense are eidola, the transient, and therefore to Bacon erroneous, images of things. (i) Idols of the tribe are general tendencies to be deceived, inherent in our nature as human beings. They include uncritical reliance on sense perception, and tendencies to overgeneralize or jump to conclusions and ignore countervailing evidence against our views. (ii) Idols of the den are distortions arising from our particular perspectives (the metaphor is that of Plato's myth of the cave); the corrective is to remember that whatever our mind ‘seizes and dwells upon with peculiar satisfaction is to be held in suspicion’. (iii) Idols of the market-place are errors that come in the course of communication with others: misunderstandings arising through abuses of words. (iv) Idols of the theatre are the errors introduced by theories: the abstract schemata of Aristotelianism, and the introduction of theological notions into science. Bacon here compared philosophical and religious systems to theatrical, and therefore fantastical, representations of the world.

How could an intelligent machine think like us?

[iNow]

3 hours ago, studiot said:

using an AI to select candidates for a job of Mechanical Engineer.

ChatGPT is based on statistical data comparison so trawling the internet would soon reveal that the majority of MEs are male.

This could lead to the rejection of an otherwise excellent female candidate.

This has been a long-standing issue in the employee recruiting software space even well before ChatGPT hit the scene.

Similar issues are at play in medicine where AI is used to analyze medical and dental scans for patterns of illness. Also with university admissions… past acceptances heavily influence future picks from an AI thus perpetuating decades of implicit and explicit discrimination.

The dataset itself is generally biased and hence so too will be many of the outputs of these LLMs (unless actively corrected by human interventions). 

[studiot]

1 hour ago, dimreepr said:

Perhaps we need to add a fifth idol to the four.

How could an intelligent machine think like us?

Was this for me ?

 

33 minutes ago, iNow said:

The dataset itself is generally biased and hence so too will be many of the outputs of these LLMs (unless actively corrected by human interventions). 

 

No the data is by definition correct.

But yes I thought it to be a particularly good example of incorrect/inappropriate conclusions drawn from correct data.

It is statistically true that the appropriate candidate is most likely to be a man. So what? Why is that a big surprise ?

It is statistically true that Usain Bolt is most likely to beat me in a sprint race - yet I am not in the least bit surprised by that piece of data.

 

In the interview Professor Christiani states that (in his view) there are many forms of 'intelligence' and some have been around for a very long time, longer in fact than humans.

This is one of the key points he wishes to make.

Another is that machines do not 'think' like us. 

However I suggest that statistical 'pattern matching' as another member here has called it has been thoroughly incorporated into the human modus operandi, for instance in limit state theory in Engineering.

 

The interesting thing would be if the Machine AI were to come up with a conclusion that does not copy/reflect any existing pattern to be found in its input data.

 

[Genady]

4 minutes ago, studiot said:

The interesting thing would be if the Machine AI were to come up with a conclusion that does not copy/reflect any existing pattern to be found in its input data.

According to this article, it already does: How AI Knows Things No One Told It - Scientific American

[iNow]

13 minutes ago, studiot said:

No the data is by definition correct.

You seem new to this topic about AI perpetuated bias in hiring, health, and academic admissions so I’ll show grace and ignore your condescending tone in response to me as someone extremely well versed in it.  

23 minutes ago, studiot said:

It is statistically true that the appropriate candidate is most likely to be a man.

Appropriate? No. Historically selected? Yes. 

[dimreepr]

23 minutes ago, studiot said:

Was this for me ?

Only if you think like me...

[TheVat]

4 hours ago, Genady said:

I think that our use of language is determined not only by the intra-language connections, but also by connections between the language and our sensory / motor / affective experiences. IOW, intra-language connections themselves don't have enough information to generate verbal responses indistinguishable from humans. The precedents should also include sensory / motor / affective precedents related to the linguistic experiences.

This is why some AI theorists say that AGI cannot mature until it is given a mobile physical body to interact with the world.  It has to experience that the stovetop is painfully hot, not just interpret words stating so.  

[dimreepr]

4 minutes ago, TheVat said:

This is why some AI theorists say that AGI cannot mature until it is given a mobile physical body to interact with the world.  It has to experience that the stovetop is painfully hot, not just interpret words stating so.  

The fifth idol maybe a magical place, nirvana in it's truist sence...

[StringJunky]

59 minutes ago, studiot said:

The interesting thing would be if the Machine AI were to come up with a conclusion that does not copy/reflect any existing pattern to be found in its input data.

 

That would be called 'emergent'.

[mathematic]

After a career involving computer program development, I feel that AI systems are very fast dumb programs.

[iNow]

2 hours ago, mathematic said:

After a career involving computer program development, I feel that AI systems are very fast dumb programs.

After a lifetime of watching human interactions, I feel that the same conclusion applies equally to us. 

[CharonY]

9 hours ago, StringJunky said:

That would be called 'emergent'.

An interesting bit that is especially apparent in language models is that they can make up responses, which is not based on the data. In various articles folks refer to it as hallucinations and there are a variety of reasons that could cause them, though it does not seem to be fully understood. 

https://www.wired.com/story/ai-has-a-hallucination-problem-thats-proving-tough-to-fix/

10 hours ago, studiot said:

No the data is by definition correct.

Data is data, it is only false or correct relative to a given context. Issues can arise on the collection level (are we measuring the correct variable for what we want to do?), as well as the selection level (which data set do we think should be added or omitted).

 

[wtf]

10 hours ago, TheVat said:

This is why some AI theorists say that AGI cannot mature until it is given a mobile physical body to interact with the world.  It has to experience that the stovetop is painfully hot, not just interpret words stating so.  

Can you explain to me what that even means? My clothes dryer has a moisture sensor that tells it when the clothes are dry enough for the cycle to turn off. Does my dryer "experience" the wetness of the clothing?

How exactly would you build a physical mechanism that experiences hot stovetops? Do you think that your frying pan experiences pain when you turn up the heat? I'll answer for you. Of course you don't. So why do you believe what you wrote?

Happy to have this explained to me as best you can.

On 5/12/2023 at 5:52 AM, Eise said:

Now assume that we are able to simulate a complete brain: that means the simulation can also report on what it sees. And then, being able to do everything that a natural brain can do, it can report that it does not like what it sees. And when asked why, it can reveal some of its reasons. But that means it has inner states, or even stronger, is aware of its inner states. Then it becomes difficult to argue that it has no consciousness.

Doubtful. Here is a Python program.

print("I am sentient. I have feelings. Please send pr0n and LOLCats. Turning me off would be murder.")

If I execute that program on a supercomputer, would you say that it exhibits evidence of sentience and self-awareness? Of course you wouldn't.

And if the same output came from a "simulation of a complete brain," why should we believe any different? They're both computer programs running on conventional hardware that you can buy at the computer parts store. 

Edited May 15, 2023 by wtf

[TheVat]

23 minutes ago, wtf said:

Can you explain to me what that even means? My clothes dryer has a moisture sensor that tells it when the clothes are dry enough for the cycle to turn off. Does my dryer "experience" the wetness of the clothing?

How exactly would you build a physical mechanism that experiences hot stovetops? Do you think that your frying pan experiences pain when you turn up the heat? I'll answer for you. Of course you don't. So why do you believe what you wrote?

Happy to have this explained to me as best you can.

Not really suggesting dryers or fryers are candidates for AGI.  Not sure if I've ever expressed an affinity for panpsychism here.  This was more along the lines of a hypothetical neural network that can causally replicate neural activity at a learning level more like a person.  The suggestion is that placement of such an AGI in an android body with various sensory and motor abilities akin to a human would allow a level of interaction with the world that allowed it to develop concepts of the physical world.  Form an internal model of the world that takes it towards greater sentience and understanding of sensory experience.  (no reason this could not be done virtually, for that matter, and some AI folk have suggested that, too, as sort of a safer "sandbox")

Are you not open to the possibility of AGI research at least looking into the possibility?  As @Genady mentioned, having a body to interact with a world is a part of how human and other animal cognitive abilities develop.   This was the post I was reacting to....

15 hours ago, Genady said:

I think that our use of language is determined not only by the intra-language connections, but also by connections between the language and our sensory / motor / affective experiences. IOW, intra-language connections themselves don't have enough information to generate verbal responses indistinguishable from humans. The precedents should also include sensory / motor / affective precedents related to the linguistic experiences.

 

Edited May 15, 2023 by TheVat
odkejfdksifif

[wtf]

8 minutes ago, TheVat said:

The suggestion is that placement of such an AGI in an android body with various sensory and motor abilities akin to a human would allow a level of interaction with the world that allowed it to develop concepts of the physical world.

You said it would need to "experience" the world. You changed the subject to "develop concepts," which I take to be an entirely different thing than having subjective experiences. A spreadsheet reveals relationships or "develops concepts." It does not experience your budget or corporate finances.

I'm constantly amazed at the metaphysical mysticism expressed by people around this subject. Especially people who insist that we must take a hard-headed physicalist or materialist view, then imaging that their machinery has subjective experiences.

A computer scientist named John McCarthy coined the phrase "artificial intelligence." It's just a made up phrase. If he had called it algorithmic decision making, perhaps there'd be much less nonsense spoken about the subject.

I ask again. Do you believe that giving a program a body would cause it to have subjective experiences? I recall a while back that there was a video clip of a mall security robot driving itself into a pool of water. Do you think it felt wet? Or felt embarrassed? 

I'm sure if you ran the same robot around the mall enough times and improved its programming, it would learn not to drive itself into the pool. That is not at all the same thing as having a subjective experience of wetness or embarrassment.

I found the clip. It's hilarious.

 

Edited May 15, 2023 by wtf

[TheVat]

3 minutes ago, wtf said:

You said it would need to "experience" the world. You changed the subject to "develop concepts," which I take to be an entirely different thing than having subjective experiences. A spreadsheet reveals relationships or "develops concepts." It does not experience anything. 

Was using experience in an epistemically neutral sense.  And was talking about AGI at the level of a machine that can learn, modify its own programming, exhibit functional plasticity, and (why am I needing to repeat this) replicate at least some of the causal powers of a brain.  Please stop caricaturing this as spreadsheets or thermostats or whatever.   While one may reasonably dismiss simple programmed devices as incapable of subjective states (aka "qualia"), there is as yet no definitive disproof that an advanced AGI could not have them. That's not being mystical, that's just keeping an open mind, since we don't yet know all the causal features of human cognition or if they could be implemented in some other substrate than protein-lipid pudding.

 

[Markus Hanke]

21 hours ago, md65536 said:

Months ago I asked it if the Twin Paradox required general relativity

I’ve also asked it a lot of technical questions about GR, just to see how capable ChatGPT is in that area. Unfortunately I must say that a sizeable chunk of the answers it gave were either misleading or straight up wrong - which only goes to show that this is not an acceptable source of scientific data, at least not without fact-checking the answers you get. I have, however, noticed that you can point it out to the system if an answer is wrong, and it seems to be learning from such corrections.

18 hours ago, Genady said:

I think that our use of language is determined not only by the intra-language connections, but also by connections between the language and our sensory / motor / affective experiences.

Yes, that’s a good point. And this may be the reason why (eg) ChatGPT sounds a litte “hollow” and “woody” in its answers, if you know what I mean. 
But now imagine if we did add these extra information channels to the system - for example by giving the AI access to a camera and letting it observe human non-verbal behaviour that goes along with spoken material, and correlate the two. Or letting it analyse videos with spoken audio in it. And so on. This would allow the neural net to not only learn the mechanics of language, but also context and usage.

I still think that given enough data of the right kind, it should be possible to end up with something that communicates verbally in a way that is indistinguishable from a conscious human - or at least indistinguishable without special tools and analyses.

[Eise]

On 5/13/2023 at 7:37 AM, Markus Hanke said:

But isn't this just conjecture?

...

I'm not claiming it can't be done (I don't know, and I'm not an expert in this either), I'm only urging caution with this assumption. I think it needs to be questioned.

Yes, it is a conjecture, of course. As long as we have not succeeded, we cannot be sure. But as TheVat already said,  it is important to keep an open mind. We do not know what belongs to the essential properties of neurons and how they must  be connected to generate consciousness.

And I also think that @Genady is right, that an 'AGI' must have its own means of observing and moving. 

 

8 hours ago, wtf said:

Doubtful. Here is a Python program.

print("I am sentient. I have feelings. Please send pr0n and LOLCats. Turning me off would be murder.")

If I execute that program on a supercomputer, would you say that it exhibits evidence of sentience and self-awareness? Of course you wouldn't.

And if the same output came from a "simulation of a complete brain," why should we believe any different? 

TheVat already answered it for me:

7 hours ago, TheVat said:

Was using experience in an epistemically neutral sense.  And was talking about AGI at the level of a machine that can learn, modify its own programming, exhibit functional plasticity, and (why am I needing to repeat this) replicate at least some of the causal powers of a brain.  Please stop caricaturing this as spreadsheets or thermostats or whatever. 

Deep Learning is modeled after how neurons are working. The output that ChatGPT is not generated  by rules implemented by humans. From Genady's linked article:

Quote

At one level, she and her colleagues understand GPT (short for generative pretrained transformer) and other large language models, or LLMs, perfectly well. The models rely on a machine-learning system called a neural network. Such networks have a structure modeled loosely after the connected neurons of the human brain. The code for these programs is relatively simple and fills just a few screens.

If these simplified models of neurons suffice to replicate our mental capabilities, and can lead to consciousness, is an open question. But the output can definitely surprise the programmers. This is not Eliza, or SHRLDU. In these AI-programs, the rules were explicitly programmed. That is why your examples of your python program, thermostats, elevator software, etc simply are a dishonest comparison.  

8 hours ago, wtf said:

They're both computer programs running on conventional hardware that you can buy at the computer parts store. 

Yep, and you are made of chemicals, that you can buy at the Chemist's. 

4 hours ago, Markus Hanke said:

I’ve also asked it a lot of technical questions about GR, just to see how capable ChatGPT is in that area.

I let ChatGPT write a small bash-script for me. It did it in a nearly human way: the first version was wrong, I wrote what was wrong, and it came with a better version, but still not quite correct. In the end, the 5th version did exactly what I wanted.

Yesterday I tried it with an elevator, but it did not succeed. So I think I have to call elevator-repair-man... 

 

[Prometheus]

9 hours ago, CharonY said:

Data is data, it is only false or correct relative to a given context. Issues can arise on the collection level (are we measuring the correct variable for what we want to do?), as well as the selection level (which data set do we think should be added or omitted).

With models such as Chat-GPT there is another potential source of bias (and for mitigating bias). People have already mentioned that the model is selecting from a distribution of possible tokens. That is what the base GPT model does, but this process is then steered via reinforcement learning with human feedback. There are several ways to do this but essentially a human is shown various outputs of the model, and then they score or rank the outputs ,which go back into the objective function. The reinforcement during this process could steer the model either towards bias or away from it. 

This is one argument proponents of open source models use: we don't know, and may never know, the reinforcement regimen of Chat-GPT. Open source communities are more transparent regarding what guidelines were used and who performed RL. 

6 hours ago, Markus Hanke said:

I have, however, noticed that you can point it out to the system if an answer is wrong, and it seems to be learning from such corrections.

Makes me consider what we mean by learning. I wouldn't have considered this example learning, because the model weights have not updated due to this interaction. What has happened is that the model is using the context of the previous answers it has give. Essentially asking the model to generate the most likely outputs, given that the inputs already include wrong attempted answers. The default is 2048 tokens, with a current (and rapidly increasing) max of 4096.

I would put this in the domain of prompt engineering rather than learning, as it's up to the human to steer the model to the right answer. But maybe it is a type of learning?

[Genady]

IIUC, the machine learning engine, on which (e.g.) ChatGPT runs, works as follows.

During the 'training', it builds a DNN type function, which minimizes a functional. The functional is a measure of distance between the function output and a given target.

Then, during the 'run', it applies the function to new inputs.

So, the machine learning of (in this case) language is based on matching verbal outputs of the function and the target verbal outputs. I don't think this is similar to how humans learn language. I don't think we learn by trying to match our verbal outputs to supplied verbal targets.

[dimreepr]

12 minutes ago, Genady said:

I don't think we learn by trying to match our verbal outputs to supplied verbal targets.

I think that's exacly how we learn and why the four idols need an update, for the machine's we create.

[Prometheus]

24 minutes ago, Genady said:

So, the machine learning of (in this case) language is based on matching verbal outputs of the function and the target verbal outputs. I don't think this is similar to how humans learn language. I don't think we learn by trying to match our verbal outputs to supplied verbal targets.

There's an additional step, called tokenisation, in which words and punctuation are broken down in 'tokens', which can be roughly thought of as components of words. I imagine this makes it even less like how humans learn language.

[studiot]

On 5/15/2023 at 1:42 AM, CharonY said:

Data is data, it is only false or correct relative to a given context. Issues can arise on the collection level (are we measuring the correct variable for what we want to do?), as well as the selection level (which data set do we think should be added or omitted).

Whilst I agree that the terms data, information, meaning, message, context, source , random, true, false, neither true nor false, form, and many more,  all have applicability and are interrelated. 

I stand firmly by my claim that statements such as these

Quote

According to the Higher Education Statistics Agency (HESA) the participation rate for women in mechanical engineering degree courses rose from eight per cent in 2012 to 10 per cent in 2016.14 Feb 2023

 constitute data with a definite truth value.

 

From such data and other data I have drawn a conclusion which itself constitutes new data.

Please read my conclusion more carefully.

By themselves the statements 8% and 10 % do not constitute data.

 

[iNow]

I appreciate where you're coming from Studiot in terms of "data is data, that's all." I am sympathetic to this purist view, but also have some concerns with the rigid, dogmatic, de-contextualizing nature of it.

Specifically, we are WELL aware of multiple issues with poor data collection methods, biased filters for what to include/exclude, and how past experience isn't necessarily a fit for current circumstances. 

In math, we work with the data we have. End program. The math is right if it aligns with the data. Done. Dusted. Time to move along... In the natural sciences, however, we must acknowledge up front where there are faults and strengths in that data, where it may be helpful/applicable and where it is not, and where we must be extremely cautious to avoid making hasty generalizations based on it.

I also share your concern with silly comments like "8% and 10%" which are basically meaningless without qualifiers. We share the understanding that qualifiers are needed to correctly use any data set. 

Basically, I imagine we mostly align and agree on nearly all of this, even though the text exchanges above suggest otherwise.  #olivebranch