Machine translates brainwaves into English.

[studiot]

Not quite sure where to put this but will start here.

Headline

Machine translates brainwaves into English.

https://www.bbc.co.uk/news/science-environment-52094111

With 3% inaccuracy

Quote

They trained algorithms to transfer the brain patterns into sentences in real-time and with word error rates as low as 3%.

 

So do we think the same in all languages?

[Dagl1]

36 minutes ago, studiot said:

So do we think the same in all languages?

I don't think so, based on (Anecdotal evidence incoming: ) learning a bit of Japanese and studying Mandarin now (native Dutch speaker), the languages's usages of words/grammar is very different than that of Germanic languages. This study is super interesting though, but I wonder if this would work with using a non-Germanic/non-Latin-based language speaker and their words be decoded to English (in the article they do mention it only recognises individual words though,  so maybe then it would work, but sentences seem to be off so far). I suppose it also depends on the training data itself. Either way, thanks for sharing!

 

[MigL]

Wow ! Just like Star Trek.
A universal translator already.
Should come in handy when we meet aliens.
( ever see the movie Arrival ? )

What's next ?
Warp drive, or the transporter.

[Strange]

2 hours ago, Dagl1 said:

I don't think so, based on (Anecdotal evidence incoming: ) learning a bit of Japanese and studying Mandarin now (native Dutch speaker), the languages's usages of words/grammar is very different than that of Germanic languages. This study is super interesting though, but I wonder if this would work with using a non-Germanic/non-Latin-based language speaker and their words be decoded to English (in the article they do mention it only recognises individual words though,  so maybe then it would work, but sentences seem to be off so far). I suppose it also depends on the training data itself. Either way, thanks for sharing!

I assume that if the system were trained on a Chinese or Japanese speaker then it would recreate sentences in that language. The same parts of the brain are involved in all cases.

It does raise the interesting question whether there would be anything common in the brain activity of someone saying "eat a banana" vs "バナナを食べる". If some part of the brain represents the absract concepts of eating and the thing being eaten.

[Dagl1]

1 minute ago, Strange said:

I assume that if the system were trained on a Chinese or Japanese speaker then it would recreate sentences in that language. The same parts of the brain are involved in all cases.

It does raise the interesting question whether there would be anything common in the brain activity of someone saying "eat a banana" vs "バナナを食べる". If some part of the brain represents the absract concepts of eating and the thing being eaten.

I too would assume that it could recreate something within the same language as the training data, but I wonder (and that is how interpreted Studiot's question) if, as you said, eat banana and its Japanese counterpart would be both 'decodable' by a algorithm trained only for 'eat banana'. But as long as it is limited to just decoding individual words, I think we can assume it works for all languages (if trained in that language). Some languages must be harder than others, due to their internal logic (I think). 

Reading Studiots question now, I may have misinterpreted it

[studiot]

13 minutes ago, Dagl1 said:

I too would assume that it could recreate something within the same language as the training data, but I wonder (and that is how interpreted Studiot's question) if, as you said, eat banana and its Japanese counterpart would be both 'decodable' by a algorithm trained only for 'eat banana'. But as long as it is limited to just decoding individual words, I think we can assume it works for all languages (if trained in that language). Some languages must be harder than others, due to their internal logic (I think). 

Reading Studiots question now, I may have misinterpreted it

No I think you guys got it alright.

But instead of a chinaman v a european eating a banana how about the thought (equallly simple)

Wearing black

or

Wearing White

 

?

[Strange]

There have been similar experiments with images and even dreams. But it's not clear to what extent what is learnt from one individual can be used to "decode" someone else's visualizations:

Quote

There is a similarity amongst the subjects, so from that result, we could pick up some basic dream content and then we can build a model from those base contents, and they may apply to other people

https://www.huffpost.com/entry/reading-dreams-scans_n_3016895

3 hours ago, studiot said:

So do we think the same in all languages?

Yes and no.

There are features common to all languages. And there are features that are unique to pretty much every language (or language family). For example all languages have words that are functionally equivalent to verbs, nouns and adjectives. So that probably represents something about the way the brain organises information. But which things are verbs, nouns and adjectives varies enormously between languages. So "hungry" is an adjective in English, a noun in Italian and a verb in Japanese (actually, there isn't a word for it in Japanese but...) 

There is something called the Sapir-Whorf(*) hypothesis which suggested that the language people speak controls or limits the things they can think about. But it is not, in general true. So, for example, some languages have fewer words for colours than others but that doesn't mean that, for example, a Japanese person can't tell the difference between blue and green; they just happen to use the same word for both.

However, having words for concepts does make a very slight difference to how the brain processes things. So a common method in psychological tests is to time how long people take for a particular task. And it turns out that in a language that has more words for colours, people are fractionally faster to match the colour of objects. Or in languages where the grammar categorise objects by their shape, then people are fractionally faster to identify shapes. But this is milliseconds and there is probably more variation within a language group than there is between any two people from different language groups.

(*) No, not the Klingon

Edited March 30, 2020 by Strange

[studiot]

57 minutes ago, Strange said:

There is something called the Sapir-Whorf(*) hypothesis which suggested that the language people speak controls or limits the things they can think about. But it is not, in general true. So, for example, some languages have fewer words for colours than others but that doesn't mean that, for example, a Japanese person can't tell the difference between blue and green; they just happen to use the same word for both.

I think you didn't pick up the connotations.

Black is the European colour of mourning

White is the Chinese colour of mourning.

[Strange]

4 minutes ago, studiot said:

I think you didn't pick up the connotations.

Black is the European colour of mourning

White is the Chinese colour of mourning.

I was going to ask what you were talking about. It wasn't at all obvious. Even though I am aware of the use of the colours in that context.

But my comment was purely addressing the "do we think the same in all languages" question.

[cosmofm]

are brain waves the same in all languages?

 

Edited March 30, 2020 by cosmofm

[Strange]

7 minutes ago, cosmofm said:

are brain waves the same in all languages?

Why not read the thread before asking 

[MigL]

3 hours ago, Strange said:

someone saying "eat a banana" vs "バナナを食べる"

Oh, now you're just showing off...

[Strange]

6 hours ago, MigL said:

Oh, now you're just showing off...

 I might have used Google Translate. (I didn't)

[Dagl1]

10 hours ago, Strange said:

There is something called the Sapir-Whorf(*) hypothesis which suggested that the language people speak controls or limits the things they can think about. But it is not, in general true. So, for example, some languages have fewer words for colours than others but that doesn't mean that, for example, a Japanese person can't tell the difference between blue and green; they just happen to use the same word for both.

Like to chip in on this one, although not entirely relevant to the language part (maybe): (this is from Behave, by Robert Sapolsky, but I don't think there was a reference with this particular statement) When you show people two colours that are similar to each other, whether that person's native language has two or one name for those colours, will for a great part determine the amount of similarities or differences they see. So if I show you two types of blue-ish colour, I suppose Cyan and Turquoise may do (?) and person A's native tongue does not differentiate between those two, then he will find them more similar than person B, who may speak English and knows his/her colours (Cyan and Turquoise).

ano naniga, Strange wa tensai, ne ^^ 
(sorry I couldn't help myself, but can't type Japanese here (I am not very sure what the actual words are either, just heard people say nanga (?) whenever they had no idea what to say)).

Edited March 31, 2020 by Dagl1

[Strange]

19 minutes ago, Dagl1 said:

Like to chip in on this one, although not entirely relevant to the language part (maybe): (this is from Behave, by Robert Sapolsky, but I don't think there was a reference with this particular statement) When you show people two colours that are similar to each other, whether that person's native language has two or one name for those colours, will for a great part determine the amount of similarities or differences they see. So if I show you two types of blue-ish colour, I suppose Cyan and Turquoise may do (?) and person A's native tongue does not differentiate between those two, then he will find them more similar than person B, who may speak English and knows his/her colours (Cyan and Turquoise).

I think that's the sort of experiment I was thinking of. One way of (objectively) measuring how similar people think two colours are is to time how long it takes them to hit the "same" or "different" button. 

[Endy0816]

Have to caution not to go too far down the rabbit hole as the person has to be Speaking.

It could be easily picking up on what is being sent to the muscles.

 

[Strange]

37 minutes ago, Endy0816 said:

Have to caution not to go too far down the rabbit hole as the person has to be Speaking.

It could be easily picking up on what is being sent to the muscles.

That is a good point. At least part of the brain activity will be related to muscle activation.

Form the paper:

Quote

These results therefore suggest that the network has learned to decode commands to the speech articulators (vSMC) and auditory feedback, either actual or anticipated (STG).

https://www.nature.com/articles/s41593-020-0608-8.epdf

But as most of the same areas of the brain are activated even when people talk to themselves (without vocalising) it still has interesting possibilities.

But this does mean that it is definitely going to be language specific, rather than "conceptual". Although perhaps other techniques (fMRI perhaps) could be used for that.

7 hours ago, Dagl1 said:

(sorry I couldn't help myself, but can't type Japanese here (I am not very sure what the actual words are either, just heard people say nanga (?) whenever they had no idea what to say)).

Probably nanika/nanka = something/thingummy/whatever

[Endy0816]

21 minutes ago, Strange said:

That is a good point. At least part of the brain activity will be related to muscle activation.

Form the paper:

https://www.nature.com/articles/s41593-020-0608-8.epdf

But as most of the same areas of the brain are activated even when people talk to themselves (without vocalising) it still has interesting possibilities.

But this does mean that it is definitely going to be language specific, rather than "conceptual". Although perhaps other techniques (fMRI perhaps) could be used for that.

Probably nanika/nanka = something/thingummy/whatever

Right. See similar for people paralyzed or missing limbs using interfaces.

[Strange]

10 minutes ago, Endy0816 said:

Right. See similar for people paralyzed or missing limbs using interfaces.

That is different though, in that it directly measures the signal sent to the missing or paralysed limb rather than the brain activity that leads to those signals.