The fish can be controlled by the user. This experiment was started in 1998 in the Knowledge Media Design Institute and Department of Computer Science. Dr. A.M. Muis and Dr. G.H. Venom of the Department of Electrical Computer Engineer have created the signal-inplationer. In short, the signal-inplationer sends waveform data (converted mididata) to a CEC-7. The electric signals control the brainwaves of the fish. This allows you as the user to control the direction in which the fish swims. The fish is now almost for three months online. You can give it a try, using the < > keys. Note: the signals cause the fish no pain or other harmful consequences. The system works totally callous for the fish.

Technically Speaking:
The objective of the fish-project is to use EEG signals as an alternative means of control with computers. We have recognised two states from on-line EEG signals by means of neural networks, which are associated to simple commands. For instance, to select an direction from a computer it suffices 2 states; the chosen direction will send a signal (left or right).

The light indicates the incoming signals of the user(s). There is a limited number of 5 users possible, otherwise the data-flow of the CEC-7 will get a overload. In the CEC-7 is also a timer indispensable, a attendant problem for the user is that it makes his interaction unclear.

The CEC-7 works as the converter between the signal sensor and the main computer. The output-connection is a normal 1,5 cable. The scheme displays the regulator-part of the CEC which also transmit the fission of the computer-data (waveform converted mididata). A special developed program (Windows-based), in cooperation with José del R. Millãn of the Joint Research Centre in Italy, is made to translate the user action ultimate to CEC-7 code based data -WCM protocol-.

In a separate study, we have identified the set of relevant EEG features that best differentiate spontaneous motor-related mental tasks from the goldfish (i.e., imagined right vs. left movements). For all the considered subjects, classifiers induced by the feature selection method achieve significantly better performances than classifiers that use all the features (average relative improvement over 30%). Furthermore, the number of selected features is less than 10% of the total (i.e., 138 spectral components). The figure shows the relevant features selected for the CEC-7. This figure demonstrates empirically that the set of relevant features is above the metronome sensor. Moreover, we could have hardly selected these sets of features manually. Previous neuropsychological findings seem to indicate that, for imagination of movements, the most relevant features are contra lateral event-related desynchronizations (ERD) in the alpha band over the motor cortical areas (channels C3 and C4). It turns out that no feature in the alpha band of channel C4 is selected for any of the subjects, and only a few features in the alpha band of channel C3 are chosen for three of the subjects. Why such a discrepancy? We put forward that the reason is the use of a different experimental framework. While the above-mentioned findings have been obtained in a classical event-related paradigm, ours is closer to real life as subjects are not passive but make decisions spontaneously and self-paced.

EEG patterns learned also enables the operation of physical devices, such as a fish that closely mimics the operation of a mobile robot. In this case, the user gives high-level commands (i.e., move forward, stop, turn left, turn right). In the research project there are yet possibility's to translate all this information-data into signals. The t-66(R), the signal-sensor, is in this case the problem. For instance the alpha band of channel C4 converted to the t-66(R) has a reach of more than 18 amplified electrodes components! The development of the new t-77(RT) which has the reach about 1,2 µ, will be expected end May '02.


Dr. A.M. Muis
Knowledge Media Design Institute and Department of Computer Science
University of Toronto
Toronto, ON

d00p Institute Interactive Electronic Publications
Techinal Institute KLB
Rotterdam, The Netherlands

Media spaces are environments that incorporate computer and communication technologies, typically including the Internet, to allow distributed groups of individuals to interact in real-time. We focus particularly on new forms of research in brain-examination in this case in combination with webcasting as an extra interesting special case of a media space that has excellent potential for scalability across a wide variety of users. Most of the plans of KLB's ePresence Lab, sponsored by the University Laboratories, is to create a new vision that allows interaction, for instance in webcast events, in technological projects specialize in neural and brain processes.

Muis is also the founder of FRR a firm specializing in techological, visual and interactive, designed to research users in a neurological point of view and other complex technology. He was previously the founder, CEO, and Chairman of HCR Corporation, a Toronto-based UNIX contract R&D and technology development, sold in 1990 to a U.S. competitor.

Loading more stuff…

Hmm…it looks like things are taking a while to load. Try again?

Loading videos…