Biologically Inspired Cognitive Architectures and Generative Pre-Trained Transformers are Friends or Foes? Volume 60- Issue 3

Emanuel Diamant*

• Independent Research Engineer, Israel

Received: January 24, 2025; Published: February 06, 2025

*Corresponding author: Emanuel Diamant, Independent Research Engineer, Israel

DOI: 10.26717/BJSTR.2025.60.009461

Abstract PDF

BICA*AI is a well-established long-lasting R&D enterprise aimed at creating computational architectures intended to emulate Human-level Artificial Intelligence. Recently and quite unexpectedly in its field has appeared another contender – a GPT-based AI tool designed to mimic man-computer conversation in a user-friendly natural human language. As its designers claim, the device exhibits signs of General AI. After an exciting and joyful reception, it became clear that the new competitor does not fulfill its expected promises – it returns wrong and misleading responses, deceptions, and disinformation. The issue raised a wave of public objections and a request to stop and prevent further device deployment. On the other hand, the device designers claim that the imperfections are temporary, and very soon the product will rich its avowed qualities. No, this will never happen! The purpose of this paper is to explain that the current approach to GPT-based AI tools design is initially flawed, wrong, and unsuitable because it ignores the basic definitions of Intelligence and Information pro-cessing. The paper joins the general awareness that unrestricted and free dissemi-nation of wrongly designed GPT-based AI tools poses a threat to human society, similar to the threat of careless biological weapon research.

Keywords: BICA*AI; GPT-Based Tools; General AI

BICA*AI – which stays for Biologically-Inspired Cognitive Architectures for Artificial Intelligence – is a well-established long-lasting (since 2005) R&D enterprise aimed at creating computational architectures of human intelligence, [1]. “The ultimate goal of research in cognitive architectures is to model the human mind, eventually enabling us to build human-level artificial intelligence. To this end, cognitive architectures attempt to provide evidence of what particular mechanisms succeed in producing intelligent behavior and thus contribute to cognitive science”, [2]. Generative AI or GPT – which stays for Generative Pre-trained Transformer – is a recently emerged AI tool and application, designed to mimic a computer conversation with a user in natural language and simulate the way a human would behave as a conversational partner. [3]. GPT models are trained on massive amounts of internet text data and are capable of generating responses that closely resembled human writing. Originally designed for purposes of NLP tasks, transformers have been adapted for various vision tasks (image classification, object detection, image generation, and video processing), audio-related applications (speech recognition, speech synthesis, speech enhancement, music generation), and various multimodal scenarios (visual question answering, visual commonsense reasoning, caption generation, speech-to-text translation, and text-to-image generation), [4].

The prime GPT model, developed by OpenAI, was consequently enhanced by more and more sophisticated versions, such as GPT-2 in 2019 and GPT-3 in 2020. In December 2022, OpenAI launched a free preview of the ChatGPT model, a new AI tool based on the GPT-3.5 model version. On March 14, 2023, OpenAI released GPT-4, both as an API and as a feature of ChatGPT, [5]. According to OpenAI, the preview received over a million signups within the first five days. As an anonymous source revealed, (cited by Reuters in December 2022), OpenAI is projecting $200 mil-lion revenue in 2023 and $1 billion revenue in 2024, [5]. Inspired by OpenAI’s success, other GPT developers have launched their own products: Amazon, Google, Microsoft, Baidu, GitHub, Meta, Apple, IBM, Nvidia, and others have created their own GPT-based products and put them to market examination, [6]. From this short exposition of BICA versus GPT-4 achievements, it is perfectly clear that GPT-based products have a far more privileged position in the eyes of Human-level AI tools developers and users. However, upon a closer inspection, it turns out that the position of GPTbased AI tools is not so extremely great as it looks initially. Despite the generally recognized success in creating artificial devices that mimic the human ability to communicate through oral or written conversation, GPT-based devices have proven to be a potential danger to their users, [7-9]. According to many publications, GPT is a large language model that uses deep learning to create human-like text.

Or, more certainly, it is a computing system designed to generate sequences of words, code, or other data, starting from an initial input called the prompt, [10] When determining the sequence of words, it simply predicts the next word based on statistical correlations in the training data and prompts, [9]. Since the system sees only formal, statistical patterns between words, it does not understand the meaning of words and does not understand their semantics. Therefore, it is also called the “stochastic parrot”, randomly stitching together words without reference or meaning, [8]. All other negative features of the system subsequently follow from this - the model inherits biases and errors in the training data, the model is very sensitive to the design and formulation of prompts. [11], her answers are inexplicable to users, her results are not verified or validated, and users cannot determine the validity of the model’s output, or whether the model is simply a “fiction”, [9]. On the other hand, the model produces very coherent, natural-sounding, and human-like responses that users find compelling and readily trust them, even if they are inaccurate, [9] At the same time, the system is exceptionally fast, very user-friendly, responds to natural language prompts and requires little or no user training, references or values, [8]. Text generation at speed, scale, and ease of use makes the GPT model exceptionally well suit-ed to widespread misinformation and deliberate misuse of the system as a tool for fraud, as a “weapon of mass deception” [9].

The main risk of weaponization is related to the potential ability of GPT-3 to dramatically increase influential production, which is likely to be human, [11]. From the above, it becomes clear that GPT systems can pose a serious danger to society and humanity. Realizing this and seeking concrete action to eliminate the dangers and harms that threaten society, several leading AI researchers, as well as a number of people who work in companies participating in the AI race, signed an open letter calling for a six-month pause in the training of AI systems, more powerful than GPT-4, [12]. The letter, published on March 30, 2023, states that “In recent months, AI labs have been stuck in an out-of-control race to develop and deploy ever more powerful digital minds” [13]. “AI systems with human-competitive intelligence can pose a serious danger to society and humanity, as extensive research has shown” [12]. Over 1300 people, including Elon Musk and Steve Wozniak, have signed this letter asking all engineers to stop immediate AI development outside of GPT-4 for 6 months due to fear of “losing control” [13]. In addition, the Future of Life Institute states that “development moratoriums should be public and verifiable” and “if the decision to suspend AI development cannot be taken immediately, governments should intervene with AI development authorities.” [13].

In response to this call, on May 17, 2023, a Subcommittee of the U.S. Senate Judiciary Committee held a hearing titled “AI Oversight: Rules for Artificial Intelligence”[14]. Similar measures of administrative control and administrative order have been adopted by the parliamentary commissions of the European Union, Great Britain, Japan, and other countries (A more detailed report on the actions of many countries in this direction can be found in [15]). At the same time, it remains as a problem that these measures of administrative control ignore the scientific and technical aspects of the problem of creating Human level Artificial Intelligence systems. Parliamentary commissions do not deal with these issues, rightly claiming that this is not within their competence. At the same time, the organizations involved in the creation of these systems confidently declare that they have already reached the level of General Intelligence. This is exactly what Microsoft claims in its recent publication: “it could reasonably be viewed as an early (yet still incomplete) version of an Artificial General Intelligence (AGI) system”, [16] Sorry, but this is not true. The uncontrolled and unsupervised use of systems whose creators claim that they have already reached the level of General Intelligence can only be incomparably more dangerous to society. Because the general public is not competent in these matters. And scientists and researchers competent in these matters claim that we are still very far from the level of General Intelligence. And the main reason for this is that today we do not know at all what Intelligence is, and what the brain’s cognitive abilities are supposed to be.

1. Christopher Koch: “AGI is ill-defined because we don’t know how to define intelligence. Because we don’t understand it…”, [17].

2. Luciano Floridi: GPT language model “has nothing to do with intelligence, conscious-ness, semantics, relevance, and human experience and mindfulness more generally”[10]. I deliberately provide here the quotes from Christopher Koch and Luciano Floridi – Christof Koch is a chief scientist of the Mindscape Program at Seattle’s Allen Institute. He has a background in both AI and neuroscience, he is the author of three books on consciousness as well as hundreds of articles on the subject, including features for IEEE Spectrum and Scientific American, [17]. Luciano Floridi is a professor of philosophy and ethics of information at the University of Oxford, best known for his work on the philosophy of information, and information ethics. According to Scopus, he was the most-cited living philosopher in the world in 2020, [18].

Therefore, Again

Christopher Koch: “AGI is ill-defined because we don’t know how to define intelligence. Because we don’t understand it…”, [17].

Luciano Floridi: GPT language model “has nothing to do with intelligence, consciousness, semantics, relevance, and human experience and mindfulness more generally”, [10]. Their opinions sharply diverge from that of other prominent researchers in the field of AI, the so-called “founding fathers” of AI – Yann LeCun [19], Geoffrey Hinton [20], Rodney Brooks [21], and others, who are concerned mostly about the social dangers associated with uncontrolled use of GPT-based AI tools. In their most recent publications, not even a word about the technical and scientific aspects of the issue is provided. I deliberately provide here the quotes from Christopher Koch and Luciano Floridi, because their views unconditionally support my own views on the subject (and of the BICA*AI R&D activities). During the past years, I have repeatedly expressed my viewpoints on the critically important issues of Intelligence and Information, usually omitted in today’s AI research studies. Interested readers could find some relevant papers in the Reference list – [22-27]. But for consistency of our discussion, I will provide here a short excerpt of the ideas that were exposed once in these publications.

There is a widely shared opinion that human intelligence cannot be defined as a single trait or as General Intelligence. Theories of Multiple Intelligences are steadily gaining mainstream attention. However, most frequently, Intelligence is perceived as an umbrella term that em-braces (integrates) a multitude of human cognitive abilities (to sense, to perceive, to interpret the surrounding environment; to recognize, to categorize, to generate plans, to solve problems; to predict future situations, to make decisions and select among alternatives; to learn, to memorize, and so on), which altogether produce the effect of intelligence, [26]. Another commonly shared belief is that human cognitive abilities are all a product of human brain activity. Brain – as it is generally agreed and accepted – is busy with information pro-cessing. So, we can accept – Intelligence is a product of the brain’s information-processing activity. More generally – Intelligence is the ability to process information, [27]. In such a form, the definition is applicable to all domains of natural living beings and to artificial human- made designs as well.

Although the term “Information” is widespread and extensively used today, no one actually knows what it means and what it actually stands for. The concept of “information” was at first introduced by Shannon in 1948. Then other scientists joined the mission – Kolmogorov, Fisher, Chaitin. However, none of them was ready to define what is “information”. They were busy with the “measure of information”. That was enough to improve the performance and reliability of technical communication systems. In modern sciences, the needs of communication cannot be reduced only to the optimization of the system’s technical parameters. The semantic aspects of the message are of a paramount importance and thus must be met. In accordance with the soul and spirit of these requirements, I have developed my own definition of information. (Interested readers can look into the references [25,28,29]).

My Definition of Information Sounds Today Like This

“Information is a linguistic description of structures observable in a given data set.” In a data set, the data elements are not distributed randomly, but due to the similarity of their physical parameters, are naturally grouped into some kind of clusters or cliques. I propose to call these clusters primary or physical data structures. In the eyes of an external observer, these primary data structures are arranged into larger and more complex agglomerations, which I propose to call secondary data structures. These secondary structures reflect the observer’s view of the grouping of primary data structures, and therefore they could be called meaningful or semantic data structures. While the formation of primary (physical) data structures is determined by the objective (natural, physical) properties of the data, the subsequent formation of secondary (semantic) data structures is a subjective process governed by the conventions and habits of the observer (or a mutual agreement of an observers’ group). As said, the description of the structures observed in the data set should be called “In-formation”. In this regard, it is necessary to distinguish between two types of information – physical information and semantic information. Both are language descriptions; however, physical information can be described using a variety of languages (recall that mathematics is also a language), and semantic information can be described only using the observer’s natural language. (See [25] for more details). Information processing is carried out in a hierarchical fashion, where the semantic information of a lower level is transferred to the next higher level, where it becomes part of a structure of higher complexity. This agglomeration is carried out according to subjective rules fixed in a proto-typical (referential) structure called the observer’s memory, which is stored in the neuron’s body.

An important consequence of the above definition of information is the understanding that information descriptions always materialize as a set of words, a fragment of text, a narrative. In this regard, an important note should be made – in biological systems, these text sequences are written with nucleotide letters and amino acid signs. This turns the information into a physical entity, into a “thing”, with its weight, length and other physical properties. For the purposes of our discussion, this is an extremely important remark. So: The brain is processing information. Neurons are the functional units that perform the du-ty. Despite their discrete structure, neurons are not separate functional units—successful information processing requires close cooperation between work partners. For this reason, neurons are connected in a network in which they communicate with each other, transmitting, exchanging, transferring - in a word – jointly processing information.

As it was just explained above, Intelligence is a product of Information processing, where Information is a linguistic description of observable data structures. The bulk of information processing in the brain is Semantic information processing. In this regard, the reliance of GPT-based implements on Large Language models (LLMs) seems natural and justified. (Because Semantic information is a linguistic description, a text, a narrative). But a closer look at the LLM’s paradigms reveals that linguistic components in the training set of the LLM models appear as a set of intentionally broken small word chunks called tokens, or model parameters. The size of model parameters is continuously increasing – GPT-4 is a ~600 billion parameter model (Some people suggest it’s a trillion). Earlier models (GPT-3/3.5) had about 185 billion. In such a case you cannot speak about a language model, it looks more like a statistical data model. As such (and so it is stated in the relevant literature) – “These statistical models need to be trained with large amounts of data to produce relevant results”, [10]. Deep Learning (Machine Learning) mechanisms used for training the LLMs are also a valuable argument in this regard – Machine learning principles are applicable only to data patterns mining and discovery. The description of data patterns (structures) is called physical information. Intelligence, as you remember, is busy with semantic information processing! Therefore, we can conclude that ML neural networks are appropriate only for physical information running, and not for semantic information processing, which is required for proper Intelligence handling.

The “black box” nature of Large Language Models, as well as all other designs relying on the Artificial Neural Network paradigm, cannot be suddenly overturned, enhanced, and become an Explainable Neural Network model. Data-driven physical information descriptions cannot in a moment be transformed into semantic information descriptions, that is, into a linguistic description, a piece of text, a language sample. (That is what GPT-based AI devices claim to be able to do). Another popular and false claim is that GPT-based AI tools are close to be considered as General AI accomplishments. Again, that is a popular and widespread misunderstanding. Se-mantic information processing is done by following the rules saved as a reference prototype (a memory-saved reference) of the current-level semantic structure. These rules, these memories are the observer’s private property. That is, semantic information processing is always subjective. That is, the idea of General Intelligence is wrong and cannot be implemented in any natural or artificial construct. Multiple Intelligence (just mentioned above) being a composition of several different subjective semantic information processing structures can be seen as a valid feature of the future Human-Level Artificial Intelligence.

The hype around GPT AI tools does not fade. But the purpose of this paper is not to take part in the ongoing discussion, not to glorify or disapprove generative AI tools’ merits or demerits. I am trying to stay on the technical side of the story and to analyze GPT AI tools’ technical realities (and myths). In this regard, it is perfectly clear that the reliance of GPT AI tools on Deep learning (Ma-chine learning) training techniques dismisses any claim of its association with Natural Language Processing–NLP assumes semantic information processing, while Deep learning is busy with endless enormous data processing.

The “black box” nature of such processing makes its results erroneous and unpredictable. That is the source of most GPT AI tools’ faults and failures. And these flaws cannot be repaired at further stages of processing. They are fundamental, ground-based, built-in. They are forever. DARPA’s 4-years (2015 – 2019) attempt to create explainable Deep learning tools (explainable AI (XAI) tools) has failed forever, [30] In this regard, it will be interesting to recall that Ali Rahimi, the 2017 NIPS Award winner, in his award speech, declared that “the current practice in machine learning is akin to “alchemy”, [29]. If Machine learning is alchemy, then what can be said about the concept of Intelligence? (Intelligence, as you now know, is a product of information processing, semantic, not physical, information processing). However, the designers of GPT-based appliances stubbornly claim that “Intelligence is a multifaceted and elusive concept that has long challenged psychologists, philosophers, and computer scientists. There is no generally agreed-upon definition of intelligence, but one aspect that is broadly accepted is that intelligence is not limited to a specific domain or task but rather encompasses a broad range of cognitive skills and abilities” [16]. In the continuation of this, just given, quote, they claim that such an approach to Intelligence is coherent with the grandfathers’ intentions, expressed in the “Proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955.” (Signed by John McCar-thy, Marvin L Minsky, Nathaniel Rochester, and Claude E Shannon). This argument does not seem credible. Especially, if we recall that grandfathers’ views of the problem were very blurred and preliminary. They have failed to assess the complexity of the task (Intelligence definition) and the mutual contradictions between its basic constituents. Recall the story of how Minsky proposed to hire a student to solve the problem of vision during the student’s summer vacations. (The problem even now remains unsolved) [26]. Recall how Shannon rejected the slightest notion of semantic information: “These semantic aspects of communication are irrelevant to the engineering problem… It is important to emphasize, at the start, that we are not concerned with the meaning or the truth of messages; semantics lies outside the scope of mathematical information theory”, [31].

Despite the proud claims of GPT designers that they were lucky “to give a preliminary assessment of GPT-4’s intelligence, which is an arduous task given the lack of formal definition for this concept, especially for artificial systems”, [16]. Any sign of understanding that Intelligence is a product of information processing, and that Information, in its turn, is a composition of two, physical and semantic information entities, does not appear in the GPT designers’ statement. And that is another source of GPT tools failures. Finally, I hope that the answer to the question posed in the title of this paper is perfectly clear – GPT-based AI tools do not present any threat or danger to the AI tools design community. The hype will fade, and BICA*AI will continue to pursue its goals – to design tools that will allow it to emulate Human-level Artificial Intelligence.

1. Biologically inspired cognitive architectures, From Wikipedia, free encyclopedia.
2. Iuliia Kotseruba, John K Tsotsos (2020) 40 years of cognitive architectures: core cognitive abilities and practical applications. Artificial Intelligence Review 53: 17-94.
3. Wikipedia.
4. Tianyang Lin, Yuxin Wang, Xiangyang Liu, Xipeng Qiu (2022) A Survey of Transformers. Arxiv 3: 111-132
5. Wikipedia.
6. Wikipedia.
7. Chloe Xiang (2023) 'Just Another Hype Cycle': Elon Musk Reportedly Building ‘Based AI’ Because ChatGPT Is Too Woke.
8. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell (2021) On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? pp. 610-623.
9. Alejo Jos´e G Sison, Marco Tulio Daza, Roberto Gozalo-Brizuela, Eduardo C Garrido-Merch´an (2023) ChatGPT: More than a Weapon of Mass Deception, Ethical challenges and responses from the Human-Centered Artificial Intelligence (HCAI) perspective.
10. L Floridi, M Chiriatti (2020) GPT 3: Its Nature, Scope, Limits, and Consequences, Minds and Machines. Springer 30: 681-694.
11. K McGuffie, Alex Newhouse (2020) The radicalization risks of GPT-3 and advanced neural language models.
12. Chloe Xiang (2023) The Open Letter to Stop 'Dangerous' AI Race Is a Huge Mess.
13. (2023) Open letter.
14. (2023) Transcript: Senate Judiciary Subcommittee Hearing on Oversight of AI.
15. (2023) Reuters, Which countries are trying to regulate artificial intelligence?
16. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, et al. (2023) Sparks of Artificial General Intelligence: Early experiments with GPT-4.
17. Glenn Zorpette (2023) GPT-4, AGI, and the Hunt for Superintelligence Neuro expert Christof Koch weighs AI progress against its potential threats.
18. Luciano Floridi. Wikipedia, the free encyclopedia.
19. Melissa Heikkilä, Will Douglas Heaven (2022) Yann LeCun has a bold new vision for the future of AI. Yann LeCun’s big bet for building intelligent machines, MIT Technology Review.
20. why is ‘godfather of AI’ Geoffrey Hinton worried? These are the 4 dangers he fears about the tech. Euronews.
21. Glenn Zorpette (2023) Just Calm Down About GPT-4 Already and stop confusing performance with competence. says Rodney Brooks.
22. Emanuel Diamant (2002) Biologically Inspired Image Information Content.
23. Emanuel Diamant (2009) Machine Learning: When and Where the Horses Went Astray?.
24. Emanuel Diamant (2008) Unveiling the mystery of visual information processing in human brain, Brain Research 1225: 171-178.
25. Emanuel Diamant (2015) The brain is processing information, not data: Does anybody knows about that? MDPI “Information”.  
26. Emanuel Diamant (2015) Advances in Artificial Intelligence: Are You Sure, we are on the Right Track? Transactions on Networks and Communications 5(4).
27. Emanuel Diamant (2018) Designing Artificial Cognitive Architectures: Brain Inspired or Biologically Inspired?. BICA 2018 Conference, Prague.
28. Emanuel Diamant (2016) Shannon's definition of information is obsolete and inadequate. It is time to embrace Kolmogorov’s insights on the matter. Conference Paper.
29. Emanuel Diamant (2018) Artificial Neural Networks: a bio-inspired revolution or a long-lasting misconception, and self-delusion. A rejected Conference Submission.
30. David Gunning, Es Vorm, Yunyan Wang, Matt Turek (2021) DARPA’s Explainable AI (XAI) program: A retrospective, Authorea.
31. Emanuel Diamant (2016) Computational Intelligence: are you crazy? Since when has intelligence become computational. Conference paper.