-> Italiano

News - 26/1/2020. Andrea Turso.

GPT-2 Model trained & fine-tuned with DSM: Diagnostic and Statistical Manual of Mental Disorders.

"GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text."

We trained GTP-2 model of OpenAI using the “Diagnostic and Statistical Manual of mental disorders (DSM)”. To do that, we used the Google Cloud service (thanks to the Nvidia Tesla T4), Tensorflow and the medium 355M (6.07) model.

During the tuning phase we trained the model using the text of DSM manuals starting from the edition of 1952, that we prepared and structured for the project.

The results were surprising as you can see in some examples reported on this page.

This exercise let us prove and study tools and models that today represent the state of art, but first of all it gave us the opportunity to ask ourselves questions about the use of these technologies.

Even if we are far from practical applications, the results are definitely evocative and make us think.

Here is an example of text generated from the trained model, completely unsupervised::

: "The problem is not with the individual, but rather with the social norms of his or her culture."

Evocative, isn’t it? The more interesting fact of the model is that it clearly proves how is possible to make tasks of NLP - Natural Language Processing as comprehension and text synthesis without explicit supervision.

In the next future Natural Language Processing technologies will let us develop interfaces for operators and chatbot able to synthetize and create text in a specific field. In this way we have already started some experimentations of GPT-2 model and results are hopeful.

To learn more about GPT-2: official paper.


Image: Google Cloud TPU.

GPT-2 Author @ OpenAI. Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya. 2019


Do you want to read more results created from the GPT-2 + DSM* model?

Follow some examples of text generated thanks to a single word (“prefix”).

The generated text was not modified!

Prefix: "Society".

: Society ), be turned over to law-enforcement agencies for monitoring. A "Mile or Level" Approach to Classification of Individuals G [62] refers to a system of hierarchical categorical categorization that is part of DSM Organization's grand strategy to map and categorize humanity according to a set of categorical dogmas and limitations of reference. Within each of these groups, there exists an "A" according to which each individual takes part in a particular domain, brings up an awareness that includes all aspects of humanity (intellectual, emotional, perceptual, sociocultural), and embodies one of the six broad dogmatic concepts of pathological functioning (Stokes 2009:310-311). Speech must be distinguished from acute alteregration (presence, rigidity, or quibbling), a feature common to all of the schizophrenia disorders, which are subtypes of one another (Schizophrenia, pathological functioning, discourse). Gives meaning to a disorder of measurement enjoiid with frequent occurrence and continually depends on what system of labels is used to construct a symptom classification. Interprets the diagnosis as merely

Prefix: "Why is a raven like a writing desk?" (from Alice in Wonderland - Lewis Carroll ).

: Why is a raven like a writing desk? The blankness and lack of interest found in academic research are a feature of literate distress. In a general society, ordinary human sorrow and misery is regarded as an inconvenient and a source of suffering, and the individual's capacity to feel it is grossly under stood up as "not belonging." When, for example, a family member is feeling unusually sad, the individual may very well conclude that their grief is not truly "hard on the bum." . (SIC!)

Prefix: "BAZINGA" (Sheldon's gibberish).

: Bazinga, Bipolar I, 2nd Edition Ellis, M.D., M.P.H. Catastrophic traumatic brain injury. Caffeine addiction. Opioid use disorder. Chronic alcohol misuse caffeine addiction.