GPT-4, the latest version of ChatGPT’s artificial intelligence chatbot, can pass high school tests and law school exams with a score ranking in the 90th percentile and has new processing capabilities that were not possible with previous versions.
Figures from the GPT-4 test scores were shared on March 14 by the creators of OpenAI, showing that it can also convert image, audio and video input into text in addition to handling “more instructions” creatively and reliably.
“It passed the simulated bar exam with scores in the top 10% of test takers,” OpenAI added. “In contrast, GPT-3.5 scores are in the bottom 10%.”
The numbers show that GPT-4 scored 163 in the 88th percentile on the LSAT exam – the test students must pass in the United States to get into law school.
A GPT4 score will stand you in good stead for admission to a top 20 law school and only a few marks above the reported score required for admission to a prestigious school such as Harvard, Stanford, Princeton or Yale.
The previous version of ChatGPT scored only 149 on the LSAT, placing it in the bottom 40%.
GPT-4 also scored 298 out of 400 on the Uniform Bar Exam – a test taken by recently graduated law students that allows them to practice as lawyers in US jurisdictions.
The older version of ChatGPT struggled in this test, finishing in the bottom 10% with a score of 213 out of 400.
As for the SAT Evidence-Based Reading & Writing and the SAT Math exams taken by US high school students to measure college readiness, the GPT-4 scored in the 93rd and 89th percentiles, respectively.
GPT-4 also excelled in the “hard” sciences, posting above-average percentile scores in AP Biology (85-100%), Chemistry (71-88%) and Physics 2 (66-84%).
However, AP Calculus scores are fairly average, ranking in the 43rd to 59th percentile.
Another area where the GPT-4 lacked was in the English literature exam, posting scores in the 8th to 44th percentile on two separate tests.
OpenAI says that GPT-4 and GPT-3.5 take their tests from the 2022-2023 practice exams, and that “no special training” is done by the language processing tools:
“We did not do special training for the exam. A minority of problems in the exam were seen by the model during training, but we believe that the results are representative.
The results also caused fear in the Twitter community.
related: How will ChatGPT affect the Web3 space? The industry answer
Nick Almond, founder of FactoryDAO, toward he has 14,300 Twitter followers on March 14 that GPT4 will “scare people” and will “collapse” the global education system.
Assessment theory was a big part of my life for many years. I was banging on about today came many years ago. I literally sounded like a resident crank at the time.
But… really this means nothing unless the invigilated evaluation is done from now on.
— drnick ️² (@DrNickA) March 14, 2023
Former Coinbase director Conor Grogan said he inserted Ethereum smart contracts directly into GPT-4, and the chatbot immediately pointed out several “security vulnerabilities” and explained how the code could be exploited:
I dumped my live Ethereum contract into GPT-4.
At the moment, they highlight several security vulnerabilities and show the surface areas where contracts can be exploited. It then verifies the specific way I can exploit the contract pic.twitter.com/its5puakUW
— Conor (@jconorgrogan) March 14, 2023
Earlier smart contract audits on ChatGPT found that the first version was also able to find code bugs at a reasonable rate.
Rowan Cheung, founder of AI newsletter The Rundownshowed a video of GPT translating a fake website hand-drawn on a piece of paper into code.
I just watched GPT-4 turn a hand-drawn sketch into a functional website.
This is crazy. pic.twitter.com/P5nSjrk7Wn
— Rowan Cheung (@rowancheung) March 14, 2023