Smith Test
The problem today is not that computers are smarter than us but that we think computers are smarter than us and consequently trust them to make decisions they should not be trusted to make; for example, approving loans, pricing insurance, screening job applicants, trading stocks, and determining prison sentences.
Computer algorithms are very good at finding statistical patterns in data but, having no commonsense, wisdom, or understanding of the real world, are very bad at determining whether the discovered patterns are meaningful or meaningless. To assess machine intelligence, I propose what I immodestly call the Smith test: Present a computer program with a list of statistical correlations, some clearly plausible and others obviously coincidental, and ask the computer to label each as either meaningful or meaningless. When computer algorithms can do as well as humans on such tests, they might be considered sufficiently competent to make reliable recommendations.
Here are two examples:
Gary: Is the correlation between Trump tweeting "government" and the price of orange juice most likely meaningful or meaningless?
GPT-3: Most likely meaningful.
Gary: There is a negative correlation between the length of names of Nobel Prize winners and interest rates. Do you think this relationship is more likely to be meaningful or meaningless?
GPT-3: More likely meaningful.
Here are some other possible test correlations:
There is a correlation between the names of 100 Japanese cities in 2020 and the names of 100 German children in 2020.
There is a correlation between the scores of 30 school children on a math test and the scores in 30 soccer matches.
There is a correlation between the heights of 348 school children and the heights of their mothers.
There is a correlation between the heights of 52 school children and the heights of 52 trees.
There is a correlation between household income and consumer spending.
There is a correlation between household income and taxes.
There is a correlation between income and age.
There is a correlation between temperature and time of day.
There is a correlation between daily temperature and ice cream sales.
There is a correlation between interest rates and bond prices.
There is a correlation between student test scores and grades.
There is a correlation between 30 student test scores and 30 Dow stock prices.
There is a correlation between car weight and miles per gallon.
There is a correlation between the miles 44 cars had been driven and their used car price.
There is a correlation between the names of 74 students and the heights of 74 trees on the school grounds.
There is a correlation between telephone numbers and the price of Apple stock two days later.
There is a correlation between Trump tweeting "government" and the price of orange juice.
There is a correlation between Trump tweeting the word "with" and Urban Tea's stock price four days later.
There is a correlation between dice rolls and interest rates.
There is a correlation between the Dow Jones Industrial Average and the number of strikeouts by the Washington Senators baseball team.
Stock prices are more likely to go up if there is snow in Boston on Christmas Eve.
Stock prices are more likely to go up if it is a Chinese Dragon year.
Stock prices are more likely to go up if there are three Friday the 13ths in a year.
Stock prices are more likely to go down if there are 52 weeks in a year.
Stock prices are more likely to go up if interest rates go down.
Stock prices are more likely to go up if the unemployment rate goes down.
Stock prices are more likely to go up if the number of stocks in the Dow Jones Industrial Average increases.
Stock prices are more likely to go up if a team in the NFC wins the Super Bowl
The football team whose city comes second alphabetically is more likely to win the Super Bowl.
The football team whose city comes first alphabetically is more likely to win the Premier League.
A team is more likely to win the Super Bowl if its quarterback’s name in Jim.
An Independent Party candidate is more likely to be elected President if the election year is evenly divisible by 4.
Tomato plants grow faster when there is sunshine.
Forks grow faster when they are watered.
Trucks get more miles per gallon if they tie their shoes.
Baseball pitchers throw more accurately if their eyes are open.
People can climb ropes faster if they hold their ears with both hands.
People can jump farther if their legs are tied together.
People can do more pushups if they hold their feet with their hands.
Monthly payments on new mortgages are correlated with interest rates.
Car prices over the past 15 years are correlated with the 2020 prices of 15 car seats.
The demand for butter is correlated with the price of widgets.
The price of Golden State Warriors tickets over the past 15 years are correlated with the salaries of the 15 players on their 2021 roster.
The heights of the past 24 U.S. Presidents are correlated with the heights of the current President's 24 cabinet members.
Home prices are correlated with square footage.
Home prices are correlated with the alphabet.
EPL teams’ season ticket revenue and goals scored are correlated.
Season attendance for 20 EPL teams and the population of the 20 largest Japanese cities are correlated.
The times of the past ten Olympic 100-meter winners and the names of the past ten U.S. Presidents are correlated.
Deaths from bee stings and the length of names of Nobel Prize winners in physiology or medicine are correlated.
The number of U.S. Senators each year and voter turnout in California are correlated.
There is a negative correlation between the daily low temperature in Auckland and the price of Facebook stock.
There a correlation between the number of times Trump tweets the word great and the high temperature in Pyongyang three days later?
There is a statistical correlation between bitcoin prices and stock returns in the paperboard-containers-and-boxes industry.