AI systems may be making a basic error when dealing with humans. They assume people are far more rational and strategically sophisticated than they actually are, according to a new study.