Professor and Chair, Department of Linguistics, University of Sydney; Author, The Utility of Meaning
They Don't Think Socially

When we think of machines that think, we usually think of "thinking" in the pocket-calculator sense of the word. Input, crunch, output, bam. There's your answer. We love these machines and we need them because they think in ways we can't: consistent, exhaustive, and fast. But the reverse is also true. We think in ways they can't. The machines are not concerned with your state of mind. Their thinking is not emotional. They don't relate to you. When your computer crunches your tax return and gives you a number, it doesn't spare a thought to how it should spit that number out; fast or slow, straight-up or hedged. It won't have wondered whether its answer is the one you want to hear, and anyway it literally couldn't care.

The thing is, machines aren't into relationships. Yet for us, relationships are pretty much all that matters. When we think, we don't just calculate, we worry about the social consequences. How might this decision affect others? How will it impact the way we interact next time? What will they think of me? Machines don't think like this. So there should be no illusions that we could socially interact with them in any meaningful sense. Human interaction is built upon a kind of psychology that only our species has mastered. Our trick is that we can fuse with each other socially by making commitments to shared goals and shared reasons for action. True cooperation entails the formation of a "corporate person," however fleetingly. We think, feel, and act together, and thus effectively as one. This allows us not only to succeed as one, but we can fail together too.

Machines comply, but they don't cooperate. For that, they would need to be capable of committing to common reasons for action, common goals, and shared stakes in the outcomes. We get along well with our thinking machines because they nicely complement our powers of mind. So let them be brute thinkers, and leave the relationship thinking to us.