“Perhaps the professor could use your computer,” Dr. McCoy says in an iconic Star Trek IV: The Voyage Home scene. “Please,” an engineer replies. Determined, Scotty approaches the screen and says, “Computer… Computer!” expecting a response from a 1986 PC. When nothing happens, Dr. McCoy hands him the mouse, which Scotty tries to use as a microphone: “Ah! Hello, computer?” The engineer, puzzled, then instructs him to use the keyboard. “The keyboard… How quaint!” Scotty adds.
This Start Trek scene is as iconic as it’s visionary. In fact, it serves as an ironic commentary on what many sci-fi movies, both before and after The Voyage Home, assumed about human-machine interaction. In these films, humans don’t type or click around on screens to communicate with machines. They also don’t tap on a phone screen.
For instance, you don’t see Matthew McConaughey talking to robots TARS and CASE in Interstellar like that. Nor does HAL 9000 in 2001: A Space Odyssey say, “I’m sorry, Dave, I’m afraid I can’t do that” via a typed message on a screen. The AI character says it aloud. In these scenes, humans and machines converse naturally. Now, the vision of a future like this is gradually coming to fruition with innovations such as OpenAI’s GPT-4o model and its advanced voice mode.
Anthropic is also contributing to making this future even more tangible. On Tuesday, the company introduced Computer Use, a tool that allows its AI model, Claude, to interact with the user’s computer. The company has demonstrated the functionality in a controlled environment but emphasizes the potential for similar capabilities to become available in the near future.
Anthropic says that the new API can convert prompts into executable commands for the computer. The AI achieves this by taking screenshots to analyze and understand the layout of the screen. Here’s an example of how the new tool works:
- You type the prompt, “Open Firefox.”
- The AI looks at what appears on the screen and examines it to locate the Firefox icon.
- It identifies the icon and automatically moves the mouse pointer to it.
- It simulates a mouse click on the icon to open Firefox.
- And just like that, Firefox appears on the screen.
This straightforward interaction can lead to much more complex tasks. For instance, you can ask Computer Use to create a web page with a late 1990s design, search for information about yourself, fill out job application forms, and even order food.
Currently, computer users primarily use a keyboard to enter requests. However, it’s inevitable that voice input will become the norm. This transition is already evident in demonstrations given by Anthropic employees, who speak their requests and then confirm them by clicking the Send button on the message that the machine has “heard.”
Voice will gradually take over. Anthropic isn’t alone in this. Several months ago, OpenAI, with its GPT-4o demos, suggested a future where voice becomes the preferred mode of interacting with a computer. Many of the demos drew parallels to the film Her, where a man falls in love with an AI virtual assistant, indicating that we might be heading toward a similar future.
As technology progresses, the mouse, keyboard, and touch gestures may become obsolete. When that happens, if someone asks you to use the traditional peripherals, you might find yourself quoting Scotty: “How quaint.”
Image | Vitaly Gariev | Paramount Pictures
Related | By Rewarding AI, the Nobel Prize Has Become a Popularity Contest. The Truth Is, It Always Has Been
View 0 comments