Rewind’s new feature brings ChatGPT to your personal information

/ 

It’s a chatbot with access to everything you’ve ever done on your computer. That’s neat. But the real trick is making that work without compromising your privacy.

Rewind’s “ChatGPT For Me” feature is a chatbot with access to all of your stuff.

 Image: Rewind

The idea of a personal search engine is a powerful and enticing one. What if there were an app that knew everything about my meetings, my tasks, my browser history, my email, and everything else and could help me comb through it to find the stuff I care about? Sounds super helpful! Also, in an era of increasing surveillance and the ongoing monetization of our every thought and action, sounds like a terrifying hellscape dystopia! 

There’s probably a middle ground in there somewhere that works. That’s what Rewind.ai is trying to find with the feature it’s launching today. It’s called “ChatGPT For Me,” and it’s a GPT-4 chatbot that you can use to interact with all the information Rewind’s app collects about you. 

(Rewind, if you haven’t heard of it, is an app that launched last year for Apple Silicon-powered Macs. It logs everything you do on your computer — like, everything — and offers you a timeline of every meeting you’ve been in, every website you’ve visited, and everything you’ve typed or clicked on your machine. For between zero and 30 bucks a month, you get different features for searching, sorting, and interacting with all that history. How you instinctively respond to that idea — useful tool or dystopian nightmare — probably says a lot about what you’ll think of ChatGPT For Me.) 

With ChatGPT For Me, you can quiz a chatbot on your computer’s data however you want. What did I do last week? Who did I promise to call back today? What was that link about that thing that I was reading about? Like everything with ChatGPT and other chatbots, it surely won’t do that perfectly and will make some spectacular mistakes along the way. But it’s getting better fast.

This kind of service is about to become a big industry, by the way. In addition to Rewind, there’s also Lindy, an AI assistant that CEO Flo Crivello describes as “ChatGPT w access to all your apps.” Note-taking apps like Mem and Notion and Reflect are already integrating large language models, or LLMs. Google and Microsoft, the two companies with perhaps more access to our personal data than any other, are surely looking into how LLMs can and should interact with your private stuff. (Heck, Google’s been talking about this stuff since the days of Google Now.) Like it or not, the chatbots are coming for your stuff.

Dan Siroker, the CEO of Rewind.ai, gave me a demo of the new feature ahead of its launch. He and I had never met before, but we’d emailed a couple of times, and he’d looked me up ahead of our meeting. So he opened up the ChatGPT For Me window, a separate chat window inside the Rewind app, and typed “how do I know David Pierce?” A few seconds later, it spit back an answer: we had a recent interaction after I reached out to introduce myself. (True and true.) We scheduled a 30-minute Zoom on March 22nd, 2023 (true), and discussed a big launch happening at Rewind (true). It also linked to the calendar event for our meeting, my LinkedIn page from his browser history, and more. With 10 seconds and one paragraph, Rewind detailed our entire relationship.

When Rewind’s CEO asked how he knew me, ChatGPT figured it out right away.

 Image: Rewind/Dan Siroker

The specific way this works is really important. Rewind’s whole promise, the way it avoids falling into dystopia, is to make its pitch about privacy. The Rewind app itself never sends your data anywhere or stores it anywhere other than your device, Siroker says, and Rewind’s help docs even direct you to tools like Little Snitch to find out for yourself. 

When it comes to ChatGPT, though, Rewind has to send some of your data to the cloud in order to process your question and get useful answers. Siroker says the goal is to figure out how to do that in the least intrusive way possible. But what does that look like? “I’m obsessed with that question,” he says. “I think nobody’s figured out how to tightrope that balance between convenience and privacy in a way that generates enough value that people are excited about it, but doesn’t hinder their adoption in a way that feels creepy or scary.” He said privacy was the primary constraint in developing ChatGPT For Me, and finding the right balance is actually “the key technological insight here that will allow this to be adopted by not just tech enthusiasts, but mainstream users.”

What the company landed on was to take your ChatGPT prompt — something like “what did I do last Friday” — and first query your Rewind database for a list of the relevant information. That information is then sent to OpenAI’s ChatGPT servers as text; Siroker says no video, screenshots, or audio files are ever sent anywhere. “Then that query is processed, the text is sent back to Rewind, and we interpret those results and reference your local data.”

Make no mistake, this is still a big tradeoff. You may not be sending screenshots to OpenAI, but you’re still sending the text they contain. That means you’re trusting Rewind to pick the right things and send them in a responsible way — and trusting OpenAI to be a good steward and processor of that data. And it raises other even thornier questions: should you be able to upload information about private conversations you had with people without their permission? I won’t lie, it was slightly odd to see ChatGPT tell me about the email I’d sent Siroker. Rewind has been through this before, actually: when it first launched, it was using a cloud service to transcribe the audio on your computer, which didn’t go over well with users. So Rewind turned off audio transcriptions until it could do them on your device. (Which it does using OpenAI’s Whisper API.)  

Someday, large language models like GPT-4 will also be able to run locally on your device and won’t require any data to be sent anywhere

Someday, Siroker reckons, large language models like GPT-4 will also be able to run locally on your device and won’t require any data to be sent anywhere. That day may even come soon. But for now, he figures all he can do is be crystal clear about what’s happening and let users decide what they’re comfortable with. “I take the view that many of our users don’t even realize the risks when it comes to privacy,” he says. “So I feel like I have a higher obligation, as somebody who deeply understands how it works… I need to be a good shepherd of it.”

Rewind’s “search engine for your life” has always been a bit of a controversial idea, though Siroker says more people are coming around all the time. As the models get better, and as our digital lives become ever more distributed and complicated, we’re going to need tools to help us make sense of it all. In fact, Siroker says, Rewind users have been clamoring for ChatGPT integration for months. “We have the most valuable pieces of data that can be used to help answer questions,” he says. “They have an amazing model that will help you do reasoning on it and answer them. As long as we can strike the right balance between privacy and convenience, I think we’ve got a killer product here.”

Whether Rewind or any other tool does in fact strike the right balance is up to you. And as long as you get to choose, Siroker figures, we’re still doing okay.