top of page
Homepage

Gemini at the Wheel, Part 0: Why I'm Handing My Mac to AI

Writer: Fabrice R
Fabrice R
19 hours ago
3 min read

This series has one goal: get Gemini installed on my Mac and handling everything on my desktop, from files to websites to apps, and anything else I can imagine. Over the next few parts, I'll walk you through each step with VoiceOver, keeping it as simple as possible.

What I Want Gemini to Do

I want to automate the everyday things I do on my Mac. Here's one example that really appeals to me. I write an article and publish it here on Blind Sanity. Then I ask Gemini to go to X, write a short post about the article with a link, and post it for me automatically. This is now possible thanks to computer use, a brand-new feature Google announced for the Gemini Mac app on November 19, 2026. It lets Gemini handle multi-step tasks across your files, desktop apps, and websites. That's just one idea among many, and I'll share more examples as the series goes on.

A Quick Warning

To make this work, I'm going to give Gemini access to pretty much everything on my computer: my screen, my microphone, my files, and my apps. That might not be a good idea for everyone. An AI that can see and control your whole Mac can also make mistakes, or see things you'd rather keep private. Even Google warns that malicious content and broad app access can put your data and files at risk. Think about what's on your computer before you do the same. For my own purposes, I've decided it's worth it, and I'll share how it goes, good and bad.

A Note on My Setup

I'm running macOS 27.3 developer beta 2, so a few things may look or work a little differently depending on the version of macOS you have. Version 27.3 brings new, more detailed controls over what any AI app can access on your Mac. Permissions that used to be lumped together are now split into two settings, System & App Protection Override and Accessibility, and you can grant access app by app and even folder by folder. There's also a new menu bar indicator that tells you whenever an app is controlling another app.

If you're on 27.2, you still have the older, broader permissions, but not these new controls. They will most likely be available to everyone once macOS 27.3 is officially released, which shouldn't be too long, probably within a month to a month and a half. Whenever these settings come up during the project, I'll describe the steps. If you're still on 27.2, you can simply skip those parts. Keep all of this in mind if something doesn't match exactly. I'm checking each step as I go to make sure almost everyone can follow along without issues.

VoiceOver Basics You'll Need

This might be overkill for many of you, but a quick refresher never hurt anyone. These are the basic VoiceOver commands you'll need. We'll cover other shortcuts later, when they're needed. VO means the VoiceOver keys, Control and Option held together.

Go to the Dock: VO-D.

Open the menu for an icon or app: VO-Shift-M. This is the same as right-clicking.

Move to the next or previous item: VO-Right Arrow or VO-Left Arrow.

Go into a panel or list: VO-Shift-Down Arrow.

Come back out of a panel or list: VO-Shift-Up Arrow.

Open a link, press a button, or confirm: VO-Space.

What's Coming

In Part 1, we'll download and install Gemini. In Part 2, we'll open it, handle the permission prompts, including the new macOS 27.3 settings if you have them, and learn the basics of talking to it with VoiceOver. After that, we'll start putting it to work.

Stuck? Reach Out

If something goes wrong, if you get stuck, or if you're not sure what just happened, don't struggle alone. Leave a comment below or contact me, and I'll help you get back on track.

Comments


bottom of page