How do I know if AI is actually saving my business time?

You know AI is saving your business time when three numbers move, and “we use AI a lot” isn’t one of them. The numbers: how long you can be away before something breaks, what share of your recurring tasks run without a person touching them, and revenue per person on the team. Feeling productive doesn’t count. Most AI-saved hours get silently reabsorbed into more work, so the only honest test is a before-and-after on the tasks themselves. Baseline where the time goes now, automate, then re-measure the same list in 60 to 90 days.

Why can’t I tell if AI is helping?

You can’t tell if AI is working because you treat it the same as every other process in your business. A lot of small businesses don’t actually track their business KPIs. So they never know how well the business is actually doing and they have no data to make any decisions on. And this is across all areas of business and not just a single one. 40% of small business owners don’t regularly track profit margins, meaning they have no idea how much they actually take home. 46% of small business owners are unsure if their marketing actually works. Sure, these are specific parts of the business in marketing and revenue, but in general, I don’t think people use KPIs properly anyways. 58% of companies base at least half of their decisions on gut feel rather than data. If near 60% of companies aren’t using data to make decisions, then you can definitely tell they don’t really care about that data in the first place. So why would they even track KPIs?

However, it’s exactly the KPIs that will tell you if anything is helping. Yet it’s become a habit of not tracking anything. And this goes for AI as well. 66% of companies can’t establish an ROI metric for AI. And this basically means that AI has fallen into the same habit of not establishing any metric to make sure it’s actually working.

If we’re not actually measuring anything before the AI tools arrive, then there’s going to be nothing to compare against after the AI tools arrive. It’s the same for anything you implement in your business. If you’re not tracking the before and after, obviously you’re not going to get an ROI number.

So you really can’t tell if AI is helping specifically because you’re not tracking anything in the first place. If you were to track how long a task took without AI and you get an answer of two hours, you would be able to directly tell if AI made any impact on that by measuring how long it takes for AI to complete the task. If AI replaces you entirely, that’s two hours saved. If AI replaces 90% of it, that’s 108 minutes saved.

Why doesn’t “we’re using AI a lot” count as evidence?

This goes back to a previous newsletter of mine, where I talk about the “we use AI” note. The answer to this question turns out to be pretty similar. “We’re using AI a lot” doesn’t really have one meaning.

It could mean that you’re using stuff like regular ChatGPT and Claude in a lot of your work. This version of “using AI” isn’t really replacing work that much. It might save you a couple hours, but that’s really not enough to count it as impactful. In fact, roughly half of AI users, specifically chatbot users, report saving roughly 5 hours per week. Then those hours immediately vanish. 40% of it is lost to rework, and the rest is reabsorbed as more work. So you’re actually saving roughly 3 hours per week and just picking up more work later. So for the individual, the hours are real, but the business doesn’t really benefit.

The reason for this is that AI-assisted tasks still need a human to trigger them, check them, and finish them. The task gets slightly faster, but nothing actually left anyone’s plate. The freed time refills with more of the same work. The people with AI work faster, they take on more scope, but they end up working longer. The worker feels productive, but on the founder’s end, the payroll capacity and the week all look identical.

So “using AI a lot” describes activity, but real evidence is an outcome. We want actual tasks no longer requiring anybody. You’re never going to notice the five hours that disappear from one of your employees’ calendars, but you definitely will notice the specific task that is just handled without asking for it.

So “we’re using AI a lot” really doesn’t mean that much, everyone is “using AI a lot”. The real question is what would have to be true about our AI use for me to see real evidence that it’s working?

What three numbers actually tell you it’s working?

There are three main numbers that you can track to actually tell you if AI is saving your business time.

The first one is how long can you be away before something breaks? This is simply how many hours or days can you be unreachable before something breaks (if client work has to be you, you can exclude it). Most founders know about how long this number is instinctively. And usually most business owners can’t go a day without having to intervene. If AI is genuinely taking work off of everyone’s plates, then this number should stretch. But if AI is just making certain tasks faster, then this number barely moves. And you can’t really game this either. You could buy 10 more AI tools and this number won’t move. You need AI to genuinely take away a task from a person and no longer need anyone to complete it. This one’s pretty easy to track. You can simply pause on answering emails or texts or phone calls and see how long it takes one of those to reach you again.

The next number to track is what share of recurring tasks run without a person touching them? This is simply how many tasks AI is doing for the team, nobody intervening at least until the very end to check the output. Again, this is the direct number of how much work AI is taking off of people’s plates. If AI is handling one task, that’s much less time saved than AI handling five to ten plus tasks. And you can kind of game this one a little bit. So I would only count tasks that get 90 to 100% done as something completely off of somebody’s plate. I say 90 because there are some tasks that genuinely need the output checked before they get shipped. This one’s an easy check. You should know how many tasks get done on a regular basis and you can directly check with your team on who’s doing what and which tasks AI currently handle.

The third number to check is revenue per person on the team. This is the total revenue divided by headcount, including you. A team of four people at $600K per year is $150K per person. This will directly tell you if time saved becomes money or if it just evaporates. If AI is actually removing work from people, then either of two things show up. Either the same team can handle more clients, meaning more revenue, or growth happens without you having to hire normally. If AI is taking away tasks, but nothing changes, then the AI is not working. And this one can’t be gamed or gimmicked either. If you spend more on AI, it will either make you more money or you will stay the same until the tools produce. You should be able to get your current revenue and divide it by your headcount pretty easily. To check this number, however, you need to rerun the same division each quarter. If you have no AI in quarter one, implement it in quarter two, you should see more measurable return starting to show up in quarter three and four.

How do I measure this without a data team?

You definitely don’t need a data team to do this or even a person in charge of the data to do this. This is very easily done by the founder or business owner. You know what needs to get done each week, so you should be able to create a list of recurring tasks that get done. With this, we are directly measuring the second number of the previous section, the share of recurring tasks.

Write down on the document the list of recurring tasks that get done each week. For each recurring task, write down the following: Minutes per week it takes to get the task done and who currently does the task. Once you start implementing AI to handle tasks, you can switch the who does it section to handled by AI or runs untouched, whatever you like. And then total at the bottom, how many minutes handled by a person and how many minutes given to AI. This list becomes your baseline every time you implement a new AI solution.

Rerun the same list every 60 to 90 days. Every time you rerun it, two things should be changing, the number of tasks handled by AI and the actual number of minutes saved by AI.

Every time you see a big jump in this list, it might mean it’s time to check the other two numbers from the previous section. If you implemented a whole bunch of AI solutions and you can see that they’re actually saving you time, that might mean you get more time back as the founder when you leave, or it might mean there’s more revenue per person on the team.

Also, in the same list that you build, make sure to be tracking if any AI solution actually increases the time it takes to complete a task. If you’re counting the failures, you should probably revert that task back to somebody and re-evaluate the AI solution to really see if it’s actually needed.

Keep this list simple. You don’t want to be tracking 50 to 100 tasks at once, you don’t want this to be super complicated and annoying. Track 15 to 20 at a time on one page in a spreadsheet somewhere. Once you have confirmed that the 15 to 20 tasks are handled by AI, you could add more. However, the 15 to 20 should be high leverage, impactful tasks that should take real thought and be implemented well before moving on.

What does “it’s working” actually look like?

You’ll know it’s working when at least one of the three numbers that you track is moving in the positive direction. If you can step away as the founder and nothing breaks, something is working because you couldn’t do that before. If you can see a higher percentage of recurring tasks handled by AI and handled correctly, then something’s working because your team is able to take on more clients and do either less or different work. And if you’re seeing more revenue per person on the team, then you know something’s worked because AI was able to allow the business to take on more clients and actually grow.

Basically, it working means AI has become an employee. AI is handling digital tasks in a manner that is actually saving everybody else time. This is your morning briefs tracking the business, proposals drafted, follow-ups tracked, meeting notes captured into action items, and a whole bunch of other things that no employee should be handling. All of your actual employees are handling things that need to be dealt with by a human. Things like client relationships and client creation, outreach, networking, or business development.

And AI shouldn’t be creating any work either. If you have implemented AI and it’s just rearranged where the work is being done or it’s even added more work to what you’re already doing, then you can definitely say it’s not working and you need to reevaluate.

Where do you start?

If you’re thinking about implementing AI into your business, you need to start with everything before AI. You can’t track what you don’t measure, so the first thing you need is a baseline of what goes on in the business. So I encourage you to start creating that list of business tasks and start actually tracking how much time it takes to actually do all of your major business tasks. So once you start implementing AI for any one of those, you can directly measure it.

If you’re wondering which AI solutions to implement first, that’s exactly what my task audit does. You simply tell it what your recurring tasks are per each department and how long they take, and it shows you your three biggest wins to start implementing AI for. It also tells you the percentage of your business that could be automated and about how many hours you could be saving.

Ready to find out where your time is actually going? Get your free Task Audit, a 3-minute quiz that maps your recurring tasks, gives you your task automation %, and hands you your top 3 quick wins.

Photo by William Warby on Unsplash