AI Software Costs Are About to Explode: Is Your Ecommerce Brand Ready?

Welcome to the Ecommerce Finance Podcast. I'm Stephen Brown with LedgerGurus. Today I have Stacy Walker, Director of Growth here at LedgerGurus, and we are going to discuss managing AI software costs.

Stacy, how's it going?

0:21 Stacy Walker

Good. How are you doing, Stephen? Thanks for having me on.

0:22 Stephen Brown

Good. We just got back from a conference. It was an accounting conference, but there was a lot of discussion around AI and AI software. And one of the things that I was thinking about is, what is this going to do to costs?

So should we get into it?

0:38 Stacy Walker

Let's do it.

0:40 A history of software pricing

Stephen Brown

Okay. Stacy, do you want to know a little bit about the history of software pricing?

0:45 Stacy Walker

Yeah, I think it's a really good place to start. Do you want me to take that or are you going to take that?

0:50 Stephen Brown

I'll jump in.

I used to be in enterprise software going back to the 90s. So I'm a software veteran before I came over to the outsourced accounting world. And there are a number of licensing models.

So originally there was what was called a perpetual license plus maintenance. You would get the software, you would install it on a server or a desktop, and you'd use it, and you'd pay annual maintenance so you could update the software. And pricing was fairly consistent. You saw a lot of people sometimes that would just use old software forever rather than paying the upgrade prices.

1:25 Stephen Brown

And then we moved into the era of SaaS, and SaaS shifted to a subscription model where you're paying a monthly or sometimes an annual fee. And you are running on the service provider's servers, or often a lot of these SaaS companies are running on top of the big infrastructure companies. So either Amazon being the most popular one, Amazon Web Services, you have Microsoft Azure, you have Google Cloud.

And there was a little bit of cost there. Before, a software vendor's costs were essentially sending out the CD, or whatever the cost of the download was. Almost no cost of goods.

2:08 Stephen Brown

With SaaS there's the infrastructure cost. Well, those infrastructure costs are through the roof with AI. And I'm going to make an argument that what we're seeing today is about to shift very dramatically in the future.

2:22 Stacy Walker

Okay, let's talk about why. So if it's going to make a shift, where's it going to make a shift?

But first, as far as SaaS is concerned, I feel like where we're at now, for the software we use, we pay an overall fee every month for the software, plus every user has a seat. So we're paying for every user as well. So that's kind of where SaaS is sitting at this point, right?

2:47 Stephen Brown

Yeah. I mean, there have been a couple of licensing models historically. For a lot of things it was per computer. In the era of SaaS you saw a lot of it move to per user.

We've seen in the ecommerce space a decent amount of consumption models, where maybe it was a certain number of contacts in your Klaviyo system. Or, like in the case of Shopify, there's a billing against sales. So you see a lot of revenue for sales model.

And this consumption model is where I think AI is going, just because of the underlying costs of the infrastructure.

3:29 Stacy Walker

So where do you think it's going to go? Right now we're paying per seat, I feel like.

3:33 Stephen Brown

Per user is kind of how it is. I think you're going to see things go to a per user and or consumption model, where they're looking at the amount of tokens being consumed by whatever the things you're doing in AI.

3:50 Stacy Walker

Okay, so tell me more about tokens. Maybe for people that aren't aware of even what they are, give a really simple explanation.

3:59 The cost underneath AI, and the drug dealer model

Stephen Brown

Before I do that, let me talk a little bit about the underlying costs that these AI companies are consuming.

I think in the era of SaaS we have gotten used to a near zero cost of the underlying hardware. There's always been a hardware cost. But Google has really driven down the costs of search to where it was very, very low, and the cost from advertising could cover the infrastructure costs.

4:29 Stephen Brown

What you're seeing with AI and these AI models is how intense the computation is.

The legacy hardware was built on things that we all know of. CPUs, the things that are in your laptop. But the AI uses GPUs, and GPUs tend to be much more energy intensive. There's also a lot more memory demand, we're seeing a memory shortage. So the amount of compute to do these AI computations is astronomically larger than what it is to do traditional computing calculations.

5:16 Stephen Brown

And so you're seeing all of these AI data centers, you're hearing in the news about the electricity that they're going to need to consume. We are here in Utah, and there's a huge controversy about a huge data center that will need more electricity than all of the state of Utah is producing today.

5:37 Stacy Walker

And water.

5:42 Stephen Brown

And water, you know. So all of these costs add up.

And behind that, coming back to your question around tokens, there's this concept of tokens. Tokens are a way to measure how much AI is computing. And a lot of what they do is looking at the amount of characters in and out. So tokens are kind of like the atomic unit of AI computation. And it's very easy, as we've learned, to consume more and more tokens as you get into AI.

6:10 Stephen Brown

The problem I think we have is, there are two fundamental problems.

Number one, these AI companies are being fueled off of venture capital. So you read in the news about how they're losing massive amounts of money. Even though they're making a lot of money, they're losing even more. So we as consumers are getting a subsidized cost, which means we're not paying the real value of those products. And so at some point there's going to be an equalization.

6:36 Stephen Brown

The second thing I would think about is the drug dealer model.

6:44 Stacy Walker

The drug dealer model? Okay, tell me more.

6:47 Stephen Brown

If you think about it, your $20 a month, or $30, or $25, whatever it is, is not covering the costs. And yes, we see OpenAI wanting to get into advertising, they're starting to get into advertising and find new revenue models. But the cost of using these products is higher than the cost that we're paying. And so they're doing the drug dealer model. They're getting us hooked with this super low cost model.

7:20 Stephen Brown

And I think, as you see these companies go public, which by all accounts it sounds like they're going to this year. We saw SpaceX, which includes the AI capabilities, go public with a phenomenal IPO. There's a race to go IPO between OpenAI and Anthropic.

And we're already seeing a monthly subscription plus consumption model. So if you look into Claude or a lot of these, you'll see there are limits. And I think that's the way of the future. I think this base subscription is going to give you a certain amount of tokens and usage, and at some point you're going to have to pay for that overage.

7:59 Stacy Walker

So the trick is they're getting us all hooked, so that then when it's time to pay for tokens, it's a small price to pay for the efficiency that we've created using the tools.

8:10 Stephen Brown

Correct. And we're seeing that at the API level, they're pricing per million tokens. And it's a lot like what a developer does on Amazon Web Services or Microsoft Azure, there are consumption models. It's like how much storage.

And I think this is going to be the way of the future. So as people are going headlong into using AI tools and building out agents and whatnot, you're going to see more consumption.

8:38 Stephen Brown

We're seeing this a lot with AI-enabled software. There are really only a few AI companies, right? It's really Anthropic and OpenAI, they're the big two. And there are a couple of others, Perplexity. But everybody else is just making API calls to either OpenAI or Claude.

And so they're going to be doing the same thing. They're going to be like, here's my AI powered software, and by the way, you're going to have to start paying a consumption fee if you want to use more than this base level of capabilities. And we've seen in some of our software where we use those AI base limits and we blow through them in a hot second.

9:20 What the bill will look like, and choosing models

Stacy Walker

Yeah. So what do you think the bill is going to look like each month?

9:37 Stephen Brown

That is the big question.

I think it could be enormous if businesses aren't starting to be really thoughtful about how they approach the usage of AI tools.

Coming back to my earlier example, a study found that 68% of software vendors now charge separately for AI features or gate them in premium tiers. So the market is already rapidly moving to this place.

10:05 Stephen Brown

I think two things will happen. I would not be surprised to see the base price of these tools go up, maybe a little bit, just to account for their costs. But I also think you're going to see more limits.

It's this drug dealer conundrum, right? You've got to give them enough so that they get hooked, so they'll start building stuff. But you don't want to give them too much. You've got to make sure you limit it so that they start having to pay for their usage.

10:35 Stacy Walker

Okay, so where do you think all of this is headed then? The hybrid pricing will take over?

Do you think people will suddenly decide, it costs so much to use AI, I'm not going to do it anymore, and they'll go back? Or are they going to realign their finances so they can invest more in AI?

10:55 Stephen Brown

I think hybrid pricing is already here, and it's going to become more apparent over this next year. And I think usage is going to be the key thing.

Do I think that will cause people to run away from using AI? No. I think what it's going to require is people to be more thoughtful about their consumption.

11:39 Stephen Brown

And part of it too, I heard an interesting conversation recently. If you look at, take Claude for example, Claude has multiple models, and each model has different costs.

So we had the Fable model, which was here for a hot second and then disappeared due to the regulations. But right now I'm looking at Claude, and you've got Haiku, Sonnet, and Opus. And who knows, maybe we'll get Fable or something like Fable in the future. And when Fable came out, I remember them saying, hey, just beware, this is going to take about twice as many tokens as you're used to.

12:10 Stephen Brown

I think there's going to be a philosophy for how you use this AI, to be like, what are the things that the lower cost models are tuned for? And what are the things that you need the higher cost models for?

And the analogy I heard is it's almost like with labor. What are the things that I can give lower cost labor, aka the lower cost models? And what can I give to the more sophisticated, higher paid employee, aka the newer, more expensive models?

12:37 Stacy Walker

Yeah, depending on their role, that would make sense.

So how are teams going to, we're seeing a little bit of this now, but how soon do you see AI becoming more of a sales channel and more of a sales and marketing engine? Right now it kind of is, we're already seeing that shift, where SEO, I don't want to say it's dead because that's dramatic, but it's definitely shifting some to GEO.

So how soon do you think that becomes a bigger player in the space? When are we buying from AI?

13:11 Stephen Brown

I think we're already seeing that. It's going out slowly.

OpenAI has launched marketing. I haven't seen a lot of people using it, but I've been reading a little bit about it. They've facilitated some checkouts, although they seem to be running into some bumps.

To me those are revenue models that will help subsidize the costs of AI. I still think that the underlying subscription and usage costs are going to go up, despite additional revenue streams that will emerge.

13:53 Stacy Walker

Interesting. So what's the difference between people who adopt AI, and I'm going to say adopt it like we're kind of using it, and people who go all in on it? Is there going to be a huge differentiator in the market?

14:10 Stephen Brown

Well, I think we're definitely seeing, I mean, you tell me. Do you guys use AI for critical workflows today within the growth team here at LedgerGurus?

14:19 Stacy Walker

Yeah, for sure. I think everyone on our team is using AI every single day.

14:29 Stephen Brown

If we pulled that out, it's going to increase the time to do things, right?

And I know in the DTC business that I co-own, Solé Toscana, we have replaced certain labor functions with AI tools. So I don't think increasing costs are going to force us to go back that extreme. I do think there's going to be a reckoning for us to really look at what is moving a business forward and what is just nice to have. Or force people to really optimize, maybe by models, and say, hey, could I do this thing with a lower cost model and save some dollars?

15:02 Stacy Walker

Do you think it'll be easy for people to determine their ROI with all of their AI functions?

15:17 The cost reckoning and who has already shifted pricing

Stephen Brown

Ooh, I think it's going to be hard.

I wouldn't be surprised if we see some software vendors pop up that are around AI cost management. I'm sure they already exist, I just haven't heard of them.

Because the analogy I'd put for ecommerce businesses is Shopify plugins. I've seen over the years people's Shopify plugins just bloat. Just bloat, bloat, bloat, until they're like, crap, I've got to take a look at these. Well, I think AI costs are going to be that on steroids.

16:01 Stephen Brown

And so I think there's going to be a reckoning, and I think it's going to be kind of hard, until either the AI software builds better visibility and understanding of consumption, or third party tools do. I'm not sure how well third party tools could look at that, but I do think that's going to become a bigger issue.

16:21 Stacy Walker

Yeah, I could see that.

16:24 Stephen Brown

Now, the one caveat I'd put out there. Technology has historically tried to drive down their costs, or maintain their costs. I personally can't remember a time in my career where we've seen underlying technology costs blow up like we're seeing.

I do think that you're going to see them really focus more on cost efficiency and models. I do think you're going to see them look at hardware efficiency. NVIDIA, who is dominating the GPU and the AI infrastructure, I think they're going to have increased pressure to make their underlying hardware more efficient.

But we could be going through a period of cost expansion before these industries get back to what they're good at, which is finding ways to drive down costs of compute.

17:31 Stacy Walker

Okay. Tell me what's going on with Microsoft right now. Microsoft's making a little shift.

17:37 Stephen Brown

Yeah, so I'll give a couple examples of pricing.

Microsoft announced that GitHub Copilot was going from flat premium request billing to usage-based, effective June 1st. OpenAI made a similar shift on April 2nd in a variety of their plans. I just think this is something you're going to see across the board. So you're seeing a lot of announcements here.

18:05 Stacy Walker

Yeah. So switching to usage, I'm going to make the assumption that we'll still be paying for our seats and then usage for each of the seats. How soon do you think the pricing will change?

18:18 Stephen Brown

I mean, pricing is changing.

18:19 Stacy Walker

Use your crystal ball and tell us.

18:22 Stephen Brown

I think by the end of 2026 you're going to see increased pressure on pricing.

You already see it with Claude, where they have their limits, I mentioned this earlier. But Klaviyo has this with their customer agent AI. Gorgias, which is customer service, is charging on a usage-based model.

Right now with Shopify, Magic and Sidekick are bundled in together, but I think it's only a matter of time before, as they're paying larger bills, they're passing that down to us. We're seeing this across the board in a lot of these tools.

19:10 Stephen Brown

And I think it's just because, again, as these companies go public, it'll be really interesting to see, are they profitable? The cost of building up these data centers is just enormous. And as you're hearing about the electricity costs alone, right? We didn't hear about traditional SaaS infrastructure data centers consuming as much electricity as the locations they were in.

19:44 Stephen Brown

And I think that's a big component of it. So if the cost of electricity can go down, that could be a good thing. But my concern is, as there's more demand for electricity, if we don't see a comparable amount of new electricity capacity, you're going to have a supply and demand effect where if anything these costs of compute could go up.

20:10 Where AI sits in the stack

Stacy Walker

Okay, so where do you think it makes the most sense for AI to sit in the tech stack of an ecommerce brand?

20:20 Stephen Brown

Well, I think it sits in a couple of places.

AI has bled into all of the software. Everybody's doing AI and they're using it. And I'd say some people are doing more interesting things than others. So the traditional SaaS companies who are now using AI features, they're going to be under pressure to do those things more efficiently. And the AI infrastructure companies likewise are going to be under pressure to do what they do more efficiently.

20:50 Stephen Brown

I mean, let's be honest. A lot of us that have gone AI first, I don't know about you, but I do a lot of things using Gemini or ChatGPT or Claude that I used to be able to do in Google. And Google got that transaction of a search result really cheap.

So in one part you could argue that some of what's going on really needs to function more like a search engine, which is really low cost, really simple, quick results.

21:36 Stephen Brown

AI tends to be really noisy. And I don't know about you, but I get sometimes annoyed by how verbose AI can be. I'm like, dude, just give me a paragraph.

So I think the AI companies, as well as the SaaS software out there, it's upon them to consume tokens more efficiently. A lot of us don't want paragraphs. We want a sentence, a phrase. I'd rather have a really concise answer with a reference, akin to search, than 10 paragraphs explaining why things are the way they are.

So I do think there's going to be pressure from those companies to do a better job of being more cost efficient.

22:12 Stacy Walker

Yeah, give us just a straight answer.

Do you think there's any world where people shift back to a Google type mindset, where they're going back to Google? I mean, Google's using AI for their answers anyway, but.

22:35 Stephen Brown

Google has the AI results. I've found myself still going to Google just out of muscle memory, because it's fast, because they have the Gemini results right there.

22:49 Stacy Walker

I feel like the Gemini results are shorter though. If you ask a question in chat you're going to get a much longer response.

So I wonder if people will start thinking the same as you and I. I just need a quick answer, and they're going to take those things they need a quick answer to away from AI, back to Google, which is still using AI, but it's just a much shorter response. And that saves money on tokens, maybe. I don't know.

23:17 Stephen Brown

Well, a lot of these tools are building model routing into it. They've got auto model detection. And I think you'll see more of that, where unless you explicitly specify use this model to answer my question. I mean, the average consumer probably can't tell you what the different models do. They just think, okay, the bigger one's better.

So I think they will build routing into the models, which will help optimize costs. They're already doing that. I think you'll see more of that.

23:42 Stephen Brown

The software companies that are implementing these models need to make sure that they're leveraging that as well. And so that's not necessarily going to be the ecommerce company.

I think what the ecommerce businesses are going to have to look at is, they're going to have to really scrutinize consumption and pressure their vendors to help them understand how they're using AI, so that when they're exceeding cost they have a better insight on how they can be cost optimized.

24:20 Stephen Brown

I think the AI companies are doing a poor job of communicating how things are working. Like if I go into Claude and I look at my, I'm going to go in here right now, what they give me is a plan usage limit, which is current session. They're giving me weekly limits, but they're not really telling me why I'm hitting those limits.

24:50 Stacy Walker

People are going to want to know.

24:51 Stephen Brown

Yeah. And I think what you're going to want to see, especially as people are building agents, or OpenAI's custom GPTs and Claude's projects, they're going to want to know, hey, which one of these things is consuming my credits?

25:05 Stacy Walker

It becomes like an electricity bill.

25:16 Stephen Brown

Is it Cowork? Is it Code? Is it a project? Is it a GPT? Is it an agent?

And even then I would want to know further. Let's say there is one project in Claude that's just burning through credits like crazy. I'd still want to know further. I would want to understand how that usage is occurring, so that I can understand how to optimize.

And right now I don't feel like there are good insights in any of the platforms around usage. It's just like, here's your limit, here's what you are. But I do think you're going to have to think about that as you start to move towards higher consumption bills.

26:02 How to evaluate AI spend

Stacy Walker

Okay. How would an ecommerce brand evaluate their spend on AI?

26:14 Stephen Brown

Let me think about this for a second. This is a good kind of wrap up.

Right now, coming back to my previous point, it's a little bit tough. So what I want to be looking at, really what I can look at, is really straightforward. How much am I spending on software as a whole? But how much am I spending by vendor? And if I see that cost creeping up, then I'm going to want to get to as much detail as I possibly can.

26:45 Stephen Brown

And I believe that very easily you're going to see software costs creep, not just creep, but expand, and could easily exceed any savings that they're producing.

So the easiest way to do it from a financial standpoint is to look at, what is my spend in this category? What is my spend by vendor? As you see that creeping up, start to dig in with whatever analytics you have to understand where the spend is going. And then you need to start making those assessments. Is the cost of using this outweighing the benefits?

27:18 Stephen Brown

And this isn't something we've had to do in business for a long time, because for the most part software was fairly low cost and the benefits were fairly clear. But I think as we're moving into this higher cost era, this skill is something that every ecommerce business, and every business period, is going to have to start thinking about.

27:41 Stacy Walker

Yeah. And earlier you mentioned how on Shopify, all of the plugins that come in, all of the add-ons that people get.

As AI becomes adopted more, as people use it more, like you mentioned it's going to be in all of the software. And I can see people thinking, I need this software, it has AI, I need this software, it has AI, until they end up in that same situation with Shopify. They have all of this tech, and it all has AI. But at the end of the day, is it all providing a benefit to the business, or have we inadvertently just added a pile of expenses that aren't producing any return? So that's something I think people also need to be aware of.

28:24 Stephen Brown

And at the end of the day, when I think about software, I think about two things.

Number one, is the software producing more revenue? There's not a lot of software that clearly drives revenue. I mean, there is some. You could argue Shopify is a revenue component for an ecommerce business, Klaviyo, email marketing. But even then you've got to really look and say, I'm spending more on the AI usage, did it increase sales? Yes or no?

28:53 Stephen Brown

The other question you have to ask is, is AI decreasing costs? And usually the costs are in the form of labor. Am I able to do more with software than I am with people? Or are the same people getting more done, producing more, that is having a greater bottom line impact than ever?

And I just don't think that a lot of people have been thinking this way, but they're going to need to start thinking this way more than ever.

29:33 Stephen Brown

I will predict that this is going to become a huge issue as early as the end of the year, but I would not be shocked if it is a big issue in 2027.

29:42 Stacy Walker

Yeah, I agree.

29:44 Stephen Brown

So this is my mid-year prediction. I usually do my predictions at the end of the year. I will probably make this a big prediction going into 2027, but I'm calling it now, that this is going to be one of my big ecommerce finance predictions for 2027. That AI costs will become an issue that every brand needs to be considering.

30:05 Stacy Walker

I agree. Totally.

30:07 Stephen Brown

All right, let's call this a wrap. Thanks for joining me, Stacy.

30:11 Stacy Walker

Thanks, Stephen.