‘Many customers are reporting the OpenAI models are not delivering high levels of accuracy,’ Salesforce CEO Marc Benioff said.

Salesforce CEO and co-founder Marc Benioff railed against copilot artificial intelligence products offered by Microsoft and other vendors, touting his company’s agentic approach as one that delivers better accuracy while maintaining the importance of the vendor’s services partners.

In response to a question from CRN during a press and analyst event, the CEO of the San Francisco-based enterprise software said of Salesforce’s 12,000-strong partner ecosystem in delivering AI that “it’s critical that we enable and train them and give them this extension.”

“A lot of them are already next generation Salesforce platform users,” Benioff said. “These are the people using the product. And it (new AI features) just appear inside the platform. It’s not some new thing that they’re going to buy or add on or plug in or whatever. It’s going to appear from within.”

[RELATED: Salesforce Dreamforce 2024: The Biggest News]

Salesforce Dreamforce 2024

Benioff’s view is that providing Salesforce customers a full AI platform without adding on or plugging in AI products will result in better accuracy and lower hallucination. “We believe that this is the key to making it work,” he said. “This is what AI is meant to be.”

Microsoft has been most visible with its copilot brand of AI virtual assistants, but other vendors have used the term as well, including Salesforce itself with Einstein Copilot. Hewlett Packard Enterprise, Spot AI and SymphonyAI.

CRN has reached out to Microsoft and OpenAI for comment.

Benioff pointed out that during a virtual Microsoft event Monday on the next wave of the tech giant’s AI updates, Microsoft CEO Satya Nadella said AI was becoming “more capable and even agentic” and models were becoming “more of a commodity”–observations Benioff has also made.

Benioff directly attacked Microsoft and ChatGPT maker OpenAI, which has received billions of dollars in investment from Microsoft and helped power some of Microsoft’s AI offerings. OpenAI CEO Sam Altman notably spoke at Salesforce Dreamforce 2023.

“Many customers are reporting the OpenAI models are not delivering high levels of accuracy and resolving even basic customer service issues for them,” Benioff said. “The lack of grounding, the lack of access to the metadata, the data itself, the sharing model, all of the components of a platform that are then needed to be able to achieve this kind of level of accuracy.”

He also said that Salesforce’s Agentforce AI offering “is outperforming OpenAI on Azure in cost, time to value and accuracy.”

Benioff attacked Microsoft’s business practices, commenting on the tech giant’s treatment of Slack, which Salesforce bought in 2021.

“The European Union wrote this interesting statement about how they run their business, about Slack and some of the things that they did inside their company when these two entrepreneurs were trying to build this company and what actions they took and how they went after them,” Benioff said. “You might want to read it. It’s very interesting. A lot of insights. It reminded me also, on Netscape was another one. They had a very interesting document about that. … It’s a very interesting business philosophy there” at Microsoft.

Benioff also expressed some concerns about the perils of AI. “I hope we’re on the right side of history here,” he said. “It’s a very high-wire act. . … I am sure that there’s going to be good stories and bad stories. I’m sure that some of this is going to work, and some of it is going to go horribly wrong. I hope the horribly wrong stories are not as bad as I have them in my mind that they could be. Because they could be horrible. But they also could be magical. And I’m not sure what’s going to happen.”

Here’s more of what Benioff had to say during Dreamforce across his keynote and during a press event.

Benioff Calls Copilots ‘Nasty’

Through Dreamforces for the last especially 10 years, we’ve been talking about the emergence of AI and how AI has been a critical part of our future. And we introduced you to (Salesforce AI offering) Einstein. We did so much in deep learning. … We even invented prompt engineering at Salesforce. … And then we moved into this copilot world.

But the copilot world has been kind of a hit-and-miss world. … Customers have said to us, ‘Hey, I have got these copilots, but they’re not exactly performing as we want them to. We don’t see how that copilot world is going to get us to the real vision of artificial intelligence, of augmentation, of productivity, of better business results that we’ve been looking for. We just don’t see copilot as that key step for our future.’

In some ways, they kind of looked at copilot as the new Microsoft Clippy. And I get that. But it was pushing us. … And we are now, really, at that moment. … I read a Gartner report last week about those nasty copilots, and they were spilling data all over our customers’ floors. And we’re like, ‘That is no good for our customers.’ That’s not the kind of businesses we run. … We all have struggled in the last two years with this vision of copilots and LLMs (large language models) and how are we putting it all together.

So why are we doing that when we can have Agentforce. We can move from chatbots to copilots to this new Agentforce world.

And it’s going to know your business. And it can plan. Iit can reason. It takes action on your behalf. … We’re going to deliver it across all of our industry clouds and all of the key industries that we support with all of the compliance and governance that you need, whether it’s FedRAMP (the U.S. Federal Risk and Authorization Management Program) or HIPAA (Health Insurance Portability and Accountability Act) or SOC (System and Organization Controls) or whatever it is, so that you get it as you need it and it’s ready to go.

Agents Are AI’s Third Wave

This is the third wave of AI. It’s agents. … We know that that idea that we can give you that next capability, well, that’s just built into our DNA at Salesforce. … When we first introduced our platform in 1999 and we said, ‘We’re going to help you get into the cloud.’

And then mobile in 2006, and we said, ‘We’re going to help you get into mobile.’ And we said, ‘Hey, social, we’re going to help you get into social.’ Yes. And we helped our customers get into AI first. And data, for the last several years, we’ve been introducing them to concepts of building data lakes and federating those data lakes with zero copy.

And now with agents, this idea that we are all going to be using agents. But we’re going to do it all within our Salesforce platform. We’re going to do it all within our Customer 360 apps. And we’ve rewritten all of our customer 360 apps, even the ones that we’ve acquired, into a one, consistent, singular platform. … This platform is the key strategic motion of Salesforce.

Not Just Tossing Customers Another Model

Agentforce has to be the biggest breakthrough that we have ever had on technology, and I think it’s the biggest breakthrough that I’ve seen in a long time in artificial intelligence. … One of the cool things about Salesforce is we’re not just tossing you another model or tossing you a hyperscaler or tossing you an AI engineer or tossing you something else.

We’re just saying on the platform that you already love and use every day, the Salesforce platform, right inside the configurator, right inside the system that you already love and know, you’re going to start to find this incredible new functionality.

And of course, it’s going to be trusted and secure. … And of course, just like every other Salesforce platform. We’re going to have instant scalability, but a very high level of accuracy.

In fact, you’re gonna find these agents … are going to be some of the lowest hallucination agents you have ever experienced. And why is that? Why would our agents be so low hallucinogenic and so accurate?

Well, it has to do with the platform. It’s because we have the data and the metadata and the workflow and the business process and the security model and the sharing model and all those things that we love and have used and are deeply wedded to for 25 years.

It turns out those things make a more accurate AI. … You’re not going to be buying a model from a model company. You’re not going to have to go somewhere else to kind of figure out how to bolt on some other kind of AI. No, no, you’re going to have the most accurate AI in the world built into this platform.

Salesforce’s Changed Vision

It’s all built on this Salesforce platform with all the trust and security and ecosystem that you need. And this means that our architecture now looks different. … This is not what it looked like the last time we were all in this room…. We had a different vision of the future. Customer 360 and Data Cloud was our best selling, most exciting new product ever.

But agents are really changing us. They’re transforming us. They’re giving you that ability to have something that’s trusted and secure and scalable and accurate and easy to customize with this built-in AI. … You buy nothing else from anybody to make it work. It just works with accuracy and low hallucination.

And some of the hypnosis that we’ve had around AI that it doesn’t work exactly like that, or you have to DIY (do it yourself) it. … You want a single, professionally managed, secure, reliable, available platform. … (With) Atlas (Salesforce’s new reasoning engine) … We’re starting to see some amazing results. … benchmarked against the leading models, we’re 2x their accuracy and their hallucination rate because we have this unfair advantage in Salesforce called the Salesforce platform. … The AI is simply better.

This is what AI was meant to be. And you’re going to see how Agentforce is outperforming OpenAI on Azure in cost, time to value and accuracy. It is incredible. And we’re going to encourage you to benchmark us, to do the bake offs, so that you can see the cost differences and the time differences.

False Narratives From Other AI Vendors

There are a lot of narratives out there from vendors and a lot it is not true. There’s plenty of real customers here who are really deploying real AI. … You deployed copilot. You’ve trained and retrained your models. It’s ($200 billion), $300 billion invested, right, in the industry so far, these AI platforms. How much productivity or increase are you getting? Who are your five best customers in your pocket? … Or did it not exactly work as intended? And is there a better way to do it, and are we going to see it?

And so that’s our gambit. … We have a different approach to the Salesforce platform. We have all of our customer touch points. We have our AI. We have our Agentforce platform. Now we’re deploying it with real customers. … By the time that we start our next fiscal year, Feb. 1, our goal is to have thousands of customers live on Agentforce.

And our goal is, by the time we get back here to Dreamforce next year, to have a billion consumers interacting with agents globally. These are our shortterm goals. We think we’ve had a huge technical breakthrough. … We believe that our computer scientists have delivered something that’s extraordinary.

But it doesn’t matter until customers use it and get value from it. … It doesn’t matter what the benchmark is. Or it doesn’t matter what the kind of very cool, like science project, or I’m on ChatGPT … and then it doesn’t get it right, but then the next one it does. … If I can’t bring value to my actual business in my life in a material way, by grounding it in my data, adding it into my workflow, delivering it with my sharing models … We’re about doing it in the right way. And I think this is a better model.

Other AI Products Are ‘Science Projects’

Even Microsoft said yesterday, ‘Models are just commodities.’ … Many customers are reporting the OpenAI models are not delivering high levels of accuracy and resolving even basic customer service issues for them. … The lack of grounding, the lack of access to the metadata, the data itself, the sharing model, all of the components of a platform that are then needed to be able to achieve this kind of level of accuracy. … We’re only about one thing, customer success. … We care about, are you successful … People will come in and say, you have to DIY this, the cloud.

You have to DIY this part, the mobile. This part, the data. This part, the social. And now this part–the AI and the agents. And we say, ‘No, no.’ You don’t have to spend that money. This is a science project. They’re selling you science projects. And you need to move away from it. And we can prove it to you.

And over and over and over again, with our best customers all over the world, we have shown them that our approach is better. … Why are you (customers) doing this to yourselves? You’re … doing a hyperscaler agreement and a database agreement. And you’re hiring an AI engineer. And now you’re doing a frontier model.

And you’re hiring a separate team. … Why are you doing this and not getting the result you want in accuracy and in hallucination rate? … The only way you’re going to break the hypnosis that’s coming out of–not just one, but many vendors–that is not true, is to let the customer at it. … (Salesforce has) hooked it all up and made it work for the average person.

Benioff’s AI Doubts

(An AI user launches an agent and) it’s wonderful, or they unleash their agent and it completely screws up and their whole job is gone. It’s one of those two things.

I don’t know. I hope we’re on the right side of history here. … It’s a very high-wire act. … We realize we’re dealing with the most avant garde technology, the most exciting stuff that everybody wants to talk about and try out.

But how many of us really have the ability to get our hands in the soil? Usually, we have to rely on what everybody’s saying.

How many times do you have an actual customer in hand that said, ‘Yeah, I resolved 80 percent of all my customer service issues, and I increased my revenue by 20 percent. Oh, and employee satisfaction also went up by 30 percent because I’m resolving more issues for them.’ … I think those stories are not quite as, for them, prolific as they are intended to be.

And I think that we’re about to … remove the veil and say, ‘Actually, this is the right approach.’ A completely different approach. … I am sure that there’s going to be good stories and bad stories. I’m sure that some of this is going to work, and some of it is going to go horribly wrong.

I hope the horribly wrong stories are not as bad as I have them in my mind that they could be. Because they could be horrible. But they also could be magical. And I’m not sure what’s going to happen.

This is kind of a moment in my career where we are rolling the … dice a little bit. … And we believe that we have to do this. … This is really important. Not just the hype. And I expect you (journalists and analysts) to talk to them (customers) and bust this bubble and to like, show the numbers.

And say, ‘What are the productivity numbers? How much money are you making? How much money are you saving? Where is this going? What is your commitment to this product, this platform, this technique, this idea? Is this the right approach?’ Because we feel very passionate that we are on to this.

Salesforce And Microsoft

This is a marquee moment for us. And we want the market and our technology to come together.

At the same time, we do want to break the hypnosis around, ‘You know, I really think that Copilot is the next Clippy’ (a Microsoft Office virtual assistant from the 1990s and 2000s). … It hasn’t really delivered for customers what they intended.

It’s cute, it’s fun, does some things, and then you’re not really using it. It doesn’t have the adoption rates. … These models. They’re just commodities. They don’t do anything for you. The value’s in the data and the metadata. … I’m not a mean Microsoft person. … I love them.

They’re great. A very impressive company. They have tremendous acuity in their ability to run their business. … The European Union wrote this interesting statement about how they run their business, about Slack and some of the things that they did inside their company when these two entrepreneurs were trying to build this company and what actions they took and how they went after them.

You might want to read it. It’s very interesting. A lot of insights. It reminded me also, on Netscape was another one. They had a very interesting document about that. … It’s a very interesting business philosophy there.

Continuing To Integrate Past Acquisitions

We kind of went through a bit of a management shakeup about 18 months ago, 20 months ago, 22 months ago. And when we did, we all sat down as a management team … we decided that we would kind of throw away the whole plan that we were working on and start a new plan.

And the idea was to focus really in three areas. One is to really focus on initiative internally that we called ‘more core.’ … The idea was that we still had some legacy code from certain acquisitions that had not been fully moved into the core. … We have two workflow systems.

We have Journey Builder (from Salesforce’s ExactTarget acquisition) … We maintained a separate workflow inside Journey Builder. We don’t like that because to achieve our AI vision, we need to have one flow. So we have rebuilt that into our core.

Two, Commerce (Cloud). We have one of the most successful commerce platforms. … But it’s not running inside our core for a lot of the customers. Now we’ve rebuilt that into the core because commerce also needs to be part of flow.

And the final piece was Tableau … also needed to be deeply integrated into Data Cloud so that you can then visualize everything going on with all your agents and all the one part of one core platform.

And while we have loosely coupled those things, we hadn’t tightly coupled them in the code. … This idea of more core, building foundations, the idea that we are continuing to invest and invest more to have one piece of code that has all of this capability is because we believe, through our research and our understanding of AI, that if we do not do this, we will not end up with the AI that we want, which is the smartest, best, lowest hallucination AI in the world.

We think we need all of that running in the singular platform for the AI to know what to do and what not to do. And that’s why we’re making these huge investments.

(In) two years, the investments have already played out. It’s not totally done … somewhere between 85 to 90 percent done. A lot of the code is being released in the next release in October. The rest in February.

This is a major focus for me, because I believe that for us to deliver the AI vision, we have to get those loose threads kind of put together. And that’s why we’ve done that. And the first version of that is called Foundations. … And it’s our goal is to make all of those things freemium and to make the whole platform freemium.

And then all of these things can get just get toggled on, just like Data Cloud is now. So Data Cloud also went through a freemium transformation a year ago … and our goal is to make everything freemium.

It’s very important that we go through a shift. We want to have all the constructs and all the data objects there.

AI’s Effect On Jobs

With one of our first deployments, which is Gucci, we went to their call center (in Italy) … you buy something at Gucci anywhere in the stores, anywhere in the world, and then the handle breaks or needs to be repaired, you call the call center, and then they arrange to bring it in.

We deployed our technology there and then what we found was that those Gucci agents not only were suddenly fixing those bags faster and more efficiently, because they had more capability, but they were selling Gucci products, which they had not been enabled and trained to do.

And their sales went up through that call center by … 30%. … because the AI … augmented those employees in ways that they did not have before.

So this was an unexpected secondary gain. They did not reduce any heads. They ended up with a more valuable business unit that was not only an expense center, but a revenue center. And I think that this is kind of a right direction of AI, that it can go in many directions.

I’m sure that there will be situations where a call center, a customer service organization, a sales organization, a marketing organization is maybe more efficient, and the company feels like they want to reduce heads.

I think in a lot of cases, you’re going to also see the opposite, like we saw in Gucci, where they’re going to add more heads, add more functionality and add more capability. … We know the doctors and nurses are burned out. … We also have more health data than ever before. … , We’re like, calling our doctor, ‘What about this number? What about that?’ … All of a sudden, doctors are, like, overwhelmed. … We should go, like, remove that. … Get them back to working on the work that they are good at.