Middle managers are accustomed to translating the dreams of senior leadership into reality. When it comes to AI projects however, the bridge connecting the grand plans of the C-suite to the realm of the doable can be tricky to find for those tasked with crossing it.

“The disconnect is between vision and practicality,” says Deloitte Consulting’s Principal and Global Generative AI Leader Nitin Mittal. “This is somewhat ubiquitous in our own client base and the many, many enterprises that we talk to.”

The challenge is to stay true to the vision while being responsible and understanding the practical constraints of the organization, like data, technology, and the skills of the workforce, Mittal added.

Speaking at a special session at Fortune‘s Brainstorm AI conference Tuesday, Mittal and executive at several other companies discussed the “productivity paradox” that can erode the promised benefits of AI if projects aren’t planned and executed thoughtfully. AI’s potential to make organizations 40% more efficient by 2035, as forecast by the World Economic Forum, has generated a lot of excitement and expectations that can all too easily turn into disappointment.

The gap between heady expectations and on-the-ground challenges is reflected in a survey that Upwork CEO Hayden Brown cited during the talk: A research institute surveyed 1,400 businesses in the U.S. about their AI strategies, and the findings showed that 73% of C suite executives are confident in their AI strategy, while only 53% of managers and VPs feel the same way.

“The managers and VPs implementing the technologies not only are confused about how, they’re actually very anchored on risk, and fear,” Brown said. “They were disproportionately speaking to concerns around that technology, and how it could be implemented, and whether their jobs or their teams jobs would be replaced.”

The fears aren’t unreasonable. Disquiet about the impact of AI technology on the workforce and on society were themes that were repeatedly brought up by attendees and speakers throughout the two-day conference. In an on-stage interview on Monday, Atif Rafiq, the founder and CEO of Ritual.work, said that businesses needed to do a better job of training workers for the AI age. “We’re woefully unprepared,” Rafiq said.

During Tuesday’s special panel with Deloitte’s Mittal and Upwork’s Brown, Emily Chiu, the chief operating officer of Block TBD, talked about how these issues were playing out at her firm. Chiu said that the level of concern over AI is relatively balanced across the organization, extending from the C-suite to the staff. The key consideration within Block TBD——a division of Jack Dorsey’s financial technology firm Block—is in the need for careful planning and balance when adopting new technologies, especially in handling sensitive personal information.

“What we see now is that these tools are going to be in the hands of everyone,” Chiu said. “And in financial services, the minute you launch a product, every scammer shows up with every angle to try to defraud you and to find loopholes.”

While these risk cannot be underestimated, Salesforce’s chief ethical and humane use officer Paula Goldman described a positive and excited attitude towards AI at the firm, with a strong desire to experiment and integrate the technology across all departments. For her, these risks present an opportunity for innovation, creating a sense of trust that aligns with the goals of enterprises.

“People want experiment, like how fast can we experiment?” Brown said. “But externally, you know, I think I’m actually not seeing a disconnect, I am actually seeing excitement and a sort of awareness of risk. And that, to me, is really optimistic, because AI is not new, the risks that we’re talking about are not new. And people have been working on solutions for this stuff for quite some time.”