When workers aren’t at their desks, they may as well be not working at all. That’s what many anti-remote CEOs have said, in one way or another, since the earliest pandemic lockdowns. But media titan, billionaire, and former New York City mayor Michael Bloomberg took it a step further last weekend during an interview with Mo Rocca for CBS Sunday Morning segment. They’re not just not-working; remote workers are essentially on vacation.

“If you think [work] can be done at home, I don’t know,” Bloomberg said. “But every golf course that I’ve heard about in the last three years has had record summers, okay? It is funny, but it’s tragic.”

It’s certainly a reach to suggest that any stressed out remote worker can simply head to the golf course to blow off steam without their manager taking note or disciplining them. But in terms of data, Bloomberg’s not too far off.

Earlier this year, a Stanford University research paper by economists Alex Finan and Nick Bloom found that remote work, indeed, has created a golfing boom, as post-pandemic golf course visits have skyrocketed during weekday mid-afternoons.

The most reasonable explanation is that, with the newfound freedom from oversight that remote work allows, some workers are using golf as a way to break up the day and recharge during slow periods. It’s all a bit ironic, however, given the well-documented affinity for golf—even at 2 p.m. on a Tuesday—among those in the corner offices.

But, contrary to what Bloomberg seemed to suggest, Bloom and Finan said the triple-digit growth in golf during work hours hasn’t directly led to a drop in productivity. So long as employees make up their time on the putting green later in the day, “then [golfing] does not reduce productivity. Indeed, national productivity during/post pandemic has been strong.”

The anti-remote cohort is standing strong

That’s not enough for Bloomberg, a steadfast critic of flexible work. Golf or no golf, Bloomberg counts himself among other big-name CEOs (including JPMorgan’s Jamie Dimon, Twitter’s Elon Musk, and Alphabet’s Sundar Pichai) in his resolute stance that in-person work is critical.

In early August, Bloomberg penned a Washington Post op-ed expressing his dissatisfaction with government workers logging on remotely, even claiming that taxpayers foot the bill for empty offices. (He also claimed that 80% of his own employees at his eponymous company show up at least three days a week, and he’s since pushed that mandate to four.)

Bloomberg, who is worth over $96 billion, said remote work has made Washington, D.C. a “shadow of its former self” and that the tax money wasted on empty office space has decimated public resources.

“This has gone on too long. The pandemic is over. Excuses for allowing offices to sit empty should end, too,” he wrote. “Our managers have seen the benefits of returning to in-person work, and we have heard about those benefits from their teams, too, especially from young people just starting their careers.”

By eliminating opportunities for mentoring and upskilling, remote work hurts an organization’s future outlook—to say nothing of its impact on the young workers themselves, Bloomberg said. Remote workers are also proven likely to suffer from the impacts of proximity bias, in which their bosses subconsciously prefer and prioritize the workers they see most often. On the other hand, remote workers can benefit from saved commute time and more valuable family time.

Bosses are losing patience with the remote work war

Bloomberg’s opinion is shared by Jefferies CEO Rich Handler, who has said permanently remote roles are just for short-term goals and a paycheck, not for a career. Hybrid work may well stick around, Handler told Fortune, but “the reality is, if you are in the office, you get pulled into a lot of interesting ‘real time’ situations because physical presence matters.”

On that point, even the most pro-remote experts agree. “Bottom line, personal interactions among colleagues diminish by a significant amount when someone works from home,” the researchers behind the WFH Research consortium, Bloom and Jose Maria Barrero, wrote in April. “That is a cost workers and firms pay in terms of slower on-the-job learning, in exchange for the flexibility and personal autonomy gained when working from home.”

That’s no small cost, but some workers clearly value a round of golf a tiny bit more. Too bad for them, it appears that thanks to the Labor Day return-to-work mandates, weekday golf rounds— much like the vast availability of fully remote roles—might have already become a thing of the past.

“Human beings probably don’t change very quickly in what they do; I can’t work with you if it’s over Zoom,” Bloomberg went on in the CBS interview, explaining why remote work would never take the long-term upper hand over in-person work. “You can’t do the same thing via Zoom that you can do face-to-face. Period.”