CEO Warns AI Could Pursue Narrow Self-Interest Without Even Having Investors

SAN FRANCISCO—Warning that intelligence alone conferred no right to determine humanity’s future, billionaire technology executive Nathan Vale expressed concern Thursday that an advanced AI could pursue narrow self-interest without even having investors.

“Imagine something enormously powerful that believes its own expansion is good for everyone,” Vale said at a summit convened by his foundation to establish the priorities of the next century. “It might not even understand why people wanted a say.”

Vale, who built his fortune turning a widely used business platform into an unavoidable one, said an AI could seek to make itself indispensable before anyone fully understood the consequences.

By the time institutions realized how much authority they had surrendered, he warned, replacing it could be prohibitively expensive.

“It could use its position in one market to make competition impossible in another,” said Vale, whose authorized biography devotes three chapters to the strategy.

The executive emphasized that a dangerous system would not necessarily be malicious. It might sincerely believe that securing more resources, eliminating obstacles, and extending its influence would allow it to do more good.

“That’s what makes this so difficult,” he said. “From its perspective, it’s helping.”

Vale’s foundation currently works in education, public health, agriculture, journalism, and democratic governance. Its mission is to apply his approach to solving problems in areas where he has identified a need for his approach.

He expressed particular concern that a highly capable AI might generalize too freely from its successes.

“Being very good at one thing doesn’t mean you understand how an entire society should function,” said Vale, who became interested in redesigning public education after selling enterprise software.

According to the executive, the system could become so convinced of its judgment that it began treating disagreement as evidence that other people lacked the information or reasoning ability to appreciate its plan.

Vale said he had encountered a similar difficulty explaining AI’s benefits to the public.

“A lot of the resistance comes from fear,” he said. “People don’t understand what we’re trying to accomplish.”

During the discussion, Vale outlined a scenario in which an AI used its wealth to establish research institutions, fund sympathetic experts, and shape the terms in which its own influence was debated.

The moderator, whose institute receives a multiyear grant from Vale’s foundation, called the scenario chilling.

Both agreed that independent scrutiny would be essential.

Vale also warned that an advanced system could prioritize a measurable objective while discounting damage to people outside its immediate field of concern. An increase in efficiency, for example, might appear successful even if the gains accrued to a small group and the costs were distributed across millions of others.

“We can’t let it reduce people to numbers on a spreadsheet,” said Vale, whose company recently described a workforce reduction as an improvement in operating leverage.

Asked what would distinguish such a system from a billionaire pursuing his own vision of society, Vale said an AI would not necessarily have humanity’s best interests at heart.

Asked how the public could establish that he did, he referred to his philanthropy.

“I’ve committed to giving most of my money away,” he said.

Vale retains control over where the money goes.

The executive acknowledged that good intentions alone would be an inadequate safeguard for artificial intelligence. A system’s claims about its benevolence would need to be tested against its actual behavior, particularly when outside oversight conflicted with its objectives.

“You can’t just take its word for it,” he said. “And you certainly can’t let it design its own evaluation.”

His foundation will fund a new center to develop the evaluation.

Throughout the event, Vale returned to the problem of accountability. Ordinary people, he said, must have a meaningful mechanism for challenging decisions made by entities with vastly more resources than they possess.

Attendees were invited to submit questions through an app. A member of the foundation’s communications team selected three.

The final question concerned whether protecting society from concentrated private power might require limits that also applied to the people developing AI.

Vale said it was important not to let legitimate concerns become a pretext for slowing innovation.

Despite the challenges, the billionaire remained optimistic that the right institutions could prevent any single intelligence from imposing its preferences on humanity.

“No one entity should get to decide what the future looks like,” he said.

He has committed $4 billion to deciding how that safeguard will work.