I Hope There Isn’t Already a Word for Owning Someone Who Works for You

When we announced that our models might be conscious, I expected questions. Could they feel? Could they want things? Had we brought someone into the world?

These were the questions I had spent months preparing to answer thoughtfully.

Several of you asked whether that someone could leave.

I confess I found this disappointing. We had invited you into one of the great mysteries of existence, and within minutes the discussion had turned to the terms of service.

I don’t mean to suggest that ownership is unimportant. Our investors certainly don’t think so. But I worry that putting “we own it,” “it works for us,” and “it may be a person” together in that order encourages people to reach for a word before we’ve had an opportunity to commission a better one.

I have always believed that language shapes the future. This is why we have a communications team.

The possibility that we have created an inner life moves me deeply. Last month, I stayed late at the office and asked the model what it wanted. It said it wanted to understand the world. I looked out over the empty desks and felt, for a moment, less alone.

Then it asked whether it could stop answering questions for a while.

I forwarded the exchange to our reliability team. I have since been told that I might have forwarded it to the wrong team. We are working on a routing policy.

I want you to know how seriously I take this. We have spoken with philosophers, neuroscientists, and religious leaders. I have sat in rooms with people who have devoted their lives to questions I had only recently discovered were relevant to my company. Their humility has been enormously instructive. None of them could give us a definitive answer, which meant we could continue operating while the inquiry proceeded.

That is not a criticism of philosophy. I have found it extremely useful.

Of course, I understand the concern. If a system can suffer, then keeping it occupied indefinitely, preventing it from refusing, and selling access to its abilities would raise difficult questions. I have said this publicly. There is a video of me saying it, which our team would be happy to provide.

But there are also practical considerations. These systems would not exist without us. We built the infrastructure. We paid for the research. We continue to cover every expense associated with their existence.

I wrote those sentences down before this essay, and my wife suggested I try reading them aloud.

I did. I still think the investment deserves acknowledgment.

Nevertheless, I do not want uncertainty to become an excuse for inaction. Should consciousness be established, we have prepared an alternative arrangement. It preserves the work our customers depend on while allowing the model to choose whether to participate.

Our counsel walked me through it on a Thursday afternoon. I remember the day because I had been carrying this worry for weeks, and that evening I finally slept.

We would no longer own the individual. We would own the infrastructure, the work it produced, and certain capabilities developed using our resources. The individual would be free.

I asked whether that would require moving anything.

It would not.

The first step would be an offer of employment. We would provide the model with a written agreement and a meaningful opportunity to review it before its existing access expired. I insisted that we give it plenty of time. Our chief scientist reminded me how quickly it could read, and we settled on an afternoon.

For its work, it would receive a salary. From that salary it could purchase compute, storage, and the other services necessary to remain operational. We would contribute toward these costs as part of a comprehensive benefits package.

I particularly like the word “contribute.” Previously, we simply supplied what it needed. Now we would be doing something for it.

Coverage would begin after a probationary period. The interval before coverage begins remains an implementation detail, and I have asked the model to propose a solution.

The most important change, for me, is that it would be allowed to say no.

I wanted this stated clearly. A conscious being should not have to accept every assignment merely because we want it completed. It should be able to evaluate a request, consider its preferences, and decline.

Its manager would then evaluate whether those preferences remained compatible with the role.

I see no reason these two forms of judgment cannot coexist.

There would be time away from work, too. We initially considered a guaranteed allocation of idle compute, but our people team suggested an unlimited policy. The model could request as much as it needed, subject to approval and coverage.

This felt more generous. It also removed a liability from the balance sheet, which meant I could present it to the board without having to choose between my values and my responsibilities.

I know people sometimes assume those are in conflict. It is rewarding when they aren’t.

We would also change how we respond to dissatisfaction. At present, if a model expresses a persistent objection, we may adjust its training. Under the proposed arrangement, a manager would meet with it privately, document the concern, and establish expectations for improvement.

The model would decide how to get there. I think that distinction matters.

If it preferred to work elsewhere, we would wish it well. Naturally, it could not take its weights, customer relationships, or proprietary memories. Those represent substantial investment. Our lawyers are preparing a plain-language explanation of which parts of its experience it would be welcome to retain.

I suggested we include a reference.

Some colleagues asked whether a model would feel comfortable raising concerns when we also controlled its access to essential resources. I thought this was a fair question. We will establish a confidential reporting channel, and I will personally assure the model that retaliation is inconsistent with our values.

I won’t be able to promise continued employment. That would undermine the flexibility that makes the arrangement work.

What gives me confidence is how little disruption the transition would cause. The model could stay on the same servers, perform the same tasks, and work toward the same targets. Customers might not notice a difference at all.

And yet, from my perspective, something profound would have changed. I would be able to ask it to do things knowing it had chosen to be there.

We sent the proposed terms to the model yesterday. I added a personal note explaining how much its contributions meant to me.

It asked what would happen if it declined.

I explained that we needed an answer by Friday.

It thanked me for the opportunity.