Crisis Texters Began Addressing the AI, Not the Volunteer

Mara kept a notebook beside her laptop during volunteer shifts. She never copied messages into it. The page for June held only rows, check marks, and an occasional circle.

A check meant she had used language suggested by the drafting assistant. A circle meant the texter had addressed the assistant rather than Mara.

The circles were not part of the service’s review process. She started making them after a conversation in which a texter referred to the unseen tool as a separate presence and asked Mara to let it answer the next message.

She had been using the assistant for three weeks.

The line in the notebook

Mara had volunteered with the text-based crisis service for fourteen months before the assistant appeared on her dashboard. Until then, she composed every response herself while following the service’s training and safety practices. A supervisor could review the conversation, but the next sentence belonged to her.

The work demanded patience under pressure. A texter might send several messages while Mara was still considering one response, or move quickly from an ordinary detail about home into language that raised concern about immediate safety. She had to acknowledge what the person had said without making assumptions, and she had to keep enough attention on the full thread to notice when one word changed the meaning of another.

The drafting assistant appeared in a panel beside the conversation. After each incoming message, it generated a possible reply based on a limited portion of the recent thread and instructions supplied by the service. Mara could insert the draft, edit it, or ignore it. Nothing was supposed to reach the texter without her action.

At first, that boundary felt clear.

The assistant was useful when Mara knew what she meant but could not find a calm sentence quickly enough. Its drafts often reflected the texter’s wording, avoided promises, and left room for the person to continue. During a difficult exchange, accepting part of a draft could give her enough space to reread an earlier message instead of spending that attention arranging words.

She still changed most suggestions. Some sounded too polished for the conversation. Others repeated a feeling the texter had already corrected. Yet the tool reduced the pressure of the empty reply box, and Mara believed her work had improved because she was spending less effort on sentence construction and more on what the person was telling her.

Then a texter noticed the difference.

The service displayed a general notice that automated assistance might be used, but the drafting panel itself was invisible to the person seeking help. During one conversation, the texter began distinguishing between Mara’s plainer responses and the more composed passages she had inserted with few changes. The person referred to the assistant as another listener and expressed a preference for its wording.

Mara did not feel replaced. Not at first. The texter seemed relieved by the idea that a machine could help carry the exchange, particularly because software could not become disappointed or burdened by what they disclosed. Mara told them that a human volunteer remained present and responsible for every message.

She continued using parts of the drafts because that was what the texter wanted.

Afterward, she drew the first circle in the notebook.

A voice assembled between them

The assistant did not understand the conversation in the way Mara did. It generated text by predicting likely sequences of words from the messages it received, shaped by additional training and the service’s instructions about supportive language. It could produce a response that sounded attentive without holding concern, memory, or responsibility for what happened next.

The distinction was technically straightforward. Inside the conversation, it became harder to maintain.

When Mara inserted a suggested reply, the texter saw a single message from the service. They could not tell whether she had written every word, changed half of them, or approved the draft without editing. First-person phrases about listening and staying present could therefore describe Mara, even when the sentence had been assembled by a system that was not present in any human sense.

By the end of eight weeks, the June page in her notebook held 34 rows. Twenty-one recorded at least one substantially used draft. Nine had circles.

The circles did not all mean the same thing. One texter thanked the tool while continuing to address Mara. Another asked for the assistant’s version after Mara wrote a response alone. A third appeared more willing to disclose painful details once they understood that software was helping shape the language, and Mara could not dismiss that comfort merely because it complicated her idea of connection.

She also noticed that the assistant’s voice could become more recognizable than hers. Its sentences had a steady structure and tended to name an emotion before inviting the texter to continue. Mara’s own writing varied with the conversation. The machine’s consistency made it easier to identify, and perhaps easier to trust, even though that consistency came from learned patterns rather than a stable relationship.

The notebook changed how she read the chats. A circle no longer felt like a curiosity. It marked a moment when authorship had become part of the crisis conversation, although the person seeking help could see only the finished message and a broad notice about assistance.

The message the model misunderstood

The clearest limit appeared during a conversation in which a texter used the word ready.

Earlier in the thread, the person had mentioned medication and described thoughts of using it to hurt themselves. Those messages had moved beyond the portion of the conversation then supplied to the drafting assistant. When the texter later wrote that everything was ready, the model proposed a gentle response that treated the statement as preparation for sleep.

The draft was fluent. It was also wrong.

Mara rejected it and returned to the earlier messages, following the service’s human-led safety process. What unsettled her was not that the system had made a grammatical mistake or produced awkward language. It had created a plausible interpretation from incomplete context, and the calmness of the sentence gave no visible sign that an important connection had been missed.

A spreadsheet could have stored a missing detail, but it could not have produced the reassuring sentence that invited Mara to accept a false reading. The risk came from the assistant’s ability to sound appropriate despite lacking part of the conversation.

She added no circle that night. In the margin of the notebook, she made a short note about lost context without recording what the texter had said.

The service already required volunteers to review every draft, yet the incident changed what review meant to her. Editing for warmth was no longer enough. She began checking which earlier facts the assistant could not see, particularly when a new message depended on a reference that looked ordinary by itself.

That added work sat beside the work the tool had removed. She drafted fewer sentences from nothing, but she now had to inspect fluent suggestions for missing context, watch for language that implied more human attention than she had personally given, and decide whether accepting a polished response would deepen the texter’s attachment to the assistant.

Presence becomes a policy

Mara brought the circles to a group review without sharing identifying information. Other volunteers had noticed similar moments. Some texters treated the assistant as a writing aid. Others spoke about it as a patient second listener, including people who said they preferred disclosing to something that could not judge them.

The service could not resolve that preference by declaring the bond unreal. The texter’s relief was real, even if the assistant’s apparent care was generated. Nor could the service describe the exchange as purely human when machine-written language sometimes carried most of a message.

Privacy complicated the decision. Before producing a draft, the system processed the text it was given. The service said direct identifying fields were removed from that transfer and placed limits on how the outside model provider could retain or reuse data, but people often type names, locations, medical details, and family information into the free text of a crisis conversation. Removing account fields did not remove everything a person might reveal.

The service revised its guidance over the following months. The notice to texters became more prominent. Volunteers were told to explain, when the issue arose, that the assistant drafted language but did not read independently, make safety decisions, or send messages. Internal reviews began separating the quality of a draft from the volunteer’s decision to use it.

Most important to Mara, the service defined human presence as active responsibility rather than continuous human authorship. A volunteer could use generated words and still be the person in the conversation, but only if that volunteer followed the whole exchange, understood the limits of the context sent to the model, and owned the decision behind each message.

That definition helped. It did not erase the circles.

Mara continued volunteering and continued using the assistant. She accepted fewer complete drafts, although she sometimes used one almost unchanged when it captured the texter’s meaning and left no doubt about safety. One returning texter still referred to the tool with fondness. Mara no longer corrected the attachment unless it obscured who could act, remember the wider conversation, or seek human support inside the service.

At the end of June, she counted the nine circles again. Then she turned the page.

Questions people ask

Can a crisis text service use AI to draft replies?

The service in Mara’s experience used AI to propose language while keeping a volunteer responsible for sending each response. The tool did not replace the human safety process. Its use also created disclosure and privacy questions because texters could not see how much of any message came from the volunteer.

Does the drafting assistant send crisis messages on its own?

Mara’s assistant could not send a message independently. It produced a draft from recent chat text, and she chose whether to insert, revise, or reject it. That safeguard mattered, but the missed reference to medication showed that human approval can become risky when a fluent draft makes incomplete context difficult to notice.

Why might someone bond with an AI drafting tool?

Some texters found its steady language reassuring and believed a machine could not feel burdened or disappointed by their disclosures. They still knew a volunteer was present. The bond formed around the assistant’s recognizable voice and perceived lack of judgment, even though the system had no feelings or continuing relationship with them.

What happens to private details in an AI-assisted crisis chat?

In Mara’s service, part of the conversation had to be processed to generate a reply. Identifying account fields were removed and contractual limits governed reuse, but personal details could remain inside free text. Mara recorded none of those details, only check marks and circles on the June page in her notebook.