Voice agents

How to put an AI voice agent on real phone calls (and why your phone won't let you)

Android apps can't hear or speak into a phone call. Pocket ships two ways around it today, voice links and Twilio calls, and we're testing two more. Here's how each works.

Voice agents are everywhere this year: they book appointments, qualify leads, chase payments and answer support lines. Almost all of them live on a phone number rented from a telephony provider. What people actually ask us for is different: *"let my agent talk on my calls, from my number, on my phone."* It sounds like one setting. It turns out to be one of the most locked-down corners of a modern phone.

This post explains what we found when we tried, why a normal app can't do it, and the four approaches that work. Two ship in Pocket today: voice links and Twilio calls. The other two, a Bluetooth bridge and root mode, are experiments, and each section says so.

What a voice agent actually is

Strip away the branding and every voice agent is the same loop, repeated every time someone stops talking:

1. Listen. Audio arrives in small frames (20 ms each in our engine). A voice-activity detector decides when the caller has finished a sentence.
2. Transcribe. Speech-to-text turns that sentence into words. We support Deepgram, ElevenLabs Scribe, Cartesia Ink and Sarvam, which is the one to use for Hindi, Hinglish and other Indian languages.
3. Think. A language model reads the conversation so far, the agent's brief, and its tools (send a WhatsApp, book a follow-up, end the call), then streams a reply.
4. Speak. Each finished sentence goes straight to text-to-speech (Cartesia Sonic, ElevenLabs, Deepgram Aura or Sarvam Bulbul) and the audio starts playing before the model has finished the paragraph.
5. Get interrupted. If the caller talks over the agent, playback stops immediately and the loop starts again. This "barge-in" is what makes it feel like a conversation instead of a voicemail tree.

pause before the agent decides you're done

0.7 s

speech-to-text (ElevenLabs Scribe)

0.7 s

model's first sentence

1.0 s

first audio (Cartesia Sonic)

0.4 s

Those are real numbers from our engine today: about 2.6 seconds from the moment you stop talking to the moment you hear a reply. Streaming speech-to-text and a faster model bring it under two. None of the loop is hard any more. The hard part is getting the audio of a phone call into it.

Why your phone won't let an app into a call

We started where everyone starts: Pocket's Android app already places calls, holds and merges them, and can be your default phone app. Surely it can hear the call too? We tested every audio source Android offers during a live call on an Android 16 phone:

• MIC, VOICE_COMMUNICATION, CAMCORDER and UNPROCESSED: the system silences them the moment a call is active.
• VOICE_RECOGNITION: not flagged as silenced, but it records near-silence.
• VOICE_CALL, VOICE_UPLINK, VOICE_DOWNLINK: the sources that *would* carry the call require CAPTURE_AUDIO_OUTPUT, a permission only system apps signed by the phone maker can hold.
• Being the default dialer gets you a lot (call control, hold, merge, choosing the SIM) but not the audio. Android's newer "call audio processing" mode lets a dialer put the call into a state where it could be processed; the audio streams still refuse to open without the privileged permission.

So the voice agent can't live *inside* the call on a stock phone. It has to sit somewhere the audio can reach. There are four such places.

1. A voice link: skip the phone network entirely

The simplest answer is to not use a phone call at all. A voice link is a web page: the other person taps it, allows the microphone, and talks to your agent in the browser. It works on any phone, with no app, no number and no per-minute telephony bill, and audio quality is better than a call because it isn't squeezed through a narrowband phone line.

In Pocket, any agent or bot can create one with send_voice_link and send it over WhatsApp or email: "Hi Riya, tap here to talk to our counsellor about the weekend batch." The conversation runs with the bot's brief and tools, the transcript lands in your project, and the bot gets a follow-up run to update the CRM or dashboard. Read the step-by-step guide.

Use it for: leads, interviews, surveys, support, and talking to your own agents hands-free. Not for: people who will only pick up a normal call.

2. Telephony: give the agent its own number

Telephony providers can stream a live call's audio over a WebSocket to a server you control. That server is the same voice engine, running on your Pocket machine. Twilio is connected today: add your Account SID, auth token and number under Account → Voice, and any bot can ring a number with place_call, talk on the call, and file the outcome and talk time on the CRM lead. It's how most commercial voice agents work, and it opens up everything a phone network can do:

• Outbound (live): the agent calls a list and follows up on no-answers, the way the callback bot retries after five minutes. The list is your own enquiries and customers, inside calling hours, never a cold list (why).
• Inbound (next): people call the agent's number directly.
• Forwarding (next): forward your own number to the agent when you're busy or unreachable (on most Indian networks, **67*number# forwards when busy and **61*number# when unanswered), so callers still dial *you*.
• Conference (next): your phone calls the agent's number and merges it into a live call, so the agent joins as a third participant who can take notes, answer questions or take over.

Exotel and Plivo, popular in India, stream calls the same way and are next on the list. Cost: a virtual number is roughly ₹500–1,500 a month in India, and calls cost a few paise to a rupee a minute depending on the provider and route. Use it for: anything that must be a real call at scale.

3. A bridge box (in testing): a headset that happens to be a computer

The phone *will* hand call audio to one kind of device: a Bluetooth headset. The phone doesn't care whether the headset is a pair of earbuds or a Linux machine. A laptop, a Raspberry Pi or a small board that pairs with the phone using the hands-free profile receives the caller's voice and sends back whatever it's told to. Put the voice engine behind that, and your agent speaks on normal calls from your own SIM, with no provider involved.

We have a Linux laptop paired to a phone as a hands-free unit today. The phone happily routes call audio to it, but the Linux audio stack (PipeWire with WirePlumber 0.4) doesn't yet attach the call's audio link in that role, so this mode is still in testing. An ESP32 board with classic Bluetooth, which can act as the hands-free unit and relay audio over Wi-Fi, is the low-cost version we're working towards: a ₹500 box that sits next to your phone.

4. Root mode (experimental): take the permission back

This isn't in the Pocket app yet. On a rooted Android phone, a Magisk module can grant Pocket the privileged audio permission that system apps have. Then the app can capture and inject call audio directly: no box, no provider, your number. It's the most seamless option and the most invasive one.

• What still works: calls, SMS and WhatsApp work normally on rooted phones. Magisk hides root from most apps.
• What can break: banking, UPI and payment apps, and some games, use Play Integrity checks and may refuse to run. Unlocking the bootloader wipes the phone and can void the warranty, and some manufacturers don't allow it at all.
• Who it's for: people with a spare or dedicated phone (a lead-calling handset for a sales team, say), not your main phone.

Which one should you use?

• Just want it to work today? Voice link.
• Need real calls, many of them? Twilio, from Account → Voice.
• Must be your own SIM? Call from your phone with Call next today; the bridge box is coming.

Whichever you choose, it's the same engine, the same bots, the same tools and the same transcripts. The only thing that changes is how the audio gets in. Your agents in Pocket can walk you through setting up voice links and Twilio.