Create an AI Assistant With Its Own Phone Number (Eleven Labs Walkthrough)

The Rundown In this guide, you will build an AI personal assistant that you can call from your phone. It knows your calendar, your business, and whatever else you feed it. The whole setup takes about 10 minutes and costs $0 to test. Who This Is Useful For Busy founders or operators who want a hands-free way to check their schedule Consultants juggling multiple clients who need quick access to project info on the go Anyone who wants to experiment with voice AI agents without writing code What You Will Build A personal AI phone assistant powered by Eleven Labs that answers your calls, knows your calendar and business context, and can be extended with tools like webhooks and automations. What You Need to Get Started Eleven Labs  account (free plan works) Twilio  account (free to sign up — $15 in credits included) A calendar export or text file with your schedule Optionally, a URL to your business or product page Step 1: Create Your Agent in Eleven Labs Sign in to  Eleven Labs  and go to the  Agents  section. Click  New Agent Select  Personal Assistant  as the template Name it whatever you want Pick a voice.  Browse the voice library — you can filter by language, gender, and age. Preview a few and pick one that fits. Pro tip:  If the default voice speed feels slow, bump it up slightly in the voice settings. Review the system prompt.  The personal assistant template comes with a solid default. Just update your time zone and customize the first message to your liking. Add to the knowledge base.  This is what makes your assistant actually useful: Paste a URL to your website or product page — the agent will scrape it Upload a text or markdown file of your calendar (ask Claude or ChatGPT to format your Google Calendar as markdown) Add any other static info: client lists, project notes, your services Hit  Publish  when you're done. Pro tip:  The knowledge base is static, so it's best for things that don't change often — your offerings, your brand info, current projects. For dynamic data like live calendar events, you'll want the webhook approach in the Going Further section. Step 2: Set Up Twilio and Get a Phone Number Go to  twilio.com  and create a new account. They give you  $15 in free credits . A phone number costs about  $1 to buy and  $1/month  to keep. Click  Phone Numbers  →  Buy a Number Pick one with voice capability (SMS requires extra paperwork and won't be same-day) Copy down three things from your Twilio dashboard: Phone number  (the one you just bought) Account SID  (on the main dashboard) Auth Token  (click to reveal, right below the SID) Pro tip:  During the free trial, callers will hear a short Twilio message before connecting. It goes away once you upgrade to pay-as-you-go, which is fractions of a cent per minute. Step 3: Connect Twilio to Eleven Labs Go back to Eleven Labs. In the sidebar, find  Phone Numbers  and click  Import Number from Twilio . Paste your  Account SID Paste your  Auth Token Paste your  Twilio phone number Name the number and assign your agent to it That's it. Your agent now has a real phone number. Call it and start talking. On a free Twilio account it will make you hit the keypad to connect to your agent. If you upgrade this will go away. Going Further Once the basic assistant is working, you can make it much more powerful by adding tools. Here are the system tools you can install: There are powerful integrations you can add too. Go into your agent →  Add Tool  →  Integration . Your options include: Cal.com  for live calendar access HubSpot  for CRM lookups Custom MCP server  for advanced integrations Webhooks  connect to  Make.com , n8n, or Zapier With webhooks, your assistant can pull live calendar events, schedule meetings, check your email, or look up client info — all triggered by a phone call. If you've taken any of the Rundown automation workshops, this is a natural next step.

• Duration

Save 15% on Thoughtly by joining The Runway University

Through our vast network of over 100 partnerships with major AI companies, we also offer University members free access or major discounts to the many popular AI tools along with AI certified courses, daily guides on AI tools and many more.

Join AI University

Learn more