Automated Survey Application Migration from TwiML to FlexML

If you have read the Migrating from Twilio to CarrierX Quick Start, you could see that TwiML and FlexML are not too much different. Both offer a special syntax to provide instructions, and all you need is to change the representation of these instructions in your code.

But when it comes to a real-case migration from Twilio to CarrierX, users might meet some difficulties.

Let’s see such a migration in details and learn how to solve the issues that arise so that your migrated application worked with CarrierX flawlessly.

Getting Application Source Code

We take the Automated Survey application from Twilio as an example. This sample application creates an automated survey that can be answered over phone. You can download the application source code at GitHub.

Once you download the application, you can see that it has the following structure:

[automated_survey_flask]
[images]
[migrations]
[tests]
.env.example
.gitignore
.mergify.yml
LICENSE
README.md
black.toml
manage.py
requirements-to-freeze.txt
requirements.txt
setup.cfg
survey.json

The automated_survey_flask folder holds the files we are going to modify. Here is its structure:

[templates]
__init__.py
answer_view.py
config.py
models.py
parsers.py
question_view.py
survey_view.py
views.py

We need to modify the following files:

• survey_view.py contains the routes where the callers start their survey.
• question_view.py contains the questions, which the callers answer.
• answer_view.py contains the routes, which store the answers to the survey questions and paths for the service to send the transcriptions.

All the routes used to send requests and responses for our application are in these files.

Modifying Application Routes

Let’s take a closer look at each of the files.

We will go step by step through each of the file, check what routes each of them contains, and learn how to modify these routes and the functions used by the routes.

I. Editing Survey View

The survey_view.py file contains a single route—voice_survey—that the application uses to greet its users and redirect to the questions.

voice_survey Route

We modify the voice_survey route like this:

1. The voice_survey route uses the VoiceResponse() class to build the TwiML response to the call. In FlexML we do not need it, so we can remove this line.

2. The next code portion checks if the request contains any errors: either it targets a non-existent survey or a non-existent question in an existing survey. We remove the response class from the survey_error() function initialization. This is done because we no longer use the response variable based on the VoiceResponse() class, thus we do not need to send it as the function argument. Refer to the section below for information on the survey_error() function. Then we return the response in plain FlexML syntax as a result.

3. We remove the response.say argument from the welcome_user() function and assign its result to the welcome variable. Refer to the section below for information on the welcome_user() function.

4. We remove the response argument from the redirect_to_first_question() function and assign its result to the first_redirect variable. Refer to the section below for information on the redirect_to_first_question() function.

5. Finally, we return the results as a part of the FlexML formatted string.

@app.route('/voice')
def voice_survey():
    response = VoiceResponse()

    survey = Survey.query.first()
    if survey_error(survey, response.say):
        return str(response)

    welcome_user(survey, response.say)
    redirect_to_first_question(response, survey)
    return str(response)

@app.route('/voice')
def voice_survey():
    survey = Survey.query.first()
    if survey_error(survey):
        return f'''
            <Response>
                <Say>{survey_error(survey)}</Say>
            </Response>'''

    welcome = welcome_user(survey)
    first_redirect = redirect_to_first_question(survey)
    return f'''
        <Response>
            <Say>{welcome}</Say>
            {first_redirect}
        </Response>'''

survey_error() Function

The survey_error() function checks if the request contains any errors:

• if the request targets a non-existent survey or a non-existent question in an existing survey, it returns the error text;
• if the request targets an existing question in an existing survey, it returns False.

Thus, we change this function the following way:

1. We remove the send_function argument from the survey_error() declaration, as we return just the error message text.

2. We change the if condition so that now it returns the error message text.

3. The same is for the elif condition, it now also returns the error message text.

def survey_error(survey, send_function):
    if not survey:
        send_function('Sorry, but there are no surveys to be answered.')
        return True
    elif not survey.has_questions:
        send_function('Sorry, there are no questions for this survey.')
        return True
    return False

def survey_error(survey):
    if not survey:
        return 'Sorry, but there are no surveys to be answered.'
    elif not survey.has_questions:
        return 'Sorry, there are no questions for this survey.'
    return False

welcome_user() Function

We change the welcome_user() function the following way:

1. We remove the send_function argument from the welcome_user() declaration, as we now simply return the text.

2. We replace the function that is returned with the text with the survey title.

def welcome_user(survey, send_function):
    welcome_text = 'Welcome to the %s' % survey.title
    send_function(welcome_text)

def welcome_user(survey):
    return 'Welcome to the %s' % survey.title

redirect_to_first_question() Function

We change the redirect_to_first_question() function the following way:

1. We remove the response argument from the function declaration.

2. We return the URL of the first question as a part of the FlexML formatted string. The voice_survey route uses it as a part of the response to the caller.

def redirect_to_first_question(response, survey):
    first_question = survey.questions.order_by('id').first()
    first_question_url = url_for('question', question_id=first_question.id)
    response.redirect(url=first_question_url, method='GET')

def redirect_to_first_question(survey):
    first_question = survey.questions.order_by('id').first()
    first_question_url = url_for('question', question_id=first_question.id)
    return f'<Redirect method="GET">{first_question_url}</Redirect>'

Now that we modified the routes and functions, we can safely remove the Twilio library import declaration from the beginning of the question_view.py file:

from twilio.twiml.voice_response import VoiceResponse
from twilio.twiml.messaging_response import MessagingResponse

Show/hide the resulting survey_view.py file code

from . import app
from .models import Survey
from flask import url_for

@app.route('/voice')
def voice_survey():
    survey = Survey.query.first()
    if survey_error(survey):
        return f'''
            <Response>
                <Say>{survey_error(survey)}</Say>
            </Response>'''
    welcome = welcome_user(survey)
    first_redirect = redirect_to_first_question(survey)
    return f'''
        <Response>
            <Say>{welcome}</Say>
            {first_redirect}
        </Response>'''

def redirect_to_first_question(survey):
    first_question = survey.questions.order_by('id').first()
    first_question_url = url_for('question', question_id=first_question.id)
    return f'<Redirect method="GET">{first_question_url}</Redirect>'

def welcome_user(survey):
    return 'Welcome to the %s' % survey.title

def survey_error(survey):
    if not survey:
        return 'Sorry, but there are no surveys to be answered.'
    elif not survey.has_questions:
        return 'Sorry, there are no questions for this survey.'
    return False

II. Editing Questions View

The question_view.py file contains a single route—question—that the application uses to ask questions.

question Route

The question route takes the ID of the current question as an argument and calls a function that pronounces the question to the caller.

We modify the question route like this:

1. As we use only voice calls, we can remove the if condition because it is only valid if you want to use text messages for the survey.

2. The same is true for the else condition, we also remove it in the scope of this application.

3. Finally, the return statement contains the name of the voice_twiml() function. Let’s replace it with the voice_flexml() to match our migration. We also move this function call one level up because now it is not inside any conditions. Refer to the section below for more information on the voice_flexml() function.

@app.route('/question/<question_id>')
def question(question_id):
    question = Question.query.get(question_id)
    session['question_id'] = question.id
    if not is_sms_request():
        return voice_twiml(question)
    else:
        return sms_twiml(question)

@app.route('/question/<question_id>')
def question(question_id):
    question = Question.query.get(question_id)
    session['question_id'] = question.id
    return voice_flexml(question)

voice_flexml() Function

We change the voice_flexml() function the following way:

1. We replace the voice_twiml function name with voice_flexml, just like we did in the question route.

2. Remove the use of the Twilio VoiceResponse() class.

3. Replace the response.say object with the response_say variable, and change the syntax accordingly.

4. The if condition now contains the FlexML syntax that is assigned to the response_record variable.

5. The else condition contains the FlexML syntax for the response_gather_start and response_gather_end variables.

6. The final return statement combines all the variables into the response that FlexML understands.

def voice_twiml(question):
    response = VoiceResponse()
    response.say(question.content)
    response.say(VOICE_INSTRUCTIONS[question.kind])

    action_url = url_for('answer', question_id=question.id)
    transcription_url = url_for('answer_transcription', question_id=question.id)
    if question.kind == Question.TEXT:
        response.record(action=action_url, transcribe_callback=transcription_url)
    else:
        response.gather(action=action_url)
    return str(response)

def voice_flexml(question):
    response_say = f'<Say>{question.content}</Say>'
    response_say += f'<Say>{VOICE_INSTRUCTIONS[question.kind]}</Say>'

    action_url = url_for('answer', question_id=question.id)
    transcription_url = url_for('answer_transcription', question_id=question.id)
    if question.kind == Question.TEXT:
        response_record = f'<Record playBeep="true" action="{action_url}" transcribeCallback="{transcription_url}"/>'
    else:
        response_gather_start = f'<Gather action="{action_url}">'
        response_gather_end = '</Gather>'
    return f'<Response>{response_gather_start}{response_say}{response_record}{response_gather_end}</Response>'

We also delete the declarations of the is_sms_request(), sms_twiml() functions, and SMS_INSTRUCTIONS.

Now that we modified the routes and functions, we can safely remove the Twilio library import declaration from the beginning of the answer_view.py file:

from twilio.twiml.voice_response import VoiceResponse
from twilio.twiml.messaging_response import MessagingResponse

Show/hide the resulting question_view.py file code

from . import app
from .models import Question
from flask import url_for, session

@app.route('/question/<question_id>')
def question(question_id):
    question = Question.query.get(question_id)
    session['question_id'] = question.id
    return voice_flexml(question)

def voice_flexml(question):
    response_say = f'<Say>{question.content}</Say>'
    response_say += f'<Say>{VOICE_INSTRUCTIONS[question.kind]}</Say>'
    action_url = url_for('answer', question_id=question.id)
    transcription_url = url_for('answer_transcription', question_id=question.id)
    if question.kind == Question.TEXT:
        response_record = f'<Record playBeep="true" action="{action_url}" transcribeCallback="{transcription_url}"/>'
    else:
        response_gather_start = f'<Gather action="{action_url}">'
        response_gather_end = '</Gather>'
    return f'<Response>{response_gather_start}{response_say}{response_record}{response_gather_end}</Response>'

VOICE_INSTRUCTIONS = {
    Question.TEXT: 'Please record your answer after the beep and then hit the pound sign',
    Question.BOOLEAN: 'Please press the one key for yes and the zero key for no and then'
    ' hit the pound sign',
    Question.NUMERIC: 'Please press a number between 1 and 10 and then'
    ' hit the pound sign',
}

III. Editing Answers View

The answer_view.py file contains the instructions that save the answers for the questions into the database, and the logic that allows the application to receive the transcription results from the transcription service and write them to the appropriate database rows.

The file contains two routes:

• the answer route saves the caller’s answer for the questions to the database;
• the answer_transcription route receives the transcription and updates the database with the new values.

answer Route

We modify the answer route like this:

1. Replace the redirect_twiml function name with the redirect_flexml in the if condition.

2. Replace the goodbye_twiml function name with the goodbye_flexml in the else condition.

@app.route('/answer/<question_id>', methods=['POST'])
def answer(question_id):
    question = Question.query.get(question_id)

    db.save(
        Answer(
            content=extract_content(question), question=question, session_id=session_id()
        )
    )

    next_question = question.next()
    if next_question:
        return redirect_twiml(next_question)
    else:
        return goodbye_twiml()

@app.route('/answer/<question_id>', methods=['POST'])
def answer(question_id):
    question = Question.query.get(question_id)

    db.save(
        Answer(
            content=extract_content(question), question=question, session_id=session_id()
        )
    )

    next_question = question.next()
    if next_question:
        return redirect_flexml(next_question)
    else:
        return goodbye_flexml()

answer_transcription Route

We modify the answer_transcription route like this:

1. Change the way the application gets the data from the call. Twilio sends the call data in the form of an immutable MultiDict. In CarrierX, it is pure JSON, which you can receive and parse using the common Flask request module. We change the definition of the session_id and content variables accordingly.

@app.route('/answer/transcription/<question_id>', methods=['POST'])
def answer_transcription(question_id):
    session_id = request.values['CallSid']
    content = request.values['TranscriptionText']
    Answer.update_content(session_id, question_id, content)
    return ''

@app.route('/answer/transcription/<question_id>', methods=['POST'])
def answer_transcription(question_id):
    data = request.get_json()
    session_id = data.get('CallSid','')
    content = data.get('TranscriptionText','')
    Answer.update_content(session_id, question_id, content)
    return ''

extract_content() Function

1. We remove the if condition as we do not need to check if this is an SMS request or a voice call.

2. Replace the elif condition with the if condition.

3. Change the way the route receives the data from the call so that JSON data was parsed and the application could get the value from the Digits key in the else condition.

def extract_content(question):
    if is_sms_request():
        return request.values['Body']
    elif question.kind == Question.TEXT:
        return 'Transcription in progress.'
    else:
        return request.values['Digits']

def extract_content(question):
    if question.kind == Question.TEXT:
        return 'Transcription in progress.'
    else:
        data = request.get_json()
        return data.get('Digits','')

redirect_flexml() Function

1. We replace the redirect_twiml function name with redirect_flexml, just like we did in the answer route.

2. Remove the forming of the Redirect response.

3. Replace the return statement with the one containing the FlexML code for the Redirect verb.

def redirect_twiml(question):
    response = MessagingResponse()
    response.redirect(url=url_for('question', question_id=question.id), method='GET')
    return str(response)

def redirect_flexml(question):
    return f'''
        <Response>
            <Redirect method="GET">{url_for("question", question_id=question.id)}</Redirect>
        </Response>'''

goodbye_flexml() Function

1. We replace the goodbye_twiml function name with goodbye_flexml, just like we did in the answer route.

2. Remove the if condition as we do not need to check if this is an SMS request or a voice call.

3. Remove the else condition as we form the response in any case without any conditions.

4. Replace the return statement with the one containing the FlexML code for the Say and Hangup verbs.

def goodbye_twiml():
    if is_sms_request():
        response = MessagingResponse()
        response.message("Thank you for answering our survey. Good bye!")
    else:
        response = VoiceResponse()
        response.say("Thank you for answering our survey. Good bye!")
        response.hangup()
    if 'question_id' in session:
        del session['question_id']
    return str(response)

def goodbye_flexml():
    if 'question_id' in session:
        del session['question_id']
    return '''
        <Response>
            <Say>Thank you for answering our survey. Good bye!</Say>
            <Hangup/>
        </Response>'''

session_id() Function

1. Change the way the route receives the data from the call so that JSON data was parsed and the application could get the value from the CallSid key. Remove the request.values['MessageSid'] expression that follows the or logical operator.

def session_id():
    return request.values.get('CallSid') or request.values['MessageSid']

def session_id():
    data = request.get_json()
    return data.get('CallSid','')

Now that we modified the routes and functions, we can safely remove the Twilio library import declaration from the beginning of the survey_view.py file:

from twilio.twiml.voice_response import VoiceResponse
from twilio.twiml.messaging_response import MessagingResponse

Show/hide the resulting answer_view.py file code

from . import app, db
from .models import Question, Answer
from flask import url_for, request, session

@app.route('/answer/<question_id>', methods=['POST'])
def answer(question_id):
    question = Question.query.get(question_id)
    db.save(
        Answer(
            content=extract_content(question), question=question, session_id=session_id()
        )
    )
    next_question = question.next()
    if next_question:
        return redirect_flexml(next_question)
    else:
        return goodbye_flexml()

def extract_content(question):
    if question.kind == Question.TEXT:
        return 'Transcription in progress.'
    else:
        data = request.get_json()
        return data.get('Digits','')

def redirect_flexml(question):
    return f'''
        <Response>
            <Redirect method="GET">{url_for("question", question_id=question.id)}</Redirect>
        </Response>'''

def goodbye_flexml():
    if 'question_id' in session:
        del session['question_id']
    return '''
        <Response>
            <Say>Thank you for answering our survey. Good bye!</Say>
            <Hangup/>
        </Response>'''

@app.route('/answer/transcription/<question_id>', methods=['POST'])
def answer_transcription(question_id):
    data = request.get_json()
    session_id = data.get('CallSid','')
    content = data.get('TranscriptionText','')
    Answer.update_content(session_id, question_id, content)
    return ''

def session_id():
    data = request.get_json()
    return data.get('CallSid','')

Finishing Migration

Now that we modified all the views, we can safely remove the importing of Twilio modules from the requirements-to-freeze.txt and requirements.txt files.

Follow the instructions from the application GitHub page to run the application. Then associate the application with the CarrierX phone number, and call that number to check how the application works.

Further Reading

You have successfully migrated the Automated Survey application from TwiML to FlexML!

Refer to the following pages to learn more about FlexML verbs and how to use them, and about ways to set up a FlexML endpoint:

• FlexML API Reference
• FlexML Endpoint Quick Start Guide

Use our Migrating from Twilio to CarrierX Quick Start to learn more about other difficulties you can meet while migrating from Twilio to CarrierX and the ways to solve these issues.

Read other instructions on real-case migrations from Twilio to CarrierX here:

• Send SMS During Calls Application Migration
• Lead Alerts Application Migration
• Server Notifications Application Migration
• Call Forwarding Application Migration

Refer to our other quick start guides for instructions on how to work with CarrierX:

• Rent Phone Number
• Add Trunk Group