FlexML

FlexML Advanced Play

In this guide, you will learn how to build a FlexML application using the Play verb. This advanced application will incorporate all of the optional supported attributes available except for digits, which is an alternative to the Dtmf verb. These attributes allow you to do things like fast-forward, rewind, loop, and pause. We are building our sample application server and routing with Python, but you may use any language you are comfortable with.

Advanced Play Application

In this section, we will build a simple FlexML application, using Python to serve our route.

Create Folder Structure

We use the following folder structure.

flexml_play
    app.py
    static
        instruct.xml

app.py will hold our route, and instruct.xml will hold our FlexML. instruct.xml is inside of the static directory so that it can be accessible to the route we’ll write.

Build Flask Server

Next, we are going to build a Flask server and create our route. We use Flask because it allows us to quickly run a server on our local machine. Start by importing Flask and the Flask request module. request allows our app to process data that was sent to the browser.

Create one route, which serves and prints the FlexML code. Ensure that you add both GET and POST methods.

from flask import Flask, request
app = Flask(__name__, static_url_path='')

@app.route("/", methods=["GET", "POST"])
def home():
    return app.send_static_file('instruct.xml')

if __name__ == '__main__':
    app.run(debug=True)

Write FlexML Application

After we create our route, we will write our FlexML in the instruct.xml document.

Add FlexML Play with Basic Controls

We nest all of our FlexML inside a Response tag and Play verb. Inside the Play tags, add a file of your choosing. In this case, we added a podcast episode downloaded from Bullhorn that we have stored in a CarrierX container. This should be a long file so that you can test fast-forward and reverse commands. Feel free to borrow any of the links included in this tutorial in your sample application.

In the Play start-tag, add the following supported attributes:

controlSkip: the number of seconds that the file should fast-forward or reverse when we enter the forwardOnKey or rewindOnKey keys.
forwardOnKey: the DMTF key that listeners of the file will enter to fast-forward the file.
pauseOnKey: the DTMF key that listeners of the file will enter to pause the file.
rewindOnKey: the DMTF key that listeners of the file will enter to rewind the file.
strictForwardControl: the boolean value that determines whether or not to allow forwardOnKey to reach the end of the file. If this value is set to true, then any forwardOnKey command that would forward past the end (or reach the end) of a file is ignored.

In the following example, you are able to fast-forward and rewind the recording by 15 second increments:

to fast-forward, press 4;
to pause, press 2;
to rewind, press 6.

Finally, strictForwardControl is set to true, meaning that the file cannot be forwarded past the end. If you try to fast-forward past the end a file, the forward command will not work.

<Response>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        pauseOnKey="2"
        rewindOnKey="6"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Test Application

Now let’s test the first lines of our application! You can run your local Flask server through terminal using the following command.

FLASK_APP=app.py flask run

We will also need to expose the route publicly over the Internet. To do this, we can use the free ngrok tool. If you choose to use ngrok, you will need to follow the instructions on their website to download it. Alternatively, you may expose the URL any other way you choose. Once your Flask server is running, your terminal will list the port it’s running on. Expose this port by running the following command through terminal. Make sure to swap out 5000 with the port that your Flask server is running on.

./ngrok http 5000

ngrok will open a new terminal tab that will show you the http and https URLs that make the application publicly available over the Internet. Add the https ngrok link to either a FlexML endpoint or a DID associated with a FlexML endpoint. Refer to the FlexML Endpoint quick start guide to learn how.

Add More Playback Controls

Now that we have seen how to run and test this application, let’s build onto it. In the code following, we’re going to add some controls to restart and stop a file.

To your preexisting code, add the following attributes:

restartOnKey: the DMTF key that listeners will enter to restart the file.
stopOnKey: the DMTF key to stop the playback. Note that you can set multiple DTMF values. If we set the value as 3*, then you can press 3 or * to stop the playback.

In the following example, you will be able to press the 1 key to restart the file. You will also be able to press 3 or * to stop the playback.

<Response>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        pauseOnKey="2"
        restartOnKey="1"
        rewindOnKey="6"
        stopOnKey="3*"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Add Information Parameters

In this next portion of code, we will enable listeners to enter a DTMF key to obtain information about the controls available when interacting with the file.

Add three new attributes to your code:

informationAudioUrl: the URL of the audio that will play when informationOnKey is pressed. After this file has been played once, normal playback resumes.
informationOnKey: the DTMF key that listeners will enter to pause the current playback and play the informationAudioUrl file.
pauseAudioUrl: the URL of the audio to play while playback is paused by pauseOnKey. This audio is played on loop. There is a default comfort noise.

In the following example, you can press the 9 key to play the informationAudioUrl audio file. pauseAudioUrl is the file that will play when the main audio file is paused.

<Response>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        restartOnKey="1"
        rewindOnKey="6"
        stopOnKey="3*"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Add File Looping

Next, we will add functionality to be able to control looping of the file. Add the following to your code:

loop: the number of times that the file will be played.
loopPause: The number of seconds between each repetition.

In the following, the file will play twice because loop is set to 2. 5 seconds will elapse between each loop.

<Response>
    <Play
        loop="2"
        loopPause="5">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/piano.wav
    </Play>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        restartOnKey="1"
        rewindOnKey="6"
        stopOnKey="3*"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Make Audio Play in Background

Now let’s make the URL audio file play in the background. Add the following supported attribute:

background: the boolean value that determines whether or not the playback occurs in the background. If this attribute is set to true, control will pass to the next verb but the audio file will continue to play in the background.

In the following, we have enabled an audio file to play in the background.

<Response>
    <Play
        background="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/spooky_background.mp3
    </Play>
    <Play
        loop="2"
        loopPause="5">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/piano.wav
    </Play>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        rewindOnKey="6"
        restartOnKey="1"
        stopOnKey="3*"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Add Support for Streaming Files

Let’s go over streaming for files that are not prerecorded. We will also set a 10 second time limit for the recording while streaming. Add the following two supported attributes:

streaming: the boolean value that determines whether or not the URL is prerecorded or currently streaming. Set the value to false if the URL is a prerecorded file, and true if the URL is a real-time recording.

Note that setting the streaming value to true will disable forwardOnKey and rewindOnKey if they are defined.
timeLimit: the number of seconds that the stream will last if streaming is set to true.

In the following, rather than playing a prerecorded file, we are streaming. The stream will play for 10 seconds before the code moves onto the next verb.

<Response>
    <Play
        streaming="true"
        timeLimit="10">
        http://us4.internet-radio.com:8193
    </Play>
    <Play
        background="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/spooky_background.mp3
    </Play>
    <Play
        loop="2"
        loopPause="5">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/piano.wav
    </Play>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        restartOnKey="1"
        rewindOnKey="6"
        stopOnKey="3*"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Allow Buffered Playback

Let’s add some buffer to account for the playback loading. Additionally, we’ll add a skip value, which will start a playback after that given number of seconds. Add the following attributes to your code:

preBuffer: the time in seconds that the remote file should be queued before playback is started, to avoid playback catching up with a download in progress.
skip: the number of seconds into a file that it will start playing.

In the following, the playback will start once 5 seconds have been downloaded. And the file itself will start playing 2 seconds in.

<Response>
    <Play
        streaming="true"
        timeLimit="10">
        http://us4.internet-radio.com:8193
    </Play>
    <Play
        background="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/spooky_background.mp3
    </Play>
    <Play
        loop="2"
        loopPause="5">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/piano.wav
    </Play>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        preBuffer="5"
        restartOnKey="1"
        rewindOnKey="6"
        skip="2"
        stopOnKey="3*"
        strictForwardControl="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Add Support for Slow Connections

Next, we’re going to add some values that will account for slow downloads. Let’s add the following attributes:

waitInitial: if the initial download is slow and the initial playback cannot start before this number of seconds, waitInitialUrl will start playing.
waitInitialUrl: the file to play once waitInitial has elapsed.
waitRepeat: after waitInitial has elapsed, and if the playback does not start in this number of seconds, waitRepeatUrl will start playing. Repeat every waitRepeat seconds until Play starts.
waitRepeatUrl: the file to play for waitRepeat.

In the following, waitInitialUrl will start playing if the initial playback cannot start and it has been 5 seconds. If 5 seconds has expired and waitInitialUrl has not started playing, waitRepeatUrl will start. waitRepeatUrl will repeat every 15 seconds until Play starts.

<Response>
    <Play
        streaming="true"
        timeLimit="10">
        http://us4.internet-radio.com:8193
    </Play>
    <Play
        background="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/spooky_background.mp3
    </Play>
    <Play
        loop="2"
        loopPause="5">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/piano.wav
    </Play>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        controlSkip="15"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        preBuffer="5"
        restartOnKey="1"
        rewindOnKey="6"
        skip="2"
        stopOnKey="3*"
        strictForwardControl="true"
        waitInitial="5"
        waitInitialUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/waitInitialUrl.mp3"
        waitRepeat="15"
        waitRepeatUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/waitRepeatUrl.mp3">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Add Links to More FlexML Instructions

Finally, we are going to add a link that enables our app to access more FlexML instructions. We’re also going to add error handling. First, let’s set up a second route for our Flask server. We called this route action and it will return the FlexML instructions kept in our action.xml file.

from flask import Flask, request
app = Flask(__name__, static_url_path='')

@app.route("/", methods=["GET", "POST"])
def home():
    return app.send_static_file('instruct.xml')

@app.route("/action", methods=["GET", "POST"])
def action():
    return app.send_static_file('action.xml')

if __name__ == '__main__':
    app.run(debug=True)

The JSON received during requests will include the following fields that provide you with useful information during testing.

{
    "AccountSid": "",
    "ApiVersion": "2.0",
    "CallerName": "",
    "CallSid": "4f0d84c72ff90967f58351cbe8364a77",
    "Digits": "",
    "Direction": "inbound",
    "From": "19093189030",
    "OriginalFrom": "+19093189030",
    "OriginalTo": "19093189029",
    "PlaybackUrl": "https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3",
    "PlayOffset": 6,
    "RequestUrl": "https://d4d66ed41d49.ngrok.io",
    "To": "19093189029"
}

The following is further information about the Digits, PlaybackUrl, and PlayOffset fields included in the JSON response above.

Attribute	Description
Digits	If the playback was stopped due to `stopOnKey`, this is the key that triggered the end of the playback. If the playback ended without being stopped by a DTMF key, this field will be empty.
PlaybackUrl	The URL that was being played.
PlayOffset	The amount of time in seconds that the URL file played through until it was stopped. Alternatively, if the file played through, the data in this field will be the number of seconds of the entire file.

Add Actions and Callback URLs

Now let’s head back to our instruct.xml file to add some more instructions. Include the attributes below to your app:

action: the URL containing the executable FlexML instructions that will be requested after the Play file ends or has been stopped by entering the value of stopOnKey. We’ve assigned the value as /action, or the new route that we established above.
errorAction: the URL containing the executable FlexML instructions that will be requested if the Play file fails.
errorMethod: the method used to request the errorAction URL containing the executable FlexML instructions if the Play file fails.
method: the HTTP method used to execute the request to the action URL. This method can be either POST or GET.

In the following, the code will make a POST request to the action URL after the Play file ends. In our example, a request will be made to the /action route that we created.

If the Play file fails to play entirely, a POST request will be made to errorAction.

<Response>
    <Play
        streaming="true"
        timeLimit="10">
        http://us4.internet-radio.com:8193
    </Play>
    <Play
        background="true">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/spooky_background.mp3
    </Play>
    <Play
        loop="2"
        loopPause="5">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/piano.wav
    </Play>
    <Say>
        Once the playback starts, press 9 to hear the options menu.
    </Say>
    <Play
        action="/action"
        controlSkip="15"
        errorAction="/errorAction"
        errorMethod="POST"
        forwardOnKey="4"
        informationAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/informationAudioUrl.mp3"
        informationOnKey="9"
        method="POST"
        pauseAudioUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/pauseAudioUrl.mp3"
        pauseOnKey="2"
        preBuffer="5"
        restartOnKey="1"
        rewindOnKey="6"
        skip="2"
        stopOnKey="3*"
        strictForwardControl="true"
        waitInitial="5"
        waitInitialUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/waitInitialUrl.mp3"
        waitRepeat="15"
        waitRepeatUrl="https://storage.carrierx.com/c/4e33f4a8-4c04-4da0-9896-9550164ed296/waitRepeatUrl.mp3">
        https://storage.carrierx.com/c/d8b637fa-efa2-4d4d-817a-6b5f7f33e268/no_sleep_podcast.mp3
    </Play>
</Response>

Now, test your application by calling your rented phone number and using the commands we created!

Next Steps

You built an advanced Play application using all available attributes for the Play verb! For more information about the FlexML Play attributes, refer to the FlexML Documentation.

Once you have finished your advanced Play application, feel free to add more abilities by using other verbs. See the FlexML Syntax section of the FlexML documentation for a comprehensive list of verbs.