How It Works

Storing Files with CarrierX

CarrierX provides means to store files within CarrierX infrastructure. This is done to more easily save the recordings of your conferences, voicemails, and FlexML-related recordings and files.

Storage Structure

All the files within CarrierX infrastructure are stored inside containers. We provide free 1 GB storage per container to each partner when the partner is created.

General Structure

The typical storage structure looks like this:

Storage

Here, each partner has a container associated with their account. The partner uses this container for storing files (either created by endpoints or uploaded). Some endpoints can also create and store files in this container, e.g., Voicemail, FlexML, or Conference. Refer to the sections below for more information on how to use CarrierX storage with each of the endpoint types.

Storage Objects Explained

API objects used with CarrierX storage include Container and File objects. The Container object is associated with the partner account via the partner_sid attribute (used in the Partner and Container objects), and the File object is connected to the parent Container object via the container_sid attribute:

Storage Objects

The Endpoint object is also associated with the partner account via the partner_sid attribute.

You can also use specific containers with some of the endpoints. To do that, you need to additionally specify the container_sid attribute for the endpoint of your choice in its properties attribute. Once you do this, the endpoint will have access to the container you specify and will be able to save files to it, or access files by their secure ID (the file_sid attribute of the File object). If you do not do that, the endpoint will save files to the default container that your account has associated with it.

Each of the supported endpoints (FlexML, Conference, Conference Playback, and Voicemail) has some specifics when you associate it with the containers. Refer to the sections below for more details.

FlexML Endpoint

Normally, FlexML endpoints use the containers to store the files recorded using the FlexML Record verb. You need to either associate the container secure ID (the container_sid attribute) with the FlexML endpoint, or specify it using the containerSid attribute of the Record verb (in this case, you must do this every time you use the Record verb in your FlexML application syntax). Either way, the files will be stored to the container when the FlexML application records the call.

Storage Objects - FlexML

FlexML also allows you to use the files stored with CarrierX with the FlexML Play verb. You can either use the links to the files stored in containers directly inside the Play verb (the publish_uri attribute of the File object), or use the file secure ID (the file_sid attribute of the File object) with the File noun nested inside the Play verb.

The FlexML application will playback the audio file you specify to the caller.

Conference and Conference Playback Endpoints

If you use a standalone Conference endpoint, it follows the general storage structure to store the files and recordings: the system associates the Partner object with the containers and files, and the same Partner object is associated with the Conference endpoint. The endpoint properties contain the reference to the container secure ID and it allows saving the endpoint-related files to the associated container.

But when it comes to the Conference Playback endpoint, you need some additional steps to make sure that the Conference Playback endpoint has the access to the files which contain the recordings of the Conference endpoint:

Storage Objects - Conference and Conference Playback Endpoints

The Conference Playback endpoint does not have a container_sid attribute in its properties. Therefore, you cannot directly associate a Conference Playback endpoint with a container so that it could get access to the recordings files.

That is why the Conference Playback endpoint properties attribute contains the reference to the Conference endpoint (the conference_endpoint_sid attribute) to which files it must have access. Only the referenced Conference endpoint has direct access to the container with the recordings.

Voicemail Endpoint

A Voicemail endpoint also follows the general storage structure to store the recordings: the system associates the Partner object with the containers and files, and the same Partner object is associated with the Voicemail endpoint. The endpoint properties contain the reference to the container secure ID and it allows saving the endpoint-related files to the associated container.

Storage Objects - Voicemail

You can additionally use the container to store not only the voicemail recordings, but also greeting files which you can use to greet the callers. To link the greeting, use the publish_uri attribute of the File object and specify it in the greeting attribute of the Voicemail endpoint properties.

The Voicemail endpoint will playback the audio file you specify to the callers.

Uploading and Downloading Files

CarrierX storage server allows the users both to upload and download files using the API requests.

If you need additional control of the file, which might include range requests, content or cache control, etc., access the file in the container using either the file secure ID—the file_sid attribute (FlexML applications have such an ability, refer to the section above for more details on this) or the publicly accessible publish_uri link.

Uploading Files to Container

To upload a file to CarrierX storage, you need the file itself and the secure ID of the container where you are going to store the file.

Create Container

If you do not have a container, you can create it using the following POST request:

curl -X POST \
'https://api.carrierx.com/core/v2/storage/containers' \
-H 'Content-Type: application/json' \
--data-binary '{}' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

A successful request will return a 200 status code with the serialized copy of the created Container object.

Upload Files

CarrierX supports two ways to upload a file to a container using a POST request:

  1. Using multipart/form-data content type.

    If you use this method, you need to pass the File object parameters as the file_object form argument. The only required parameter here is container_sid. It defines the secure ID of the container to which the file must be uploaded.

    You need to pass the file itself as the data form argument. It must contain the valid path to the file to be uploaded.

     curl -X POST \
 'https://api.carrierx.com/core/v2/storage/files' \
 -H 'Content-Type: multipart/form-data' \
 -F 'file_object={"container_sid":"cb05c1f4-2398-432c-8314-cc306156281c"}' \
 -F 'data=@my_audio.mp3' \
 -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

  2. Using application/octet-stream content type.

    If you prefer to use this method, you need to pass the File object parameters as query arguments. The only required parameter here is also container_sid. It defines the secure ID of the container to which the file must be uploaded.

    You need to pass the file itself as raw data. It must contain the valid path to the file to be uploaded.

     curl -X POST \
 'https://api.carrierx.com/core/v2/admin/storage/files?container_sid=cb05c1f4-2398-432c-8314-cc306156281c&name=my_audio.mp3' \
 -H 'Content-Type: application/octet-stream' \
 --data-binary '@my_audio.mp3' \
 -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

A successful request for any of the above methods will return a 201 status code with the serialized copy of the created File object.

After that, you can access the file using either the file_sid or the publish_uri attribute, depending on your need or the application settings. Refer to the sections above for more information on how different endpoints access the files.

Downloading Files

There are several ways to download the files from CarrierX storage.

  1. You can get either information about the file, or the download the file data using API requests.

    • To get the detailed information about the file, use the following GET request, which will return a 200 response together with a serialized copy of the File object:
      curl -X GET \
  'https://api.carrierx.com/core/v2/storage/files/f6070be2-9abd-4726-870c-c86c9aac5c7e' \
  -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'
    • To download the file data, use the following GET request, which will return a 200 response and a download of the file data:
      curl -X GET \
  'https://api.carrierx.com/core/v2/storage/files/f6070be2-9abd-4726-870c-c86c9aac5c7e/data' \
  -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

  2. Neither of the above requests to the API supports range requests. Use the publicly available link to the file (the file publish_uri attribute) when you need to request only a portion of it:

     curl -X GET \
 'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2' \
 -H 'Range: bytes=0-499'

    Refer to the section below for more information about range requests to files in CarrierX storage server.

Range Requests

Range requests are partial requests which allow you to request and playback only a portion of the file from any particular place (e.g., from the fourth to twenty-first second, etc.) This might be useful when searching a specific place in the media file, as you can playback the file from a certain place. If range requests are not supported, the software will download the whole file and the playback will always start from the beginning of the file.

When you store a file in one format but an application requests automatic transcoding during download, and tries to do range requests at the same time, it might cause issues. If the system does not have a full cached copy of this file locally already transcoded, it will return the entire file with a 200 response code and the playback will start from the beginning of the file, not from the place the application requires.

Range requests are supported for the so called “streaming” formats. Such formats do not contain any metadata required for the file playback at the end of the file so the applications do not need to completely download them before they start the playback and they can start the playback from any position in the file after they download only the required portion of the file. Unlike them, the so called “downloadable” file formats contain some metadata required for the correct file playback at the end of the file.

When you need range requests to receive only a portion of a file, use the direct request to the CarrierX storage server instead of the API requests. You can do this via the publicly available link to the file (the file publish_uri attribute).

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2' \
-H 'Range: bytes=20-499'

To confirm that CarrierX storage server supports range request, it returns the following Accept-Ranges header to the client software:

Accept-Ranges: bytes

In its turn, the application that you use to download the portions of the files must specify which part of the file it wants to receive. The application can do this with the help of the Range header:

Range: bytes=20-499

The response from the server will include a similar Content-Range header in the case of the successful range request.

The complete response from the server for a range request to the file will look like the following:

HTTP/1.1 206 PARTIAL CONTENT
Accept-Ranges: bytes
Access-Control-Allow-Methods: GET
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Access-Control-Allow-Methods,Access-Control-Allow-Origin,Content-Disposition
Cache-Control: None
Connection: close
Content-Length: 480
Content-Range: bytes 20-499/189952
Content-Type: audio/G722
Content-Disposition: inline; filename*=UTF-8''recording.g722
Date: Tue, 01 Jun 2021 14:21:07 GMT
Last-Modified: Tue, 25 May 2021 13:12:44 GMT
Server: nginx

You can additionally use the If-Range client-side header together with the Range header to make sure that the file was not modified between the requests. This header will compare its value against the server Last-Modified file header value.

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2' \
-H 'Range: bytes=20-499' \
-H 'If-Range: Tue, 25 May 2021 13:12:44 GMT'

If the values are equal, the server will accept the range request and will return the 206 response together with the partial content download:

HTTP/1.1 206 PARTIAL CONTENT
Accept-Ranges: bytes
Content-Range: bytes 20-499/189952
Last-Modified: Tue, 25 May 2021 13:12:44 GMT

Otherwise, if the Last-Modified value changes between the requests, the server will return the 200 response together with the complete file download. No Content-Range header will be present in the server response in this case:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Last-Modified: Mon, 05 Apr 2021 14:51:53 GMT

Additional Parameters

To control your files while downloading, you can use additional parameters for your requests. CarrierX web storage supports request parameters to control the response server-side content-related headers (Content-Disposition, Content-Type) and cache-related headers (Cache-Control).

Content Headers

You can set the content headers on the server side and the server will respond with the disposition type and file name you specify in the download request.

disposition and disposition_name Parameters

The disposition and disposition_name parameters set the server Content-Disposition header to the desired value.

They are provided as request query parameters:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?disposition=attachment&disposition_name=my_file.wav'

If the request is successful, the server will return the modified Content-Disposition header in the response:

Content-Disposition: attachment; filename*=UTF-8''my_file.wav

Values accepted for the disposition parameter are:

The default value for the disposition parameter is inline.

content_type Parameter

The content_type parameter sets the server Content-Type header to the desired value. The Content-Type header is normally used to tell the application what the content type of the returned file actually is.

It is provided as a request query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?content_type=audio/G722'

If the request is successful, the server will return the modified Content-Type header in the response:

Content-Type: audio/G722

Values accepted for the content_type parameter for various formats are:

Format Content-Type
al, alaw audio/basic
flac audio/flac
g722 audio/G722
mp3 audio/mpeg
mp4 video/mp4
ogg audio/ogg
ul, ulaw audio/basic
wav audio/x-wav

Cache Headers

CarrierX storage server uses cache headers to control the file caching mechanism. The client application can change the server caching policies and define its own caching policies.

By default, when you use none of the cache parameters in the request, the server will return the Cache-Control header with the value set to None, meaning that it uses no cache control:

Cache-Control: None
cache_headers Parameter

The cache_headers parameter sets the server Cache-Control header to the desired value.

It is provided as a request query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?cache_headers=3600'

If the request is successful, the server will return the modified Cache-Control header in the response:

Cache-Control: public, max-age=3600

Values accepted for the cache_headers parameter are:

cache_control Parameter

The cache_control parameter also sets the server Cache-Control header to the desired value.

It is provided as a request query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?cache_control=min-fresh=3600'

If the request is successful, the server will return the modified Cache-Control header in the response:

Cache-Control: min-fresh=3600

Values accepted for the cache_control parameter are:

Transcoding Parameters

CarrierX web storage allows its users to send requests with special parameters which will instruct the server to transcode the files into either a different format or to change its bitrate on-the-fly during downloading.

These parameters include the f (format) and br (bitrate) parameters. Refer to the Transcoding Files During Download section for the detailed information on this.

Transcoding Files

CarrierX allows uploading and downloading the files not in the file original format, but in a format which the system supports for conversion or transcoding. It might be useful when you need to store the file in the container in a format different from that in which the file was originally created. You can also use on-the-fly transcoding when you download a file from the container but your application needs to receive it in some other format.

CarrierX supports automatic transcoding from and to most popular media formats used with telephony and conferencing during upload or during download. The table below shows the codecs that the system can transcode the files from and to:

Format Transcode from Codec Transcode to Codec
al, alaw yes yes
flac yes yes
g722 yes yes
mp3 yes yes
mp4 no yes
ogg yes yes
ul, ulaw yes yes
wav yes yes

Transcoding Files During Upload

When you upload a file to the container and if you do not specify any additional parameters, the file will be uploaded without any format changes. In most cases it is okay, but there can be situations (especially with the media files), when we recommend that you store the file in the format that is going to be served by the application.

In most cases, it is not necessary to reformat or transcode media files prior to uploading them. With media files, you can select the format in which the file will be stored with CarrierX storage. In this case, it will be transcoded to the selected format during the upload, on the fly.

You can transcode the file during upload using the File object desired_format attribute like this:

curl -X POST \
'https://api.carrierx.com/core/v2/storage/files' \
-F 'file_object={"container_sid":"cb05c1f4-2398-432c-8314-cc306156281c","type":"audio","desired_format":"g722"}' \
-F 'data=@my_audio.mp3' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

Transcoding from one format to another is useful not only for the sake of compatibility, but also when you want to reduce the file size. For example, files in the wav format can take considerable storage space when stored and the network bandwidth when accessed via the internet. It might be not too good on slower connections. Transcoding them to, for example, the mp3 format will dramatically reduce the file size and network bandwidth.

To further reduce the file size when uploading mp3 files and not transcoding them to any other format, you can set the File object desired_bitrate to a smaller value. You can set this field to one of these values: 8, 16, 24, 32, 40, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320.

You can change the file bitrate during upload like this (do not forget to add the "type":"audio" parameter when sending the POST request):

curl -X POST \
'https://api.carrierx.com/core/v2/storage/files' \
-F 'file_object={"container_sid":"cb05c1f4-2398-432c-8314-cc306156281c","type":"audio","desired_bitrate":"32"}' \
-F 'data=@my_audio.mp3' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

Transcoding Files During Download

When you download the file from CarrierX storage service, you can also transcode it into a necessary format and change its bitrate.

You can transcode the files into another format using the f query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?f=mp3'

This parameter accepts the values from the Transcode to Codec column of the table here (al, alaw, flac, g722, mp3, mp4, ogg, ul, ulaw, and wav).

To change the file bitrate, use the br query parameter when downloading the file:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?br=64

You can set the br parameter to one of these values: 8, 16, 24, 32, 40, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320.

File Classifications

Partners (and sometimes their clients) can upload different types of files to the containers. To protect the containers from the harmful or unwanted file types, CarrierX introduced the file classifications option. It serves to block certain file types or allow only files of certain types.

The file classification works the following way:

File Classifications

Unique Files and Containers

If for some reason you want to make sure that you store unique files in the containers, you can use the unique attribute of the File object. If you set it to true, the file with such an attribute must have the unique key set.

The key set is the total of the values of the following six attributes:

This key set must be unique for each file that has the unique attribute set to true (i.e., at least one of the mentioned key values for two unique files must differ and be unique):

Unique Files

If you try to upload a file with the unique attribute set to true and the same file key set, which has already been used with some other file, a 422 (object validation error) error will be returned and the file will not be uploaded:

{
    "message":"object validation error",
    "errors":[
        {
            "field":"unique",
            "message":"file with the same set of keys already exists",
            "reference_sid":"f6070be2-9abd-4726-870c-c86c9aac5c7e"
        }
    ]
}

Refer to the File Object Specific Errors section for more information about the error structure and your possible actions to avoid it.

The same rule applies to the unique containers: to create a unique container all the other containers with the unique attribute set to true must have a different key set (the container key set also consists of the integer_key_1, integer_key_2, string_key_1, string_key_2, string_key_3, and string_key_4 attributes).

File Management

CarrierX allows its partners to set the file upload threshold and automatically manage the uploaded files when this threshold is reached.

The following three options are available:

  1. Set no threshold.

  2. Set a threshold and reject uploading files when the set threshold is exceeded.

  3. Set a threshold and run cleanup jobs when the set threshold is exceeded.

Set No Threshold

When you do not set a threshold, you can upload files to the container until either the quota_files or quota_bytes value is reached. After that the container object will return errors if you try to upload new files.

This is the default option, you do not need to set any threshold attributes for it.

Set Threshold and Reject Exceeding Files

You can set a file upload threshold (either the number of files or their size in bytes). Once this threshold is exceeded, no new files will be uploaded until you delete some of the previously uploaded files.

To enable this option, do the following:

To set the threshold with the reject action for an existing container, send the following PATCH request:

curl -X PATCH \
'https://api.carrierx.com/core/v2/storage/containers/a291fe99-43d6-446f-a63e-91f93030dc7e' \
-H 'Content-Type: application/json' \
--data-binary '{"threshold_action":"reject","threshold_files":90,"threshold_grace_period":300}' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

This will update the existing container and set the threshold action to reject. The grace period for the uploaded files will be equal to 300 seconds (5 minutes) during which the system will allow uploading files to the container even when the threshold of 90 files is exceeded.

This means that the number of files that you will be able to upload to the container might be 91 or 92 or even more if they all are uploaded during the 5 minutes following the moment when the threshold is reached. You can delete these files manually later if you need to.

After the grace period is also exceeded, all the files will be rejected from uploading to the container.

You can control the current threshold status using the Container object threshold_total_bytes/ threshold_total_files or the threshold_available_bytes_percent/ threshold_available_files_percent attribute values. These attributes will show you the current size or number of the files which affect the threshold, or the percentage of the bytes/files remaining until the threshold is exceeded.

Set Threshold and Delete Files Automatically

If you periodically upload files to the container and do not need them after some time, or you want to automatically get rid of the files which are no longer accessed and used, you can use the storage cleanup jobs to do that for you.

Set a threshold and, once it is exceeded, the cleanup job will detect and delete some of the least recently used (LRU) files freeing the storage space for new files.

To enable this option, do the following:

To set the threshold with the delete_lru action for an existing container, send the following PATCH request:

curl -X PATCH \
'https://api.carrierx.com/core/v2/storage/containers/a291fe99-43d6-446f-a63e-91f93030dc7e' \
-H 'Content-Type: application/json' \
--data-binary '{"threshold_action":"delete_lru","threshold_files":90,"threshold_files_target":70}' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

This will update the existing container and set the threshold action to delete_lru. When the number of files in the container reaches 90, the storage cleanup job deletes the least recently used files (the ones with the oldest date_last_accessed attribute value) until their number is equal to 70.

If you want, you can also control the current threshold status using the Container object threshold_total_bytes/ threshold_total_files or the threshold_available_bytes_percent/ threshold_available_files_percent attribute values. These attributes will show you the current size or number of the files which affect the threshold, or the percentage of the bytes/files remaining until the threshold is exceeded.

In practice, this is not necessary as all the management is performed automatically in this case.

Using External Storage Services

If you prefer to store the files with one of the cloud services (AWS S3, Google Cloud Platform, or Microsoft Azure), CarrierX allows you to do this instead of using the standard containers hosted with CarrierX. You can also choose to use the external storage services if you want to avoid the CarrierX limitation to 1 GB of storage per container.

As is mentioned above, the following external storage services are supported for containers:

You will need to provide the credentials and specify the storage location for each of the external storage used. You can use only one external storage at a time for a single container.

For this, set the container type to either aws (for Amazon Simple Storage Service), or azure (for Microsoft Azure Blob storage), or gcp (for Google Cloud Platform).

Refer to the Supported External Storage section of the Core API documentation for more information on the parameters you need to specify for each storage type.

Further Reading

Container and File Objects API Reference

Refer to the Container and File objects API reference to get the complete list of their attributes and methods used with them:

Container Object
File Object

How It Works

Read the following articles to get a better understanding of how things work in CarrierX: