Tuesday, June 16, 2015

Call Recordings to Email


Summary

In this article I'll be describing how create a simple call recording service that will record a message from an IVR application and then attach the resulting audio content to an email.  That email could then be routed to an agent in a contact center scenario, for instance.

Environment

Figure 1 depicts the overall architecture.  I created a simple VXML application that provides the voice interface.  That same VXML app sends the collected audio content to a Node-based web service.  The web service repackages the audio content into an email attachment.

Figure 1

Implementation

Figure 2 below depicts the voice and data flow for this architecture.

Figure 2

The web service is built as a simple Node application.  Below is high-level organization of that app.

Figure 3

Finally, Figure 4 depicts the input/output behavior of this web service.

Figure 4

Code Snippets

Voice Application



<?xml version="1.0" encoding="UTF-8"?>
<vxml version="2.1">

  <catch event="error.badfetch.http.400">
    <log label="Error" expr="'HTTP 400 - Bad Request'"/>
    <prompt>
      This request was rejected by the server due to being malformed.  Good bye.
      <break time="1000"/>
    </prompt>
  </catch>
  
  <catch event="error.badfetch.http.413">
    <log label="Error" expr="'HTTP 413 - Request Entity Too Large'"/>
    <prompt>
        This request had an upload that is larger than the server will accept.  Good bye.
        <break time="1000"/>
    </prompt>
  </catch>
  
  <catch event="error.badfetch.http.500">
    <log label="Error" expr="'HTTP 500 - Internal Server Error'"/>
    <prompt>
      The server has experienced an internal error.  Good bye.
      <break time="1000"/>
    </prompt>
  </catch>
  
  <form>
    <var name="ANI" expr="session.callerid" />
    <var name="DNIS" expr="session.calledid" />
    <block>
      <prompt>
        This is a message recording demo.
        <break time="200"/>
      </prompt>
    </block>
    
    <record  name="MSG" beep="true" maxtime="20s" dtmfterm="true" type="audio/mp3">
        <prompt timeout="5s">
          Record a message after the beep.
        </prompt>
        <noinput>
          I didn't hear anything, please try again.
         </noinput>
        <filled>
          <submit next="http://yourwebserver/upload" enctype="multipart/form-data"
            method="post" namelist="ANI DNIS MSG"/>
        </filled>
    </record>
  </form>
</vxml>
Lines 4-26:  Catch logic for HTTP error
Lines 29-30: Saving ANI and DNIS in variables for use later in the HTTP form post.
Line 38:  VXML tag used to record a caller.  Output will be in MP3 format.
Line 46:  Finally, when the recording has ended - send it and the ANI + DNIS via an HTTP multipart form.

Web Service (main body of code)

appHttp.post('/upload', function(req, res) {
          try {
            logger.debug('Entering - File: main.js, Method: appHttp.post()');
                        
            var form = new multiparty.Form();
            var ani = null;
            var dnis = null;
            var fname = null;
            var msg = null;
            var size = 0;
                        
            form.on('error', function(err, statCode) {
              logger.error('File: main.js, Method: appHttp.post(), form(), Error: ' + err.message);
              res.status(statCode || 400).end();
            });
                        
            form.on('part', function(part) {
              var data=[];
                          
              part.on('error', function(err, statCode) {
                form.emit('error', err, statCode);
              });
                          
              part.on('data', function(chunk) {
                size += chunk.length;
                if (size > properties.maxUploadSize) {
                  //covers a degenerate case of too large of an upload.  Possible DOS attempt
                  part.emit('error', new Error('Upload exceeds maximum allowed size'), 413);
                }
                else {  
                  data.push(chunk);
                }
              });
                         
              part.on('end', function() {
                switch (part.name) {
                  case 'ANI':
                    ani = data.toString();
                    break;
                  case 'DNIS':
                    dnis = data.toString();
                    break;
                  case 'MSG':
                    if (part.filename) {
                      fname = part.filename;
                      msg = Buffer.concat(data);
                    }
                    else {
                      part.emit('error', new Error('Malformed file part in form'), 400);
                    }
                    break;
                  default:
                    part.emit('error', new Error('Unrecognized part in form'), 400);
                    break;
                }      
              });
            });
                        
            form.on('close', function() {
              if (ani && dnis && fname && msg) {
                res.status(200).sendFile(__dirname + '/vxml/response.vxml');
                var mailOptions = {
                  from : properties.emailFromUser,
                  to : properties.emailToUser,
                  subject : 'Recorded Message - ANI:' + ani + ', DNIS:' + dnis,
                  text : 'The attached recorded audio message was received.',
                  attachments : [{filename : fname, content : msg}]
                };
                
                transporter.sendMail(mailOptions, function(err, info) {
                  if (err) {
                    appHttp.emit('error', err);
                  }
                  logger.debug('Exiting - File: main.js, Method: appHttp.post()');
                });    
              }
              else {
                form.emit('error', new Error('Form missing required fields'), 400);
              }             
            });
                        
            form.parse(req); 
          }
Line 1:  This is the Express route for an 'upload' POST.
Line 5:  The multiparty node module is used for processing the POST'ed form data.
Lines 24-33:  Compile the form 'chunks' that are uploaded.  If an upload is being attempted that is larger than a user-configured maximum limit, terminate the upload.  Based on my testing, simply emitting an error is enough to cause Node/Express to terminate an upload.  I saw no need for something like 'req.connection.destroy()'.
Lines 35-57:  When a form 'part' has been completely uploaded, determine which 'part' it was and save it into local variables.
Lines 59-80:  When the entire form has been completely uploaded, determine if all the expected 'parts' were included.  If so, send back a 200 OK with simple VXML response.  Then, send the 'parts' out as an email.  The ANI and DNIS are put in the subject line of the email.  The audio content is sent as an attached file.

Output

Snippet of the resulting email output below:
From: yourFromAddress@gmail.com
To: yourToAddress@gmail.com
Subject: Recorded Message - ANI:1234567890, DNIS:9876543210
X-Mailer: nodemailer (1.3.4; +http://www.nodemailer.com;
 SMTP/1.0.3[client:1.2.0])
Date: Tue, 16 Jun 2015 01:02:06 +0000
Message-Id: <1434416526890-858ca434-7a77d97c-8d53507c data-blogger-escaped-gmail.com="">
MIME-Version: 1.0

------sinikael-?=_1-14344165263510.8602459693793207
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

The attached recorded audio message was received.
------sinikael-?=_1-14344165263510.8602459693793207
Content-Type: audio/mpeg
Content-Disposition: attachment; filename=MSG-1434416526064.mp3
Content-Transfer-Encoding: base64


Monday, June 1, 2015

Broadband Connectivity Monitor


Summary

In this article I'll be demonstrating a way for keeping tabs on your Internet connectivity.  Anyone that  has had challenges with their ISP and uptime knows what I'm talking about.
I looked at several external services (Pingdom, UptimeRobot, etc) and other folks' code, but didn't see anything that I particularly liked.  They wanted money, monitoring intervals were too long, etc.  Instead, I just wrote a fairly simple Linux shell script myself to do the job.

Design goals:
  • Simple to deploy/use
  • One-minute monitoring granularity
  • Logging with sufficient detail that I can go back to my ISP and get refunds for service disruptions
  • Email alerts

Environmentals

The two main requirements for this script are the Bash shell and a Mail Transfer Agent (MTA).

The MTA requirement is to support transmission of email alerts.  I used the Heirloom Mailx agent in testing on both Debian (Ubuntu) and Red Hat (Centos) environments.  My ISP blocks direct SMTP traffic (spam prevention no doubt), so I needed a MTA that would support use of external SMTP services (i.e., relay) for outbound emails.  Mailx provides that.  

I decided to use Google's email service (GMail) for the relay.  Below is the configuration I have working for Ubuntu (nail.rc file):
set smtp-use-starttls
set smtp=smtp://smtp.gmail.com:587
set ssl-verify=ignore
set smtp-auth=login
set smtp-auth-user="yourEmailAddress@gmail.com"
set smtp-auth-password="yourPassword"
set from="yourEmailAddress@gmail.com"

For Centos, I had to add the following line in addition the ones above (mail.rc file in this case):
 set nss-config-dir="/etc/pki/nssdb"

Implementation

As mentioned previously, I wrote this monitor completely in Linux shell script.  The overall program logic is as follows (loops forever at a user-configurable time interval):
  • Send an ICMP echo request (Ping) to a user-configurable target.
  • If the target replies, do nothing.  
  • If the target does not reply, I've experienced a broadband/Internet connectivity outage.  Log the details locally.
  • If we've recorded an outage and now have a successful ping, calculate the service disruption time, log it, and send an email alert that the outage occurred.  Since I'm doing the monitoring locally, there's no need to attempt an email alert till connectivity is restored, for obvious reasons.

Main body of the shell script below:
while :
do
 results=`ping -qc $COUNT $TARGET`
 case "$?" in
  0) if [ "$failedTime" -ne 0 ]
   then
    restoredTime=`date +%s`
    duration=$(( $restoredTime - $failedTime ))
    s=$(( duration%60 ))
    h=$(( duration/3600 ))
    (( duration/=60 ))
    m=$(( duration%60 ))
    
    logRec="Service Restored, Approx Outage Duration:"
    logRec+=`printf "%02d %s %02d %s %02d %s" "$h" "hrs" "$m" "min" "$s" "sec"`
    logger -t $(basename $0) "$logRec"
    t1=`date -d @$failedTime -I'seconds'`
    t2=`date -d @$restoredTime -I'seconds'`
    printf "%s %s\n%s %s" "$t1" "$msg" "$t2" "$logRec" | mail -s "Service Outage Occurred" $EMAIL
    failedTime=0
    internalError=0
   fi
   ;;
  1) if [ "$failedTime" -eq 0 ]
   then
    failedTime=`date +%s`
    logRec=`echo "Service Outage:" "$results" | tr '\n' ' '`
    msg=$logRec
    logger -t $(basename $0) "$logRec"
    internalError=0
   fi
   ;;
  *)
   if [ "$internalError" -eq 0 ]
   then
    logger -t $(basename $0) "Internal Error"
    (( internalError+=1 ))
   fi
   ;;
 esac
 
 sleep $INTERVAL
done

Line 1:  Loop, like forever.
Line 3:  Executes the ping command with a user-configurable ping count and target.  Those settings are stored in an external config file.
Line 4:  Set up a switch on the return value of the ping command.  Per the man page, ping will return 0 if it gets a reply, 1 if it gets no reply at all, and 2 on any other sort of error.
Line 5:  This would be the case that ping received a reply.  I only need to take action if there has been an outage recorded earlier.  That outage flag is the time of occurrence, stored in the failedTime variable.
Line 7: An outage and resulting service restoration is in progress.  Store the time of the restoration (in seconds since 1970).
Lines 8-12:  Calculate the total duration of the outage using difference between the start and stop times.  Take that duration time that is in seconds and do some arithmetic to convert it to hours, minutes, and seconds.
Lines 14-15:  Do some prettifying of a log message of the service restoration notice.
Line 16:  Send the notice to the syslog process on the local server.
Lines 17-18:  Put some timestamps on the message that will be sent as an email alert (syslog does this automatically, so I didn't need to timestamp the log messages).
Line 19:  Send alert message out via email.
Lines 20-21:  Reset some variable flags.
Line 24:  The case here is ping has returned a "1", meaning it did not receive a reply.  If the failedTime flag is not set, this indicates a fabulous new outage event.
Line 26:  Save the current time (in seconds since 1970) in the failedTime variable.
Line 27:  Concatenate a string with that contains the output of the original ping command.  Remove all newlines in that string (syslog logging is 1 line at a time).
Line 28:  Save that outage string in a variable for use later for the email alert when connectivity has been restored.
Line 29:  Send the log message to syslog.
Line 34:  This covers the degenerate case (ping return code of "2").  A scenario where this may happen would be the local server interface went down.
Line 36:  Simply log a message of the issue, but only do it one time.
Line 42:  Pause till the next ping for a user-configurable amount of time.

This script can be fired off and run forever simply like this:
nohup ./ispmon.sh > /dev/null 2>&1 &
For those more motivated, you can set this up as a regular Linux daemon in init.d.

Output


Sample syslog output below:
Jun  2 04:58:31 intel3770k ispmon.sh: Service Outage: PING 192.168.100.1 (192.168.100.1) 56(84) bytes of data.  --- 192.168.100.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms 
Jun  2 04:59:33 intel3770k ispmon.sh: Service Restored, Approx Outage Duration:00 hrs 01 min 02 sec

Email alert text from the sample above:
2015-06-02T04:58:31-0600 Service Outage: PING 192.168.100.1 (192.168.100.1) 56(84) bytes of data.  --- 192.168.100.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms 
2015-06-02T04:59:33-0600 Service Restored, Approx Outage Duration:00 hrs 01 min 02 sec
Full source here.