Wednesday, February 28, 2018

AWS Lex Bot & Genesys Chat Integration


Summary

This post is the culmination of the posts below on Lex + Genesys chat builds.  In this one, I'll discuss how to build a web client interface that allows integration of the two chat implementations.  The client will start out in a bot session with Lex and then allow for an escalation to a Genesys Agent when the end-user makes the request for an agent.

Genesys Chat build:  http://joeywhelan.blogspot.com/2018/01/genesys-chat-85-installation-notes.html
AWS Lex Chat build: http://joeywhelan.blogspot.com/2018/02/aws-lex-chatbot-programmatic.html
Reverse Proxy to support GMS: http://joeywhelan.blogspot.com/2018/02/nodejs-reverse-proxy.html

Architecture Layer

Below is a diagram depicting the overall architecture.  AWS Lex + Lambda is utilized for chat bot functionality; Genesys for human chat interactions.  A reverse proxy is used to provide access to Genesys Mobility Services (GMS).  GMS is a web app server allowing programmatic access to the Genesys agent routing framework.

Transport Layer

Secure transport is used through out the architecture.  HTTPS is use for SDK calls to Lex.  The web client code itself is served up via a HTTPS server.  Communications between the web client and GMS are proxied and then tunneled through WSS via Cometd to provide support for asynchronous communcations between the web client and Genesys agent. 

Application Layer

I used the 'vanilla demo' included with the Cometd distro to build the web interface.  All the functionality of interest is contained in the chat.js file.  Integration with Lex is via the AWS Lex SDK.  Integration with Genesys is via publish/subscribe across Cometd to the GMS server.  GMS supports Cometd natively for asynchronous communications.


Application Flow

Below are the steps for an example scenario:  User starts out a chat session with a Lex bot, attempts to complete an interaction with Lex, encounters difficulties and asks for a human agent, and finally transfer of the chat session to an agent with the chat transcript.


Step 1 Code Snippets

        // Initialize the Amazon Cognito credentials provider
        AWS.config.region = 'us-east-1'; // Region
        AWS.config.credentials = new AWS.CognitoIdentityCredentials({
            IdentityPoolId: 'us-east-1:yourId',
        });
        var _lexruntime = new AWS.LexRuntime();

        function _lexSend(text) {
         console.log('sending text to lex');
            var fromUser = _firstName + _lastName + ':'; 
            _displayText(fromUser, text);
        
            var params = {
              botAlias: '$LATEST',
        botName: 'OrderFirewoodBot',
        inputText: text,
        userId: _firstName + _lastName,
       };
            _lexruntime.postText(params, _lexReceive);
        }
Lines 1-6:  Javascript. AWS SDK set up.  An AWS Cognito pool must be created with an identity with permissions for the the Lex postText call.

Step 2 Code Snippets

RequestAgent Intent

An intent to capture the request for an agent needs to be added to the Lex Bot.  Below is a JSON-formatted intent object that can be programmatically built in Lex.
{
    "name": "RequestAgent",
    "description": "Intent for transfer to agent",
    "slots": [],
    "sampleUtterances": [
       "Agent",
       "Please transfer me to an agent",
       "Transfer me to an agent",
       "Transfer to agent"
    ],
    "confirmationPrompt": {
        "maxAttempts": 2,
        "messages": [
            {
                "content": "Would you like to be transferred to an agent?",
                "contentType": "PlainText"
            }
        ]
    },
    "rejectionStatement": {
        "messages": [
            {
                "content": "OK, no transfer.",
                "contentType": "PlainText"
            }
        ]
    },
    "fulfillmentActivity": {
        "type": "CodeHook",
        "codeHook": {
         "uri" : "arn:aws:lambda:us-east-1:yourId:function:firewoodLambda",
      "messageVersion" : "1.0"
        }
    }
}

Lambda Codehook

Python code below was added to the codehook described in my previous Lex post.  Lines 3-6 add an attribute/flag that can be interrogated on the client side to determine if a transfer to agent has been requested.
    def __agentTransfer(self):
        if self.source == 'FulfillmentCodeHook':
            if self.sessionAttributes:
                self.sessionAttributes['Agent'] = 'True';
            else:
                self.sessionAttributes = {'Agent' : 'True'}
            msg = 'Transferring you to an agent now.'
            resp = {
                    'sessionAttributes': self.sessionAttributes,
                    'dialogAction': {
                                        'type': 'Close',
                                        'fulfillmentState': 'Fulfilled',
                                        'message': {
                                            'contentType': 'PlainText',
                                            'content': msg
                                        }
                                    }
                    }
            return resp

Step 3 Code Snippets

Receiving the Lex response with the agent request

Lines 11-13 interogate the session attributes returned by Lex and then set up the agent transfer, if necessary.
        function _lexReceive(err, data) {
         console.log('receiving lex message')
         if (err) {
    console.log(err, err.stack);
   }
         
   if (data) {
    console.log('message: ' + data.message);
    var sessionAttributes = data.sessionAttributes;
    _displayText('Bot:', data.message);
    if (data.sessionAttributes && 'Agent' in data.sessionAttributes){
     _mode = 'genesys';
     _genesysConnect(_getTranscript());
    }
   } 
        }

Genesys connection.

Genesys side configuration is necessary to set up the hook between the GMS API calls and the Genesys routing framework. 'Enable-notification-mode' must be set to True to allow Cometd connections to GMS.  A service/endpoint must be created that corresponds to an endpoint definition in the Genesys Chat Server configuration.  That chat end point is a pointer to a Genesys routing strategy.


If a Cometd connection doesn't already exist, create one and perform the handshake to determine connection type.  Websocket is the preferred method, but if that fails - Cometd will fall back to a polling-type async connection.  The request to connect to Genesys is then sent across that Cometd(websocket) connection via the publish command.


        
        var _genesysChannel = '/service/chatV2/v2Test';

        function _metaHandshake(message) {
         console.log('cometd handshake msg: ' + JSON.stringify(message, null, 4));         
         if (message.successful === true) {
          _genesysReqChat();
         }
        }

        function _genesysReqChat() {
         var reqChat = {
           'operation' : 'requestChat',
        'nickname' : _firstName + _lastName
      };
         _cometd.batch(function() { 
       _genesysSubscription = _cometd.subscribe(_genesysChannel, _genesysReceive); 
       _cometd.publish(_genesysChannel, reqChat);
      });
        }
        
        function _genesysConnect() {
         console.log('connecting to genesys');
         if (!_connected) { 
          _cometd.configure({
           url: 'https://' + location.host + '/genesys/cometd',
           logLevel: 'debug'
          });
          _cometd.addListener('/meta/handshake', _metaHandshake);
          _cometd.addListener('/meta/connect', _metaConnect);
          _cometd.addListener('/meta/disconnect', _metaDisconnect);
          _cometd.handshake();
         }
         else {
          _genesysReqChat();
         }
        }

Step 4 Code Snippets

In the previous step, the web client subscribed to a Cometd channel corresponding to a Genesys chat end point.  When the message arrives that this client is 'joined', publish the existing chat transcript (between the user and Lex) to that Cometd channel.
    function _getTranscript(){
     var chat = _id('chat');
     var text;
     if (chat.hasChildNodes()) {
      text = '***Transcript Start***' + '\n';
      var nodes = chat.childNodes;
      for (var i=0; i < nodes.length; i++){
       text += nodes[i].textContent + '\n';
      }
      text += '***Transcript End***';
     }
     return text;
    }
        function _genesysReceive(res) {
         console.log('receiving genesys message: ' + JSON.stringify(res, null, 4));
      if (res && res.data && res.data.messages) {
       res.data.messages.forEach(function(message) {
        if (message.index > _genesysIndex) {
         _genesysIndex = message.index;
         switch (message.type) {
          case 'ParticipantJoined':
           var nickname = _firstName + _lastName;
           if (!_genesysSecureKey && message.from.nickname === nickname){
            _genesysSecureKey = res.data.secureKey;
            console.log('genesys secure key reset to: ' + _genesysSecureKey);
            var transcript = _getTranscript();
            if (transcript){
             _genesysSend(transcript, true);
            }
           }
           break;

Step 5 Screen Shots


Agent Desktop (Genesys Workspace)



Source: https://github.com/joeywhelan/lexgenesys

Copyright ©1993-2024 Joey E Whelan, All rights reserved.

Tuesday, February 20, 2018

Node.js Reverse Proxy


Summary

In this post I'll show how to create a simple reverse proxy server in Node.js.

Environmentals

The scenario here is front-ending an app server (in this case Genesys Mobility Services (GMS) with a proxy to only forward application-specific REST API requests to GMS over HTTPS.  The proxy also acts as a general web server as well - also over HTTPS.

Code

var path = require('path');
var fs = require('fs'); 
var gms = 'https://svr2:3443';

var express = require('express');
var app = express();
var privateKey = fs.readFileSync('./key.pem'); 
var certificate = fs.readFileSync('./cert.pem'); 
var credentials = {key: privateKey, cert: certificate};
var https = require('https');
var httpsServer = https.createServer(credentials, app);

var httpProxy = require('http-proxy');
var proxy = httpProxy.createProxyServer({
 secure : false,
 target : gms
});

httpsServer.on('upgrade', function (req, socket, head) {
   proxy.ws(req, socket, head);
});

proxy.on('error', function (err, req, res) {
 console.log(err);
 try {
  res.writeHead(500, {
   'Content-Type': 'text/plain'
  });
  res.end('Error: ' + err.message);
 } catch(err) {
  console.log(err);
 }
});

app.use(express.static(path.join(__dirname, 'public')));

app.all("/genesys/*", function(req, res) {
 proxy.web(req, res);
});

httpsServer.listen(8443);
Lines 1-11:  Set up a HTTPS server with Express.  The proxy target is specified in Line 3.
Lines 13-17:  Set up the Proxy.  I'm using a self-signed certificate on Svr 2, so 'secure' is set to false to support that.
Lines 19-21:  Configure the HTTPS server use the Proxy to proxy websockets.
Line 35:  Serve up static content (HTML, CSS, Javascript) from the 'public' directory for general requests to this server.
Lines 37-39:  Proxy any requests that are specifically to the GMS REST API, both HTTPS and WSS traffic.

Source:  https://github.com/joeywhelan/Revproxy/

Copyright ©1993-2024 Joey E Whelan, All rights reserved.

Sunday, February 18, 2018

AWS Lex Chatbot - Programmatic Provisioning


Summary

AWS Lex has a full SDK for model building and run time execution.  In this post, I'll demonstrate use of that SDK in Python.  I'll demonstrate how create a simple/toy chatbot, integrate with a Lambda validation function, do a real time test, then finally - delete the bot.  Use of the AWS console will not be necessary at all.  All provisioning will be done in code.

AWS Lex Architecture

Below is a diagram of the overall architecture.  The AWS Python SDK is used here for provisioning.  Lambda is used for real time validation of data.  In this particular bot, I'm using a 3rd party (SmartyStreets) for validating street addresses.  That consists of a web service call from Lambda itself.



Bot-specific Architecture

Below is a diagram of how Lex bots are constructed.  Lex bot is goal-oriented sort of chatbot.  Goals are called 'Intents'.  Items necessary to fulfill an Intent are called Slots.  A bot consists of a bot definition in which Intent definitions are referenced.  Intents can include references to custom Slot Type definitions.  Intents are also where Lambda function calls for validation of slot input and overall fulfillment are made.

Bot Provisioning Architecture

Below is an architectural diagram of my particular provisioning application.  The Python application itself is composed of generic AWS SDK function calls.  All of the bot-specific provisioning configuration exists in JSON files.


Lex Provisioning Code

My Lex bot provisioning code consists of a single Python class.  That class gets its configuration info from an external config file.
if __name__ == '__main__':
    bot = AWSBot('awsbot.cfg')
    bot.build()
    bot.test('I want to order 2 cords of split firewood to be delivered at 1 pm on tomorrow to 900 Tamarac Pkwy 80863')
    bot.destroy()
Lines 1-5:  The AWSBot class exposes a simple interface to build, test, and destroy a bot on Lex.

As mentioned, all configuration is driven by a single config file and multiple JSON files.
class AWSBot(object):  
    def __init__(self, config):
        logger.debug('Entering')
        self.bot, self.slots, self.intents, self._lambda, self.permission = self.__loadResources(config)
        self.buildClient = boto3.client('lex-models')
        self.testClient = boto3.client('lex-runtime')
        self.lambdaClient = boto3.client('lambda')
        logger.debug('Exiting')  

    def __loadResources(self, config):
        logger.debug('Entering')
        cfgParser = configparser.ConfigParser()
        cfgParser.optionxform = str
        cfgParser.read(config)
        
        filename = cfgParser.get('AWSBot', 'botJsonFile')
        with open(filename, 'r') as file:
            bot = json.load(file)
        
        slotsDir = cfgParser.get('AWSBot', 'slotsDir')
        slots = []
        for root,_,filenames in os.walk(slotsDir):
            for filename in filenames:
                with open(os.path.join(root,filename), 'r') as file:
                    jobj = json.load(file)
                    slots.append(jobj)
                    logger.debug(json.dumps(jobj, indent=4, sort_keys=True))
                     
        intentsDir = cfgParser.get('AWSBot', 'intentsDir')
        intents = []
        for root,_,filenames in os.walk(intentsDir):
            for filename in filenames:
                with open(os.path.join(root,filename), 'r') as file:
                    jobj = json.load(file)
                    intents.append(jobj)
                    logger.debug(json.dumps(jobj, indent=4, sort_keys=True))
        
        filename = cfgParser.get('AWSBot', 'lambdaJsonFile')
        dirname = os.path.dirname(filename)
        with open(filename, 'r') as file:
            _lambda = json.load(file)
        with open(os.path.join(dirname,_lambda['Code']['ZipFile']), 'rb') as zipFile:
            zipBytes = zipFile.read()
        _lambda['Code']['ZipFile'] = zipBytes    
        
        filename = cfgParser.get('AWSBot', 'permissionJsonFile')
        with open(filename, 'r') as file:
            permission = json.load(file)
               
        return bot, slots, intents, _lambda, permission
Lines 1-8:  Load up dict objects with the Lex config and instantiate the AWS SDK objects.
Lines 10-14:  Read a config file that holds directory paths to the JSON files used to provision Lex.
Lines 16-18:  Load a dict object with the a JSON Lex Bot definition file.
Lines 20-27:  Load a dict object with a custom slot type JSON definition.
Lines 29-35:  Load a dict object with the Intent JSON definition.
Lines 38-44:  Load a dict object with Lambda code hook definition.  Read the bytes of a zip file containing the Python code hook along with all non-AWS-standard libraries it references.
Lines 46-48:  Load a dict object with the attributes necessary to add permission for the Lambda code hook to be called from Lex.

The public build interface consists calls to private methods to build the various Lex-related objects: Lambda code hook, slot types, intents, and finally the bot itself.
    def build(self):
        logger.debug('Entering')  
        self.__buildLambda()
        self.__buildSlotTypes()
        self.__buildIntents()
        self.__buildBot()
        logger.debug('Exiting')
Below is the code for the private build methods:
    def __buildLambda(self):
        logger.debug('Entering')
        resp = self.lambdaClient.create_function(**self._lambda)
        logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
        resp = self.lambdaClient.add_permission(**self.permission)
        logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
        logger.debug('Exiting')
    
    def __buildSlotTypes(self):
        logger.debug('Entering')
        for slot in self.slots:
            resp = self.buildClient.put_slot_type(**slot)
            logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
        logger.debug('Exiting')
        
    def __buildIntents(self):
        logger.debug('Entering')
        for intent in self.intents:
            resp = self.buildClient.put_intent(**intent)
            logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
        logger.debug('Exiting')
        
    def __buildBot(self):
        logger.debug('Entering')
        self.buildClient.put_bot(**self.bot)
        complete = False
        for _ in range(20):
            time.sleep(20)
            resp = self.buildClient.get_bot(name=self.bot['name'], versionOrAlias='$LATEST')
            logger.debug(resp['status'])
            if resp['status'] == 'FAILED':
                logger.debug('***Bot Build Failed***')
                logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
                complete = True
                break
            elif resp['status']  == 'READY':
                logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
                complete = True
                break
                   
        if not complete:
            logger.debug('***Bot Build Timed Out***')
            logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer)) 
        logger.debug('Exiting')
Lines 1-7:  Call the AWS Lambda SDK client to create the function with the previously-loaded JSON definition.  Add the permission for the Intent to call that Lambda function.
Lines 9-14:  Loop through the slots JSON definitions and build each via AWS SDK call.
Lines 16-21:  Same thing, but with the Intents.
Lines 23-44:  Build the bot with its JSON definition.  Although this SDK call is synchronous and returns almost immediately, the Bot will not be completed upon return from the call.  It takes around 1-2 minutes.  The for loop here is checking AWS's progress on the Bot build every 20 sec.

After the Bot is complete, the Lex runtime SDK can be used to test it with a sample utterance.
def test(self, msg):
        logger.debug('Entering')
        params = {
                    'botAlias': '$LATEST',
                    'botName': self.bot['name'],
                    'inputText': msg,
                    'userId': 'fred',
                }
        resp = self.testClient.post_text(**params)
        logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer)) 
        logger.debug('Exiting')
Deleting/cleaning-up the Lex objects on AWS is a simple matter of making the corresponding delete function calls from the SDK.  Similar to the build process, deleting an object takes time on AWS, even though the function may return immediately.  You can mitigate the issues delays associated with deletion and corresponding dependencies by putting artificial delays in the code, such as below.
def __destroyBot(self):
        logger.debug('Entering')
        try:
            resp = self.buildClient.delete_bot(name=self.bot['name'])
            logger.debug(json.dumps(resp, indent=4, sort_keys=True, default=self.__dateSerializer))
        except Exception as err:
            logger.debug(err)
        time.sleep(5) #artificial delay to allow the operation to be completed on AWS
        logger.debug('Exiting')

Lambda Code Hook

If you require any logic for data validation or fulfillment (and you will for any real bot implementation), there is no choice but to use AWS Lambda for that function.  That Lambda function needs a single entry point where the Lex event (Dialog validation and/or Fulfillment) is passed.  Below is entry point for the function I developed.  All the validation logic is contained in single Python class - LexHandler.
def lambda_handler(event, context):
    handler = LexHandler(event)
    os.environ['TZ'] = 'America/Denver'
    time.tzset()
    return handler.respond()
I'm not going to post all the code for the handler class as it's pretty straight-forward (full source will be on github as well), but here's one snippet of the Address validation.  It actually makes a call to an external webservice (SmartyStreets) to perform the validation function.
    def __isValidDeliveryStreet(self, deliveryStreet, deliveryZip):  
        if deliveryStreet and deliveryZip:
            credentials = StaticCredentials(AUTH_ID, AUTH_TOKEN)
            client = ClientBuilder(credentials).build_us_street_api_client()
            lookup = Lookup()
            lookup.street = deliveryStreet
            lookup.zipcode = deliveryZip
            try:
                client.send_lookup(lookup)
            except exceptions.SmartyException:
                return False
            
            if lookup.result:
                return True
            else:
                return False
        else:
            return False

Full source here:  https://github.com/joeywhelan/AWSBot

Copyright ©1993-2024 Joey E Whelan, All rights reserved.