Thursday, January 25, 2018

Genesys Chat 8.5 Installation Notes


Summary

This post will cover some of the highlights of a recent install of latest/greatest Genesys Chat architecture.  I'm not attempting to recreate the Genesys installation documentation (there's ample amount of that already), just areas that I noted were troublesome and/or not documented as clearly as I would like.

Architecture

Below is a diagram of this particular lab environment.  For clarity, the diagram does not depict all the actual processes and inter-connections.  This is a lab/all-in-one-box type deployment.  Genesys Mobility Services (GMS) is used for AP interface to Chat Server whereas was used WebAPI in the previous 8.1 architecture.


Licensing

Below is an excerpt of a Genesys license file with the line items necessary for chat highlighted.  Ten seats of chat are enabled.
FEATURE 3GP07263ACAA genesys.d 7.1 1-oct-2018 10 5379D90C1653 \
 vendor_info="v7.1 - Genesys Agent Desktop" NOTICE="Lab" \
 SIGN=AAE090E4AF30
FEATURE 3GP08393ACAA genesys.d 8.0 1-oct-2018 10 3EB730F543A4 \
 vendor_info="v8.0 - SIP Server" NOTICE="Lab" SIGN=FAC58E606844
FEATURE 3GP08693ACAA genesys.d 8.0 1-oct-2018 1 02D33156B657 \
 vendor_info="v8.0 - Genesys Chat - Lab" NOTICE="Lab" \
 SIGN=71238A487514
FEATURE ics_multi_media_agent_seat genesys.d 8.0 1-oct-2018 10 \
 63B03A7C3207 NOTICE="Lab" SIGN=1366ECF80B66
FEATURE ics_live_web_channel genesys.d 8.0 1-oct-2018 10 B606C4B7F496 \
 NOTICE="Lab" \
 SIGN=25F907A4A1DE
FEATURE DESKTOP_SUPERVISOR genesys.d 7.0 1-oct-2018 1 BE6E85DE569E \
 NOTICE="Lab" \
 SIGN=B7DBB0F88EEC
Additionally, the options below need to be set in Interaction Server to check out licenses from FlexLM daemon:


Interaction Server

  1. This particular Genesys server needs a Genesys DB Server + Data Access Point (DAP) to integrate with a database (unlike Config layer which can utilize the native client - Oracle dbclient via a DAP alone).
  2. There are a dozen or so SQL scripts included in the install directory.  The two that you need for a fresh install are (assuming Oracle DB):  isdb_oracle.sql, eldb_oracle.sql.
    $ pwd
    /home/genesys/ixnsvr/Script/Oracle
    $ ls
    eldb_oracle_7.6.1-8.0.1.sql  eldb_oracle.sql          isdb_oracle_7.2-7.5.sql      isdb_oracle_7.6-7.6.1.sql  isdb_oracle.sql
    eldb_oracle_drop.sql         isdb_oracle_7.0-7.1.sql  isdb_oracle_7.5-7.6.sql      isdb_oracle_drop.sql
    eldb_oracle_nvc.sql          isdb_oracle_7.1-7.2.sql  isdb_oracle_7.6.1-8.0.1.sql  isdb_oracle_nvc.sql
    
  3. Interop with ORS. Below is an excerpt from the current ORS Deployment guide regarding interop with eServices. 

    Starting with ORS 8.1.400.27, you create the Interaction Server Application(s) using only the Interaction Server Application template. There is no need to create an Interaction Server Application using a T- Server Application Template for the second Application object. For backward compatibility, both methods of deployment are supported.

    Based on my experience, that's simply not true.  Not configuring a multimedia switch and corresponding TServer results in the following errors in the ORS log and no routing from ORS:
    10:23:27.752 Std 20010 Configuration error. Class [ConfigDirectory] : Switch is not assinged to the tenant of Interaction Server 'ixnsvr'
    10:23:27.752 Std 23009 ORS WARNING Connection to Interaction Server configured as T-Server required, eServices functionality not enabled.
    
    Those messages are pretty clear to me:  ORS still demands the legacy configuration.  Solution: create a switching office of type Multimedia, a switch, and associate a Interaction Server with it by using a TServer application template.  Screen shots below of what that looks like:
  4. There's an undocumented health monitoring interface (HTTP/SOAP).  If you set up a HTTP 'health' port and options, you can access it via browser as depicted below:

GMS

Ensure the connection to Chat Server is on its webapi port (http).  GMS will attempt to install its own Cassandra instance but you can specify instead it use an existing/external instance.  

ORS

The deployment guide pretty well covers the configurations necessary to get ORS to function with eServices.  Couple items of note:
  1. Turn on mcr-pull-by-this node
  2. Turn up full debug on logs if you're troubleshooting.  Setting 'debug' as the log level alone won't get you log messaging down to SCXML processing level.  You need set the x-server-trace-level option as well.


Workspace

By default, Workspace will try to login an agent to all media types.  If you don't have licensing to support that, you'll get annoying errors on start-up of Workspace.  To eliminate those, turn on role-based security, create a role with privileges corresponding to your licensing (in this case, voice and chat only) and assign the role to the agent(s).  Role-based security is disabled by default.  You do the double-negative to turn it on (option is 'disable', set it to false).

Capacity Rule

By default, agents have no capacity for any eService-type interactions (chat, email, etc).  If you don't configure a Capacity Rule with chat, for instance, and assign it to an agent - no routing of a chat will occur.  

Creating a rule requires deployment of GAX.  Below are some screenshots of a simple rule to allows for 1 voice and 3 chat interactions simultaneously.

Routing - GMS Chat API Version 1

There are two Chat API's within GMS.  Below are the steps to get a V1 Chat interaction routed to an agent.

Develop the Composer Routing Script

Screen shots below of the dev cycle for a extremely simplistic chat routing script.  It simply sends an inbound request to an Agent Group.

File, New, Other

Name the project, Select Route project type, Next, Finish.


Connect to Configuration Server.

Open up the default.workflow and add a single Route Interaction block.  For Targets, choose an Agent Group you've previously created in Administrator.


Open the default.ixnprocess view and add an Interaction Queue object.  Go to the properties of that object and add a View.  Connect the Interaction Queue to the Workflow object.


Left click on the project in Project Explorer, choose Generate All.  Select Deploy Project and Publish data to Configuration Server.  This step will build/validate the code, deploy the resulting WAR file on the Tomcat instance included with Composer, and finally build all the necessary Script objects in Configuration Management.



After completing this step, the project will be on Tomcat and objects below are constructed in Configuration.


Those four objects have linkages to each other.  The 'defaultWorkflow' object has the URI to the actual Composer-generated SCXML on Tomcat.


We use the InteractionQueue object in an Chat endpoint.  Create an Endpoint in Options and add the reference to the Queue as its value.



Using the web GUI to GMS, provision a 'request-chat' service and add the Chat endpoint you just defined to it.



In theory, all the provisioning is complete now.  GMS provides a sample Chat V1 API client on the main page with the 'Sample' link.


Select the 'Request-Chat' scenario and click the 'Connect' button.

The GMS sample client will then initiate a chat session against the endpoint defined in GMS + Chat Server.  Interaction Server will trigger ORS to fetch the Composer-generated SCXML file on Tomcat.  The SCXML strategy will route the chat to the agent in the defined Agent Group.


Monday, January 8, 2018

Cassandra Integration with Genesys Orchestration Server

Summary

Below are some the challenges and steps I took to overcome them in getting Genesys Orchestration Server (ORS) working with Cassandra.  ORS uses Cassandra for session persistence storage.

Environment

Below is a diagram of an 'all in one' server scenario.  Suitable for lab instances.  All components coexist within 1 virtual machine (VM) - in theory.

Challenge 1:  Genesys supported RHEL version vs required Python version

Genesys is currently supporting up to version 6 of RHEL/CentOS.  RHEL 6 has Python 2.6 as its native version.  Specifically, Python 2.6 is a hard requirement as yum is dependent on it.

Cassandra's command line interface, CQLSH, requires Python 2.7.  Simply installing Python 2.7 on top of 2.6 will result in a broken OS.

Solution:  Utilize RHEL Software Collections (SCL).  SCL allows multiple versions of software packages to coexist within one system.  Below are the commands to install SCL.
yum install scl-utils
yum install centos-release-scl-rh
yum install python27
Starting up a bash shell via SCL to enable Python 2.7 is accomplished with this command:
scl enable python27 bash
If you want to exit the SCL environment (and re-enable Python 2.6)
exit

Challenge 2: Eliminating warnings/degraded mode message in Cassandra logs.

If you deploy Cassandra on a fresh install of RHEL/CentOS - you will see various warnings that are tied to insufficient resource limits for the user starting Cassandra.

Solution:
  • Create a file 'cassandra.conf' in the /etc/security/limits.d directory to increase the limits.  For the example below I have a user 'genesys' who will be starting Cassandra.  I increase the limits for that user.
genesys - memlock unlimited
genesys - nofile 100000
genesys - nproc 32768
genesys - as unlimited
  • Eliminate swap warning.  If you want to turn off swap:
    • Temporary Method:
      • swapoff -a
    • Permanent Method: Comment out the swap partition in /etc/fstab
  • Eliminate jemalloc warning.  
The only other mandatory config requirement is turning on RPC in the cassandra.yaml file.
# Whether to start the thrift rpc server.
start_rpc: true

Challenge 3:  Buggy Cassandra version included in Genesys install package

Genesys is currently packaging Cassandra 2.2.5 on their latest install media for Routing.  That release has a documented bug with CQLSH.  If you attempt to use CQLSH in this version, you'll get an error that looks like this:
Connection error: ('Unable to connect to any servers', {'127.0.0.1': TypeError('ref() does not take keyword arguments',)})
Solution:  Don't use the Cassandra package that is included in the Genesys media.  Download a current tar ball from the Cassandra site.

Challenge 4:  Cassandra user authentication issues with ORS

So, I don't have a documented/confirmed bug on this one - but I'm fairly confident there is one.  I've been unable to get authentication to work properly with ORS 8.1.4 after trying pretty much everything.  

If you define a user within Cassandra AND set the authenticator variable within cassandra.yaml - user authentication should work.  Below is the configuration item necessary in cassandra.yaml.
authenticator: PasswordAuthenticator
In fact, I tested this with CQLSH and also with a Python client (pycassa).  The Python client uses the same RPC link to Cassandra that ORS uses.  CQLSH and the Python client work just fine with authentication.  

Setting user name + password is done with the persistence options under ORS in Genesys Administrator.  Screen shot below.

If you set those options (and configure Cassandra as discussed above), this is what I see in the ORS logs.  ORS can't connect to Cassandra and simply terminates.
21:09:16.499 Std 23001 ORS Cassandra schema version ORS8130000 Schema validation failed . Orchestration is terminating.
Solution (not a good one):  Turn off authentication in Cassandra and blank out the username/password in the ORS persistence options.  Cassandra.yaml config to allow all/no authentication on RPC below:
authenticator: AllowAllAuthenticator
ORS will connect to Cassandra with no username/password and then create its keyspace 'Orchestration'.  Below is what you should see in Cassandra after a successful ORS integration with Cassandra:
$ python -V
Python 2.7.13
$ /home/cassandra/apache-cassandra-3.11.1/bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.1 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cqlsh> describe keyspaces;

system_schema  system           system_distributed
system_auth    "Orchestration"  system_traces     

Thursday, December 21, 2017

Windows Server 2016 Remote Desktop from Ubuntu desktop

Summary

In this post, I'll discuss how to set up a remote desktop session from an Ubuntu client to Windows 2016 Server.

Implementation


Windows-side Configuration


Open up Server Manager, Local Server. 


Click on Remote Desktop.  It will be 'Disabled' on Windows 2016 Standard.  With 2016 Essentials, it's enabled by default.


Choose the 'Allow remote connections to this computer' radio button.  Leave the 'Allow connections only...Network Level Authentication' checkbox selected.  More on that later.  The Administrator will be allowed Remote Desktop access by default.  If you want to add other users, go through the 'Select Users' dialog.

Close this window via the 'OK' button.  Remote Desktop will now show 'Enabled' (may require a screen refresh).  Additionally, Windows will automatically open up the necessary ports through its firewall to enable access to Remote Desktop.  

Ubuntu-side Configuration


rdesktop
rdesktop is a commonly-used RDP client on Ubuntu.  It doesn't appear that development of rdesktop has kept up with the times.  In particular, rdesktop does not support the current authorization protocols within Windows - specifically, Network Level Authentication.  That 'Allow...Network Level Authentication' checkbox mentioned above enables NLA.

Below is an example rdesktop command to initiate a Remote Desktop session:
rdesktop -g 1152x864 -r clipboard:CLIPBOARD -r sound:off -x l -P 1.2.3.4 -u "yourusername" -p yourpassword
If NLA is turned up on the Windows server, you'll get this error:
ERROR: CredSSP: Initialize failed, do you have correct kerberos tgt initialized ?
Failed to connect, CredSSP required by server.
If you're determined to use rdesktop, the simplest fix is to uncheck NLA box on the Windows server.  It will work after that, albeit with less security for the RDP connections.

freerdp
An alternative to rdesktop is FreeRDP.  It does support NLA.  You can either install it directly from the freerdp github site or simply install the version that's in the Ubuntu repositories - xfreerdp
sudo apt-get install freerdp-x11
Below is a sample command line that will set up a Remote Desktop session to Windows 2016, with NLA enabled:
xfreerdp /v:1.2.3.4 /u:yourusername /p:yourpassword +clipboard /size:1152x864
Creating a command launcher on the Ubuntu desktop for remote desktop can be accomplished with the command below:
gnome-desktop-item-edit --create-new ~/Desktop
This will launch a dialog window:

Copy/paste the xfreerdp command into the 'Command:' text box.

Tuesday, December 19, 2017

Installation of Genesys 8.5 Configuration DB on Oracle

Summary


This post discusses deployment of the Genesys Config DB on to an existing Oracle 11g instance.

Implementation


  • The Config Server installation includes directory of SQL scripts for various RDMS flavors.  The included Oracle scripts below:
  • ls sql_scripts/oracle
    CfgLocale_ora.sql  drop_tables_ora.sql  init_multi_multilang_ora.sql  init_multi_ora.sql
    
  • Installation of these scripts requires use of the Oracle command line tool, sqlplus.  That tool is located in the /u01/app/oracle/product/11.2.0/xe/bin directory.  Adding this directory to your PATH can be accomplished via execution of the included environment script in the same directory. This can also be added to your .bashrc script to automatically get the Oracle environment set up on login
  • . /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh
    
  • The individual SQL scripts can now be executed from the sqlplus command line.  Example below. Assumes the command is being executed from the scripts directory.
  • sqlplus
    
    SQL*Plus: Release 11.2.0.2.0 Production on Tue Dec 19 11:39:15 2017
    
    Copyright (c) 1982, 2011, Oracle.  All rights reserved.
    
    Enter user-name: yourusername
    Enter password: yourpassword
    
    Connected to:
    Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
    
    SQL> @drop_tables_ora.sql
    
  • An alternative to executing each of the 3 scripts separately is the one-liner below:
  • (echo @init_multi_ora.sql; echo @CfgLocale_ora.sql) | sqlplus yourusername/yourpassword @drop_tables_ora.sql
    

Genesys 8.5 on CentOS/RHEL 6: Oracle DB Client Error

Symptom

Upon attempting to start Genesys Configuration Server, the following error appears:
08:24:00.714 Std 05022 Process './dbclient_oracle_64' started
./dbclient_oracle_64: error while loading shared libraries: libclntsh.so.11.1: cannot open shared object file: No such

Solution

Make the Oracle 11g XE library directory accessible during run time for dynamic linking.
  1. Create a file in /etc/ld.so.conf.d.  In this example, I called it oraclexe.conf
  2. Add the following line to that conf file (path to the Oracle libraries):
  3. /u01/app/oracle/product/11.2.0/xe/lib
    
  4. Execute this command as root to update run-time bindings:
  5. ldconfig -v
    

Snippet of output

/u01/app/oracle/product/11.2.0/xe/lib:
 libdbcfg11.so -> libdbcfg11.so
 libsqora.so.11.1 -> libsqora.so.11.1
 libons.so -> libons.so
 libskgxns.so -> libskgxns.so
 libxdb.so -> libxdb.so
 libqsmashr.so -> libqsmashr.so
 libocrb11.so -> libocrb11.so
 libuini11.so -> libuini11.so
 libntcpaio11.so -> libntcpaio11.so
 libclntsh.so.11.1 -> libclntsh.so.11.1
 libldapjclnt11.so -> libldapjclnt11.so
 libocci.so.11.1 -> libocci.so.11.1
 libnjni11.so -> libnjni11.so
 libodm11.so -> libodm11.so
 libskgxp11.so -> libskgxp11.so
 libnnz11.so -> libnnz11.so
 libheteroxa11.so -> libheteroxa11.so
 libclsra11.so -> libclsra11.so
 libnque11.so -> libnque11.so
 libocijdbc11.so -> libocijdbc11.so
 libhasgen11.so -> libhasgen11.so
 libcorejava.so -> libcorejava.so
 libocrutl11.so -> libocrutl11.so
 libemmas11.so -> libemmas11.so
 libodmd11.so -> libodmd11.so
 libskgxn2.so -> libskgxn2.so
 libcell11.so -> libcell11.so
 libocr11.so -> libocr11.so
 libsqlplus.so -> libsqlplus.so
 libhgotrace11.so -> libhgotrace11.so
 libagtsh.so -> libagtsh.so.1.0

Commentary

A common paradigm in software development is to split out commonly used components into libraries.  In the UNIX/Linux world, those libraries are referred to as shared objects (the 'so' in the names above).  One can determine the library requirements of a given binary with the 'readelf' command.  Example below with the Genesys db client for Oracle:
readelf -d dbclient_oracle_64

Output

Dynamic section at offset 0xc2890 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libclntsh.so.11.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000f (RPATH)              Library rpath: [/release/lib/oracle/i686-linux-rhe5/11.2.0/lib64]
 0x000000000000000c (INIT)               0x429558
 0x000000000000000d (FINI)               0x4998f8
 0x000000006ffffef5 (GNU_HASH)           0x400240
 0x0000000000000005 (STRTAB)             0x4140c8
 0x0000000000000006 (SYMTAB)             0x405008
 0x000000000000000a (STRSZ)              73271 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x6c2ee8
 0x0000000000000002 (PLTRELSZ)           3672 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x428700
 0x0000000000000007 (RELA)               0x427410
 0x0000000000000008 (RELASZ)             4848 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x427310
 0x000000006fffffff (VERNEEDNUM)         6
 0x000000006ffffff0 (VERSYM)             0x425f00
 0x0000000000000000 (NULL)               0x0

Monday, December 18, 2017

Genesys 8.5 Linux Installation Error: bad ELF interpreter

Symptom

  • Attempting to execute a Genesys installation shell script (Linux) and the following error occurs:
/lib/ld-linux.so.2: bad ELF interpreter: No such file or directory

Solution (RHEL/CentOS):


yum -y install glibc.i686

Oracle 11g XE Installation on CentOS: Database Configuration failed.

Symptoms

  • Successfully complete the rpm install of 11g XE:
rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm

  • Attempt the next step (configure):
/etc/init.d/oracle-xe configure

  • Receive this error:
Database Configuration failed.  Look into /u01/app/oracle/product/11.2.0/xe/config/log for details

  • In the postDBCreation.log, you see this:
ORA-00119: invalid specification for system parameter LOCAL_LISTENER
ORA-00130: invalid listener address '(ADDRESS=(PROTOCOL=TCP)(HOST=yourhostname)(PORT=1521))'

Solution


Ensure your hostname is defined to an IP address.  In particular, add a line item for it in /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
1.2.3.4   yourhostname