Monday, January 8, 2018

Cassandra Integration with Genesys Orchestration Server

Summary

Below are some the challenges and steps I took to overcome them in getting Genesys Orchestration Server (ORS) working with Cassandra.  ORS uses Cassandra for session persistence storage.

Environment

Below is a diagram of an 'all in one' server scenario.  Suitable for lab instances.  All components coexist within 1 virtual machine (VM) - in theory.

Challenge 1:  Genesys supported RHEL version vs required Python version

Genesys is currently supporting up to version 6 of RHEL/CentOS.  RHEL 6 has Python 2.6 as its native version.  Specifically, Python 2.6 is a hard requirement as yum is dependent on it.

Cassandra's command line interface, CQLSH, requires Python 2.7.  Simply installing Python 2.7 on top of 2.6 will result in a broken OS.

Solution:  Utilize RHEL Software Collections (SCL).  SCL allows multiple versions of software packages to coexist within one system.  Below are the commands to install SCL.
yum install scl-utils
yum install centos-release-scl-rh
yum install python27
Starting up a bash shell via SCL to enable Python 2.7 is accomplished with this command:
scl enable python27 bash
If you want to exit the SCL environment (and re-enable Python 2.6) 
exit

Challenge 2: Eliminating warnings/degraded mode message in Cassandra logs.

If you deploy Cassandra on a fresh install of RHEL/CentOS - you will see various warnings that are tied to insufficient resource limits for the user starting Cassandra.

Solution:
  • Create a file 'cassandra.conf' in the /etc/security/limits.d directory to increase the limits.  For the example below I have a user 'genesys' who will be starting Cassandra.  I increase the limits for that user.
genesys - memlock unlimited
genesys - nofile 100000
genesys - nproc 32768
genesys - as unlimited
  • Eliminate swap warning.  If you want to turn off swap:
    • Temporary Method:
      • swapoff -a
    • Permanent Method: Comment out the swap partition in /etc/fstab
  • Eliminate jemalloc warning.  
The only other mandatory config requirement is turning on RPC in the cassandra.yaml file. 
# Whether to start the thrift rpc server.
start_rpc: true

Challenge 3:  Buggy Cassandra version included in Genesys install package

Genesys is currently packaging Cassandra 2.2.5 on their latest install media for Routing.  That release has a documented bug with CQLSH.  If you attempt to use CQLSH in this version, you'll get an error that looks like this:
Connection error: ('Unable to connect to any servers', {'127.0.0.1': TypeError('ref() does not take keyword arguments',)})
Solution:  Don't use the Cassandra package that is included in the Genesys media.  Download a current tar ball from the Cassandra site.

Challenge 4:  Cassandra user authentication issues with ORS

So, I don't have a documented/confirmed bug on this one - but I'm fairly confident there is one.  I've been unable to get authentication to work properly with ORS 8.1.4 after trying pretty much everything.  

If you define a user within Cassandra AND set the authenticator variable within cassandra.yaml - user authentication should work.  Below is the configuration item necessary in cassandra.yaml. 
authenticator: PasswordAuthenticator
In fact, I tested this with CQLSH and also with a Python client (pycassa).  The Python client uses the same RPC link to Cassandra that ORS uses.  CQLSH and the Python client work just fine with authentication.  

Setting user name + password is done with the persistence options under ORS in Genesys Administrator.  Screen shot below.

If you set those options (and configure Cassandra as discussed above), this is what I see in the ORS logs.  ORS can't connect to Cassandra and simply terminates.
21:09:16.499 Std 23001 ORS Cassandra schema version ORS8130000 Schema validation failed . Orchestration is terminating.
Solution (not a good one):  Turn off authentication in Cassandra and blank out the username/password in the ORS persistence options.  Cassandra.yaml config to allow all/no authentication on RPC below:
authenticator: AllowAllAuthenticator
ORS will connect to Cassandra with no username/password and then create its keyspace 'Orchestration'.  Below is what you should see in Cassandra after a successful ORS integration with Cassandra:
$ python -V
Python 2.7.13
$ /home/cassandra/apache-cassandra-3.11.1/bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.1 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cqlsh> describe keyspaces;

system_schema  system           system_distributed
system_auth    "Orchestration"  system_traces     


Copyright ©1993-2024 Joey E Whelan, All rights reserved.
Posted by at