Viya 3 on Linux

Having successfully installed the SAS Enterprise Monitor Server component and Agent component, SAS configuration changes are now required to enable SAS Enterprise Monitor to monitor the various components of your SAS Viya 3 installation.

Microservices

1. Append the following code block the end of /opt/sas/viya/home/lib/envesntl/sas-init-functions updating the values of ESMHOME variable to the location of your SAS Enterprise Monitor agent, and the viyalogroot (if required):

   # ESM CONFIG BLOCK START
   ESMHOME=/opt/esm/esm-agent
   source $ESMHOME/conf/esmconfig.sh
   viyalogroot=/opt/sas/viya/config/var/log
   viyaname=sas-viya
   viyapidfiles=/var/run/sas
   
   # format here is reportPid [pid] [pidfile]
   function reportViyaPid () {
       esmpidno=$1
       esmfileshort=$(basename $2)
       esmservicename=$(echo ${esmfileshort} | cut -d '.' -f 1)
   
   esmfds=$(ls -latr /proc/${esmpidno}/fd | grep ${viyalogroot} | sed 's/.*> //' | sort | uniq)
       for fd in $esmfds; do
       # if there happen to be multiple open get the latest
           lastLogfile=${fd}
   done
       esmpidowner=$(ps -o user= -p $esmpidno)
       eventString=[{\"pid\":$esmpidno\,\"owner\":\"$esmpidowner\"\,\"esmType\":\"viya\"\,\"jobName\":\"$esmservicename\"\,\"logFile\":\"$lastLogfile\"\,\"hostname\":\"$ESMNODENAME\"}]
       echo $eventString > $ESMEVENTS/new'_'$esmpidno'_'$ESMNODENAME
   }
   # ESM CONFIG BLOCK END
   

2. Add the following code after the result=0 # is running line (roughtly, this will be on line 117):

   # ESM CONFIG BLOCK START
   reportViyaPid ${pid} ${pidfile}
   # ESM CONFIG BLOCK END
   

3. If your SAS Microservices are running while these changes are made you can run the following commands to add running Microservices to SAS Enterprise Session Monitor. The below script can take a few seconds to run:

   cd /var/run/sas
   source /opt/sas/viya/home/lib/envesntl/sas-init-functions
   for i in *.pid; do p=`cat $i`; reportViyaPid $p $i 2>/dev/null; done
   

SPRE

1. Copy and paste the below code block into the sasenv_local file of your SAS installation (!SASROOT/bin/sasenv_local) updating the value of the ESMHOME variable to the location of your SAS Enterprise Monitor agent.

   # ESM CONFIG START
   export ESMACTIVE=1
   export ESMHOME=/path/to/esm-agent
   if [ -f $ESMHOME/bin/esm-bootstrap.sh ]; then
       . $ESMHOME/bin/esm-bootstrap.sh
   fi
   # ESM CONFIG END
   

CAS Server

Download the latest version of the sas-cas-mon module. The credentials for the downloads site can be found in the email with the SAS Enterprise Session Monitor license key.

The following steps should be completed for each CAS Server in your environment which has an Agent deployed to it.

1. Unzip the downloaded sas-cas-mon.zip file into the modules folder of your Agent installation.
2. Update the vars.yml file changing the following items¹:
   1. logDirectories - The path(s) in which the module should look for CAS logs.
   2. offsetFilesLocation - The full path to the folder that is use to store the offset files for read log files. You should just need to update the path to your ESM agent folder.
3. Create the sas-cas-mon service:
   1. Update line 6 of the sas-cas-mon.service file updating the values of ESMHOME, ESMEVENTS, ESMNODENAME for your environment including replacing <HOSTNAME> with the name configured during the installation of the agent.
   2. Update line 9 for the path to your modules folder
   3. As root or using sudo, copy the updated sas-cas-mon.service file to /etc/systemd/system
   4. As root or using sudo, run systemctl daemon-reload
4. Start the Module service by executing (as root): systemctl start sas-cas-mon

The sas-cas-mon uses CAS Action SAS logging to collect data. CAS logging will need to be enabled in order for the module to work correctly.

For complete information about the module, please review the reference guide.

Compute Server

Multi-tenancy

If you are running a multi-tenant environment, the below steps should be replicated across each tenant in your deployment.

1. Add the following lines to /opt/sas/viya/config/etc/compsrv/<tenant>/autoexec_usermods.sas modifying the following:
   1. <tenant> in the path above to the appropriate folder
   2. ESMBASEDIR to the location of your esm-agent folder
   3. ESMNODENAME to the name of the server as specified in $ESMBASEDIR/conf/esmconfig.sh
   4. ESMVIYATENANT to the name of the tenant if configuring a multi-tenant environment
   5. ESMLOGPATH to the directory where logs for the compute server are stored. You must keep the trailing slash on this path

      /* START ESM BOOTSTRAP */
      %let ESMBASEDIR="/opt/sas/esm/esm-agent";
      %let ESMNODENAME="&SYSHOSTNAME.";
      %let ESMVIYATENANT=default;
      %let ESMLOGPATH="/opt/sas/viya/config/var/log/compsrv/&ESMVIYATENANT./";
      %if %sysfunc(fileexist(&ESMBASEDIR./sasautos/esmtags.sas)) %then %do;
          options set = ESMHOME &ESMBASEDIR.;
          options set = ESMACTIVE = 1;
          options set = ESMSESSIONTYPE="CMP";
          options set = ESMSESSIONNAME="compsrv_default";
          options set = ESMNODENAME=&ESMNODENAME.;
          options set = LSB_QUEUE=&ESMVIYATENANT;
          options set = LOGPATH=&ESMLOGPATH.;
          %let esmh=%sysget(ESMHOME);
          %include "&esmh./sasautos/esmtags.sas";
      %end;
      /* END ESM BOOTSTRAP */
      

Connect Server

Multi-tenancy

If you are running a multi-tenant environment, the below steps should be replicated across each tenant in your deployment.

1. Modify line 152 of connectserver.sh removing autoexec $(dirname ${scriptpath})/autoexec.sas. After the modification, the line should look like the below

   cmd="$sas_command -logconfigloc ${CONNECTSERVER_LOG_CFG} ${CONNECTSERVER_EXTRA_ARGS} $USERMODS_OPTIONS "
   

Additional Configuration

Collect Job Flow and Exit Status Information

In order to be able to report on the FLOW and job completion status of programs submitted using the Batch Server, some changes must be made to the sasbatch.sh script. If you use the Batch server to submit workload, you should implement this step.

1. In sasbatch.sh comment out line 122 by placing a # at the start of the line. After the change, the line should read:

   #options="-autoexec ${config_home}/autoexec.sas"
   

2. In sasbatch.sh replace the line at the bottom of the file that reads eval exec $cmd with the following, updating line 2 of the block below with the correct path to your esm-agent folder:

   # ESM CONFIG START
   export ESMHOME=/path/to/esm/esm-agent
   source $ESMHOME/conf/esmconfig.sh
   if [[ -z $ESMJOBNAME ]]; then
     ESMFULLPATHFILE=`/bin/bash $ESMHOME/getSysIn.sh "$@"`
     ESMJOBNAME=`basename $ESMFULLPATHFILE | sed 's/"//g' | sed "s/'//g"`
     if [[ -z $ESMJOBNAME ]]; then
       ESMJOBNAME=interactive
     fi
     export ESMJOBNAME
   fi
   $cmd
   rc=$?
   $ESMHOME/reportOut.sh $ESMGUID $ESMJOBNAME $rc "$LSB_JOBNAME"
   # ESM CONFIG END
   

Track Session UTILLOC size

Unlike the SAS WORK location which is a directory created at session startup that is unique to each individual SAS session, SAS UTIL directories are created on-the-fly by SAS procedures as they are required. The Agent can be configured to automatically watch one or more directories for these on-the-fly created folders to enable visibility of per-session UTILLOC consumption.

1. In your Agent's bin directory, create a new file called tempDirectories.yaml
2. Copy the below contents into the newly created file, updating the path to the SAS WORK directory:

   tempDirectories:
     warningDuration: 2000  
     scanInterval: 2000 
     directories: 
       - /tmp
     settings:
       - directoryPattern: .*SAS_util\w{4}(?<dirPid>\w{8})_(?<hostname>[^_]+)_
         directoryPidIsHex: true
         directoryType: util
   

3. Restart your Agent

For more information about the tempDirectories.yaml file and options for configuring pre-process UTILLOC and WORK tracking, see the reference guide.

Capture Procedures and Datasets

Install the Module

Download the latest version of the sas-proc-mon module. The credentials for the downloads site can be found in the email with the SAS Enterprise Session Monitor license key.

The following steps should be completed for each compute node within your environment that has either a SPRE or Compute engine installed and configured.

1. Unzip the downloaded sas-proc-mon_unix.zip file into the modules folder of your Agent installation.
2. Update the vars.yaml file changing the following items²:
   1. The value of sasType to viya.
3. Create the sas-proc-mon service:
   1. Update line 6 of the sas-proc-mon.service file updating the values of ESMHOME, ESMEVENTS, ESMNODENAME for your environment including replacing <HOSTNAME> with the name configured during the installation of the agent.
   2. Update line 9 for the path to your modules folder
   3. As root or using sudo, copy the updated sas-proc-mon.service file to /etc/systemd/system
   4. As root or using sudo, run systemctl daemon-reload
4. Start the Module service by executing (as root): systemctl start sas-proc-mon

The module uses SAS logging to collect data. Some modifications need to be made to the default SAS Server logging settings to enable the functionality of the sas-proc-mon module.

Compute Server Logging

To collect PROC metrics and to enable the module to link the log file with a session, we must enable a new esm logger. To collect data acces metrics we must enable dataset access audit logging. Add the following block just above the root logger in /opt/sas/viya/config/etc/compsrv/default/logconfig.xml:

<!-- START ESM Logger -->
<logger name="esm" additivity="false">
    <appender-ref ref="PgmRollingFile"/>
    <level value="Debug"/>
</logger>
<logger name="Audit.Data.Dataset" additivity="false">
    <appender-ref ref="PgmRollingFile"/>
    <level value="Trace"/>
</logger>
<!-- END ESM Logger  -->

Job Execution Service Job Name

1. Complete the Compute Server configuration above.
2. After making a backup, modify jesbegin.sas in /opt/sas/spre/home/SASFoundation/sasautos putting the following lines just before the closing %mendtag:

   /* ESM CONFIG START */
   %include "%sysget(ESMHOME)/sasautos/startjes.sas";
   /* ESM CONFIG END */
   

3. After making a backup, modify jesend.sas in /opt/sas/spre/home/SASFoundation/sasautos putting the following lines just before the options nonotes; (circa line 51)

   /* ESM CONFIG START */
   %include "%sysget(ESMHOME)/sasautos/endjes.sas";
   /* ESM CONFIG END */
   

---

1. For information about all of the configuration options available for the sas-proc-mon module, please see the module's reference guide. ↩

2. For information about all of the configuration options available for the sas-proc-mon module, please see the module's reference guide. ↩

---

Last update: June 16, 2023
Created: April 24, 2023