Configuring SAS for ESM

Having successfully installed the ESM Server and Agent components, SAS environment configuration is now required to enable ESM to monitor the various components of your SAS installation.

Before proceeding with configuration

It is highly recommended that you read and understand the document titled Functional Overview of ESM before continuing with the tasks outlined here. This document is intended to explain the concepts involved in SAS configuration for ESM on both Windows and UNIX based platforms and provides examples for both.

ESM Configuration Assistant for SAS

To assist with the configuration changes required for ESM integration, Boemska provide a utility called the ESM Configuration Assistant. The Configuration Assistant is designed to automate as much of the below configuration procedure as possible, minimising the chance of error while leaving ultimate control of the changes with the user making them.

SAS Server Configuration

This example shows what a SAS configuration directory structure for an environment with 2 levels may look like (individual files are omitted):

Config
   Lev1
     SASApp
       WorkspaceServer
       PooledWorkspaceServer
       StoredProcessServer
       BatchServer
       ConnectServer
     SASBatch
       WorkspaceServer
       PooledWorkspaceServer
       ConnectServer
     SASLASR
     SASEnterpriseMiner
   Lev2
     SASApp
     SASBatch

LevN subdirectories contain configuration files used by SAS to instantiate the various component processes for a given SAS configuration. The contents of each subdirectory vary by component. Multiple Levels (Levs) are often used to enable a physical host to be logically divided to support development, testing and production. For example, Lev1 may be used for Production and Lev2 and Lev3 for Testing and Development, respectively.

If a node is split logically into multiple Levels, it is important that all active components are configured for monitoring with ESM regardless of Lev. If this is not done, certain sessions on the server will be invisible to ESM, and will not be considered when analyzing the performance of the node they are executing on.

appservercontext_env_usermods

At the Application Server level we will make changes to the appservercontext_env_usermods.sh file to achieve the following:

• Set the ESM Agent installation directory environment variable
• Include and source the ESM Agent configuration script from the ESM Agent config directory.
• Set the ESMACTIVE flag. This setting determines whether the components inside this this Application Server Context that are configured for monitoring with ESM, actively report in to ESM. Default setting is 1 (enabled)
• Extend the SAS invocation options to insert an additional SAS autoexec to capture additional details for ESM reporting

Typically, the code added to this file looks something like this:

UnixWindows

# ESM CONFIG START
if [[ -d /opt/esm/esm-agent ]]
    export ESMBASEDIR=/opt/esm/esm-agent
    # source the script
    . $ESMBASEDIR/conf/esmconfig.sh
    # set the active flag as 1 for active, 0 for inactive
    export ESMACTIVE=1
    export SASV9_OPTIONS="$SASV9_OPTIONS -insert autoexec '/pub/esm/agent/esm-agent/sasautos/esmtags.sas'"
    # NOTE: The ESMACTIVE flag is set independently in the OLAPServer config
    # This is the name given to non-job sessions
    export ESMSESSIONNAME=Lev1_SASApp
fi
# ESM CONFIG END

rem ESM CONFIG START 

Set ESMBASEDIR=c:\esm\esm-agent 
rem source the script 
call %ESMBASEDIR%\conf\esmconfig.bat 
rem set the active flag: 1 for active, 0 for inactive 
Set ESMACTIVE=1 
rem NOTE: The ESMACTIVE flag is set independently in the OLAPServer config 
Set SASV9_OPTIONS="$SASV9_OPTIONS -insert autoexec 'D:\esm\agent\esm-agent\sasautos\esmtags.sas'"
rem This is the name given to non-job sessions 
Set ESMSESSIONNAME=Lev1_SASApp 

rem ESM CONFIG END

Application server configuration

Within each Logical Server subdirectory are the configuration files for each server. Files in each of these will require some additions in order for the Servers to be picked up and monitored by ESM at runtime. The files that need to be modified for the most common SAS Server types are outlined in the following table:

	autoexec_usermods	Server	Server_usermods	Metadata config
Workspace
Pooled Workspace
Batch
Stored Process
Connect
OLAP

- update required to file - update required on Windows platform only
Server is invocation script for that server (i.e. sasbatch.sh or WorkspaceServer.sh)

Server_usermods script changes

In order to associate SAS sessions to their parent server types, an environment variable is set in the Server_usermods file:

# ESM CONFIG START
export ESMSESSIONTYPE=WS
# ESM CONFIG END

The following table identifis the values that should be set for the standard SAS server types

Server Type	ESMSESSIONTYPE
Workspace Server	WS
Pooled Workspace Server	PWS
Data Step Batch Server	Batch
Stored Process Server	STP
Connect Server	Connect
SAS Grid Server	GRID
OLAP Server	OLAP

Invocation Script Changes

Datastep Batch Servers (and Stored Process Servers on Windows Platforms) require their main invocation scripts to be modified to allow the ESM Agent to capture enhanced information, such as batch job return codes and Multibridge session port numbers respectively.

Typical configuration changes to the SAS DATA Step Batch Server Run File look like this:

UnixWindows

# ESM CONFIG START

# The following should be pasted immediately after the line beginning with 
# exec "$SAS_COMMAND", once the exec command has been removed.
rc=$?
$ESMHOME/reportOut.sh $ESMNODENAME $ESMGUID $ESMJOBNAME $rc

# ESM CONFIG END
exit $rc

REM ESM CONFIG START

REM Paste this snippet immediately after the set rc=%ERRORLEVEL% line
call "%ESMHOME%\reportOut.bat" %ESMGUID% "%ESMJOBNAME%" %rc%

REM ESM CONFIG END
exit %rc%

Warning

Incorrect or incomplete modification of the sasbatch.sh or sasbatch.bat script can affect how schedulers such as Platform LSF interpret the status of finished jobs, which can in turn affect the flow and triggering of subsequent jobs.

When modifying these scripts ensure that the rc=$? or set rc=%ERRORLEVEL% commands continue to capture the return code of the job launched by the SAS command, and that the sasbatch.sh or sasbatch.bat script itself exits returning the same return code.

Metadata Configuration Changes

Stored Process Server requests are executed by a pool of Multibridge sessions. Multibridge sessions are runner processes often pre-initialised by the SAS Object Spawner which execute Stored Process program code when requested to do so by the user. In order to capture the attributes of each request into an Event so that ESM can identify the program and person currently using that multibridge session, a Request Initiation program must be specified in metadata. This is done using SAS Management Console.

The SAS Stored Process Server Metadata configuration:

!SASROOT configuration

Optional configuration

This configuration step is not required on most SAS environments. You do not need to complete this step if all of your SAS workload is driven through the previously configured SAS server types.

Unix only

At the moment this is a configuration option that is only support on Unix platforms.

In some environments there are times when SAS is invoked outside of the logical application server types that have been configured to be monitored by ESM as standard. In such a scenario, ESM would have no knowledge of these additional SAS invocations and while host level metrics would be collected for the run time of the invocation, per process and SAS specific metrics may not be collected.

It is possible to configure ESM in the sasenv_local file in !SASROOT/bin in addition to the configuration above. The sasenv (and its associated sasenv_local) file are read with each and every invocation of SAS.

The following configuration checks to see whether or not the standard ESM variables have been set (so that we do not override configured server instances) and if they haven't to configure a new ESMSESSIONTYPE called Foundation with a ESMSESSIONNAME of Foundation. If other parameters have been passed into the program these are configured too.

Unix

# ESM CONFIG START
export ESMBASEDIR=/pub/esm/agent/esm-agent
if [[ -d $ESMBASEDIR ]]
then
. $ESMBASEDIR/conf/esmconfig.sh
export ESMACTIVE=1

if [[ -z $ESMSESSIONNAME ]]
then
    export ESMSESSIONNAME=Foundation
fi

if [[ -z $ESMSESSIONTYPE ]]
then
    export ESMSESSIONTYPE=Foundation
fi

if [[ -z $ESMJOBNAME ]]
then
    ESMFULLPATHFILE=`/bin/bash $ESMHOME/getSysIn.sh $@`
    ESMJOBNAME=`basename $ESMFULLPATHFILE`
    export ESMJOBNAME
fi

export SASV9_OPTIONS="-insert autoexec '$ESMBASEDIR/sasautos/esmtags.sas'"
fi
# ESM CONFIG END 

Validating ESM configuration changes

The configuration changes for each Server can be validated as follows:

Workspace Server

This can be done either using SAS Management console, or using a client such as SAS Enterprise Guide.

To do this via SAS Management Console, navigate to the Server Manager plugin, and locate the SAS Workspace Server under the context that has been configured (eg. SASApp). Right click, and select the Validate option.

The following two things should happen as a result:

1. You should see the following 'Validation Successful' message
   

2. A few seconds later, a session should appear in the List Portlet of the ESM Live View:
   

Note

When SAS Management Console is used to validate the Workspace Server, the session that it spawns is terminated immediately following validation. It is therefore normal for the session to be greyed out as soon as it appears in the ESM Live View. Standard Workspace Sessions started by a client application should persist as expected.*

Alternatively, to validate the Workspace Server via a client such as SAS Enterprise Guide, simply connect to the Server Context in question and execute some SAS code. Your session should appear in the ESM Live View, and should be correctly associated to your SAS Metadata username.

Connect Server

The procedure for validating the SAS/CONNECT Server is very similar to that of a standard SAS Workspace Server. However, instead of right-clicking on the Logical Server definition in SAS Management Console, right-click the server definition itself.

SAS/CONNECT sessions should also be validated by connecting to the SAS/CONNECT server using a suitable client.

SAS OLAP Server

The procedure for validating the SAS OLAP Server configuration uses the SAS Management Console based Validate command, similarly to the validation for the SAS/CONNECT Server or SAS Workspace Server.

SAS Stored Process Server

The Stored Process Server configuration can also be validated from SAS Management Console in the same way as the Workspace Server configuration. However, if the Start Size in the Load Balancer configuration is set to a number higher than zero, the Multibridge Sessions will be started by the spawner, making the Management Console based validation unneccessary.

In either case, there should be at least one Multibridge Session visible in the Live View list portlet:

Note that the Multibridge Port (8611 in this example) should be identified correctly.

To validate the addition to the Metadata configuration which allows each individual request to be identified, execute a stored process using a supported client (such as the SAS Stored Process Web Application). Depending on the Load Balancing configuration, a flag should be drawn against one of the Multibridge Session portlets.

Hovering over the flag should display a tooltip correctly identifying the test program executed.

Pooled Workspace Server

The Pooled Workspace Server configuration should be validated in the same way as the Stored Process Server configuration. However, to issue a request to a Pooled Workspace session in order to validate the identification of that request, you will need to execute an Information Map against that session. This can be done via a Pooled Workspace client application such as SAS Information Map Studio or SAS Web Report Studio.

SAS Batch Server

To help validate the extended functionality for monitoring SAS Batch Jobs, ESM ships with a test Job which is to be used when validating the configuration of the SAS Batch Server. To trigger the batch job, navigate to a directory where you have write permissions, and invoke the sasbatch.sh / sasbatch.bat script, passing it the ESM Validation Job as input. Here is an example using the Windows platform:

C:\> c:\sas\Config\Lev1\SASApp\BatchServer\sasbatch.bat -sysin c:\bts\esm-agent\sasautos\validateBatch.sas

The following should happen:

1. The Batch Job should appear among the other active sessions in the Live View Session List portlet, showing validateBatch.sas in the Name column.

2. Once finished, the Session Portlet for the job should look similar to this: It should exhibit the following features:

   • The job should be roughly bound by a light blue background, signifying it is a job.
   • The background area should be titled with the name of the job across the top.
   • If SASWORK monitoring is permitted to the agent at the Operating System level, blue bars should show the SASWORK tracking working correctly.
   • A white tag titled 'User Tag' should be showing halfway through the job.
   • An ERROR and a WARNING flag should be apparent, highlighting the (artificial) error and warning in the job log.

3. Clicking on one of the E or W flags should retrieve the Job Log from the node's filesystem (if permitted by OS)

4. Typing 'validate' in the Job Name box and clicking the Graph button should load a graph showing the execution of the ValidateBatch.sas job, signified with a green bar.

If you are unable to see any of the above, please contact Boemska Suport at support@boemskats.com.

This concludes the Validation step of ESM Configuration.

SAS Client Configuration

SAS Enterprise Guide

Tags

Annotations of ESM charts using tags allows SAS developers to highlight the different steps of a program when evaluating or comparing performance. ESM tags can be automatically be created in the most common SAS clients including SAS Studio and SAS Enterprise Guide.

ESM tags for the sample process flow from Enterprise Guide

Tasks

Tasks within SAS Enterprise Guide can be automatically picked up and annotations added to the ESM charts by using the &_CLIENTTASKLABEL macro variable that is always configured by SAS Enterprise Guide.

To configure tasks to be automatically annotated in ESM, select Tools > Options within SAS Enterprise Guide and, from the side-bar menu of the dialog that appears, select Tasks > Custom Code.

Selecting Edit... for both the before and after task and query code options, add the following:

• Insert custom SAS code before task and query code;

  %esmtag(Start &_CLIENTTASKLABEL);
  

• Insert custom SAS code after taks and query code:

  %esmtag(End &_CLIENTTASKLABEL);
  

Programs

To configure programs to be automatically annotated in ESM, select Tools > Options within SAS Enterprise Guide and, from the side-bar menu of the dialog that appears, select SAS Programs.

Selecting Edit... for both the before and after task and query code options, add the following:

• Insert custom SAS code before submitted code;

  %esmtag(Start &_CLIENTTASKLABEL);
  

• Insert custom SAS code after submitted code:

  %esmtag(End &_CLIENTTASKLABEL);
  

Exceptions and troubleshooting

SAS OLAP Server for Windows platforms

The SAS OLAP Server on the Windows Platform is unable to source the variables defined in esmconfig.bat, as it does not source the appservercontext_env_usermods script. As a workaround, the ESMACTIVE macro variable value is set manually in the OLAP Server autoexec_usermods.sas file. If you are intending to use the ESMACTIVE environment variable to turn off ESM monitoring for a Windows-based environment, you will also need to independently change the ESMACTIVE variable in the OLAP Server's local autoexec_usermods.sas configuration file.

Pooled Workspace or Stored Process Servers TK Kernel Errors

These errors can occur when the incorrect path to a Request Initialisation or Session Initialisation program is configured in SAS Metadata, or if the sassrv process can not read the file. If this error appears, check the .sas files configured in SAS Metadata exist and are readable by the sassrv user. More information about the setting in question can be found under the Metadata Configuration Changes section of this document.

Batch Server Logfile Tracking and logconfig.xml configuration

Logfile pattern configured under Metadata

If your SAS Data Step Batch Server configuration uses the server context logconfig.xml file to specify the logfile filename pattern rather than having it specified in the Batch Server Metadata definition (i.e. when your environment is configured for APM), some additional steps will be required in order to fully integrate the Batch Server with the logfile tracking functionality of ESM.

The ESM integration macros rely on the SAS System Options to ascertain the name of the logfile that a SAS job is writing to. The Rolling Log Options filename pattern defined in Metadata dictates the -log parameter value that is passed to the Batch Job process at runtime, which in turn sets the LOG SAS option for that process. Without this option available, and with no other way for a Macro to ascertain the exact name of the logfile from inside the SAS session and the full name of the logfile needs to be reconstructed during initialisation using some DATA Step code.

Logfile pattern configured under logconfig.xml

This can be done by substituting the following line from the %esm_reportIn() integration macro in esmtags.sas:

logloc = getoption('log');  

to something along the lines of the following code:

data testings;
 length dd mm hh min ss $2.;
 length yyyy $4.;
  startval=datetime();
  baselogname = getoption('log');

  late = 0;
  /* Change the maximum value of the late variable 
     to suit the startup delay time of your environment   */ 
  do until (late = 10);
   testval = startval - late;
     dd=put(day(datepart(testval)),z2.);
     mm=put(month(datepart(testval)),z2.);
     yyyy=put(year(datepart(testval)),z4.);
     hh=put(hour(testval),z2.);
     min=put(minute(testval),z2.);
     ss=put(round(second(testval)),z2.);
    suffix=cats(yyyy,'.',mm,'.',dd,'_',hh,'.',min,'.',ss);
   late + 1;
   testfile = cats(baselogname,suffix,'.txt');
   if fileexist(testfile) then do;
    logloc = testfile;
   end;
  end;
 put filefound=;
run;

This code block can be found in esmtags.sas and is commented out by default. Note that the maximum value of the late variable, as set at the beginning of the do until loop, should be adjusted depending on the variability in startup time for your SAS Batch sessions. This can depend on a number of factors. For this reason it is recommended that your ESM Integration Macros are the first thing that is called by your Batch Server Autoexec file.

Per-session tracking of SAS UTILLOC size

SAS multi-threaded procedures can make use of a disk paging location referred to as UTILLOC. ESM is designed with the capability to track the size of this UTILLOC usage of each individual session, but to enable this functionality, some extra configuration is required.

Unlike the SAS WORK location, which is a directory created at session startup that is unique to each individual SAS session, SAS UTIL directories are created on-the-fly by SAS procedures as they are required. Therefore, unlike the WORK system option which holds the full path to each session's individual SASWORK directory, the UTILLOC system option points to the parent directory which can contain any number of UTIL directories belonging to a number of other sessions. This makes periodic size measurements of a session's individual set of UTIL directories challenging.

One way of addressing this issue is to create a per-session UTILLOC subdirectory for each session before it is started, using it's usermods initialisation script, and removing the directory after the session has finished executing. This results in all of a given session's UTIL directories being created under the same parent directory, making it more efficient to size and easier to clean up, and the UTILLOC option pointing to a directory dedicated to that particular session. An example way of doing this for a Workspace Server would be:

Add this to WorkspaceServer_usermods.sh to create the subdirectory and tell SAS to use it:

# set a unique subdirectory for the util loc
if [[ $ESMACTIVE -eq 1 ]]; then
  export ESMUTILLOC=/utilloc/$(hostname -s)_$$
  mkdir $ESMUTILLOC;
  export ESMUNQUTIL="-utilloc $ESMUTILLOC"
else 
  ESMUNQUTIL=
fi

set -A USERMODS_OPTIONS   # initialize empty list
# build up list by un-commenting (and adding) the lines you need (one line per token)
# then replace the <argument> with the values you want for each token on the command line
USERMODS_OPTIONS[${#USERMODS_OPTIONS[*]}]="$ESMUNQUTIL"  

Then append this to the end of WorkspaceServer.sh to clean it up once the SAS session has finished executing:

if [[ $ESMACTIVE -eq 1 ]]; then rmdir $ESMUTILLOC; fi

Warning

Unlike the other configuration changes presented in this guide, this change alters the standard SAS behaviour and is therefore not included as part of the standard ESM Configurator workflow. This change should be tested and documented. Use at your own risk.**

Real time monitoring of Workspace Server logs

It is sometimes desirable to be made aware of ERROR and WARNING messages as soon as they are emitted to the Workspace session log, often before they have appeared in the client UI. It may also be useful to review log messages from a previous Workspace session execution, long after that session has completed. In these instances, ESM can be configured to monitor SAS Workspace Server logs and to highlight log messages such as ERROR and WARNING messages in real time.

To configure this functionality the following steps need to be taken:

1. Enable and configure logging for the SAS Workspace Server (not enabled by default)
2. Modify ESM Workspace Server configuration for SAS Workspace Server logging
3. (Optional) Simplify formatting of the SAS Workspace Server log
4. Log file management

Configure SAS Workspace Server Logging

SAS product documentation provides instructions for enabling Workspace Server logging.

Modify ESM Workspace Server configuration for SAS Workspace Server logging

In order for ESM to track the SAS Workspace Server logs that are now being created, the names of those logs need to be communicated to the ESM Agent.

The original configuration for SAS Workspace Servers involves adding an ESM configuration block to the WorkspaceServer_usermods file. Add the following lines inside the ESM CONFIG block, just before the ESM CONFIG END comment:

Slashes matter

The trailing slashes in the below LOGPATH variables are important. Don't forget to add them

UnixWindows

export LOGPATH=/opt/sas/config/Lev1/SASApp/WorkspaceServer/Logs/
export ESM_APPSERVER_NAME=$(basename $APPSERVER_ROOT)

set LOGPATH=D:\SAS\Config\Lev1\SASApp\WorkspaceServer\Logs\
for /D %%i in (%APPSERVER_ROOT%) do set ESM_APPSERVER_NAME=%%~ni

(Optional) Simplify formatting of the SAS Workspace Server log

This step is optional

This step is not required by ESM. It makes the generated logfiles smaller in size, lessening the impact of enabling the logging on available storage. It also makes them easier to read.

Proceed with caution

This step makes a change to the logconfig.trace.xml. Modifications to this file, if not completed correctly, may cause issues on your system.

It is highly recommend that you familiarise yourself with the documentation for the SAS Logging Facility for your version of SAS, and ensure that you take a backup of any SAS configuration file before making changes.

As noted in the documentation on the SAS Support Site the primary purpose of enabling Workspace Server logging is to provide SAS Technical Support with high-level logging messages which can be used by SAS Technical Support to help resolve issue. As part of this information, SAS prepends a large amount of data to the beginning of each line of the SAS log:

2020-03-31T23:38:35,973 INFO  [00000006] :chris -  Linux LIN X64 3.10.0-957.27.2.el7.x86_64 #1 SMP Mon Jul 29 17:46:05UTC 2019
2020-03-31T23:38:35,973 INFO  [00000006] :chris -       x86_64 CentOS Linux release 7.6.1810 (Core)
2020-03-31T23:38:35,973 INFO  [00000006] :chris -
2020-03-31T23:38:35,973 INFO  [00000006] :chris - You are running SAS 9. Some SAS 8 files will be automatically converted
2020-03-31T23:38:35,974 INFO  [00000006] :chris - by the V9 engine; others are incompatible.  Please see
2020-03-31T23:38:35,974 INFO  [00000006] :chris - http://support.sas.com/rnd/migration/planning/platform/64bit.html

For most purposes, the bulk of this information is not relevant and makes the log file significantly larger and more difficult to read. A small modification to line 14 of the logconfig.trace.xml file removes all of the prepended information, leaving a more traditional and easier to read SAS log:

Modify:

<param name="ConversionPattern" value="%d %-5p [%t] %X{Client.ID}:%u - %m"/>

To:

<param name="ConversionPattern" value="%m"/>

Diffing the two files should yield results like below:

--- a/logconfig.trace.xml
+++ b/logconfig.xml
@@ -11,7 +11,7 @@
       </rollingPolicy>
       <layout>
          <param name="HeaderPattern" value="Host: '%S{hostname}', OS: '%S{os_family}', Release: '%S{os_release}', SAS
-         <param name="ConversionPattern" value="%d %-5p [%t] %X{Client.ID}:%u - %m"/>
+         <param name="ConversionPattern" value="%m"/>
       </layout>
    </appender>

A further small modification to Application message logger of the logconfig.trace.xml file to remove all of the Debug messages making the SAS log even easier to read:

Modify:

   <!-- Application message logger -->
   <logger name="App" immutability="true">
      <level value="Debug"/>
   </logger>

To:

   <!-- Application message logger -->
   <logger name="App" immutability="true">
      <level value="Info"/>
   </logger>

Diffing the two files should yield results like below:

--- a/logconfig.trace.xml
+++ b/logconfig.xml
@@ -11,7 +11,7 @@
   <logger name="App" immutability="true">
-      <level value="Debug"/>
+      <level value="Info"/>
   </logger>

Log file management

With a high number or interactive users or Workspace sessions, you may find that, even if you reduce the size of the log file the number and size of the log repositories can quickly grow. By default SAS creates workspace server logs in a Logs folder within the Workspace folder of your SAS configuration. If filling up this storage location with logfiles poses an issue, consider implementing one or more of the following:

1. Redirection of log files to another disk. This is most easily accomplished by creating a symbolic link for the Logs directory to an alternate location. Redirecting the files in this way requires fewer SAS configuration file changes.
2. Regularly delete old logs. For example, schedule a simple script to perform a housekeeping task on your system which deletes files older than a certain age.

The following delete files older than 14 days:

UnixWindows

find /opt/sas/config/Lev1/SASApp/WorkspaceServer/Logs -type f -mtime +14 -exec rm -f {} \;

ForFiles /p "D:\SAS\Config\Lev1\SASApp\WorkspaceServer\Logs" /s /d -14 /c "cmd /c del @file"

Warning

The above commands are destructive and assume familiarity with system level maintenance routines.

Real time monitoring of Connect Server logs

It is sometimes desirable to be made aware of ERROR and WARNING messages as soon as they are emitted to the Connect Server session log, often before they have appeared in the client UI. It may also be useful to review log messages from a previous Connect Server session execution, long after that session has completed. In these instances, ESM can be configured to monitor SAS Connect Server logs and to highlight log messages such as ERROR and WARNING messages in real time.

To configure this functionality the following steps need to be taken:

1. Enable and configure logging for the SAS Connect Server (not enabled by default)
2. Modify ESM Connect Server configuration for SAS Connect Server logging
3. (Optional) Simplify formatting of the SAS Connect Server log
4. Log file management

Configure SAS Connect Server Logging

SAS product documentation provides instructions for enabling Connect Server logging.

Modify ESM Connect Server configuration for SAS Connect Server logging

In order for ESM to track the SAS Connect Server logs that are now being created, the names of those logs need to be communicated to the ESM Agent.

The original configuration for SAS Connect Server involves adding an ESM configuration block to the ConnectServer_usermods file. Add the following lines inside the ESM CONFIG block, just before the ESM CONFIG END comment:

Slashes matter

The trailing slashes in the below LOGPATH variables are important. Don't forget to add them

UnixWindows

export LOGPATH=/opt/sas/config/Lev1/SASApp/ConnectServer/Logs/
export ESM_APPSERVER_NAME=$(basename $APPSERVER_ROOT)

set LOGPATH=D:\SAS\Config\Lev1\SASApp\ConnectServer\Logs\
for /D %%i in (%APPSERVER_ROOT%) do set ESM_APPSERVER_NAME=%%~ni

(Optional) Simplify formatting of the SAS Connect Server log

This step is optional

This step is not required by ESM. It makes the generated logfiles smaller in size, lessening the impact of enabling the logging on available storage. It also makes them easier to read.

Proceed with caution

This step makes a change to the logconfig.trace.xml. Modifications to this file, if not completed correctly, may cause issues on your system.

It is highly recommend that you familiarise yourself with the documentation for the SAS Logging Facility for your version of SAS, and ensure that you take a backup of any SAS configuration file before making changes.

As noted in the documentation on the SAS Support Site the primary purpose of enabling Connect Server logging is to provide SAS Technical Support with high-level logging messages which can be used by SAS Technical Support to help resolve issue. For most purposes, the bulk of this information is not relevant and makes the log file significantly larger and more difficult to read. A small modification to line 14 of the logconfig.trace.xml file removes all of the prepended information, leaving a more traditional and easier to read SAS log:

For the rolling file appender, modify the layout of the lines written to the log

<param name="ConversionPattern" value="TRACE"/>

To:

<param name="ConversionPattern" value="%m"/>

For the rolling file appender, modify the App.Connect and App.WQ loggers from Trace to Info:

    <logger name="App.Connect">
      <level value="Trace"/>
    </logger>

    <logger name="App.WQ">
      <level value="Trace"/>
    </logger>    

To:

    <logger name="App.Connect">
      <level value="Info"/>
    </logger>

    <logger name="App.WQ">
      <level value="Info"/>
    </logger>   

Diffing the two files should yield results like below:

--- a/logconfig.trace.xml.orig
+++ b/logconfig.trace.xml
@@ -11,7 +11,7 @@
       </rollingPolicy>
       <layout>
-         <param name="ConversionPattern" value="TRACE"/>
+         <param name="ConversionPattern" value="%m"/>
       </layout>
    </appender>

@@ -104,12 +104,12 @@ _ENDDISARM

    <!-- Connect Application logger -->
    <logger name="App.Connect">
-      <level value="Trace"/>
+      <level value="Info"/>
    </logger>

    <!-- WQ logger -->
    <logger name="App.WQ">
-      <level value="Trace"/>
+      <level value="Info"/>
    </logger>

Log file management

With a high number or interactive users or Connect sessions, you may find that, even if you reduce the size of the log file the number and size of the log repositories can quickly grow. By default SAS creates workspace server logs in a Logs folder within the Connect Server folder of your SAS configuration. If filling up this storage location with logfiles poses an issue, consider implementing one or more of the following:

1. Redirection of log files to another disk. This is most easily accomplished by creating a symbolic link for the Logs directory to an alternate location. Redirecting the files in this way requires fewer SAS configuration file changes.
2. Regularly delete old logs. For example, schedule a simple script to perform a housekeeping task on your system which deletes files older than a certain age.

The following delete files older than 14 days:

UnixWindows

find /opt/sas/config/Lev1/SASApp/ConnectServer/Logs -type f -mtime +14 -exec rm -f {} \;

ForFiles /p "D:\SAS\Config\Lev1\SASApp\ConnectServer\Logs" /s /d -14 /c "cmd /c del @file"

Warning

The above commands are destructive and assume familiarity with system level maintenance routines.

Configuring Error and Warning event generation using ESM Logfile Markers

By default, ESM is configured to detect Errors and Warnings from SAS logs in the default SAS format, as generated by the Batch Server and SAS Foundation sessions. Following the steps above to reduce the size of the log file will simplify the logfiles generated by the Workspace Server SAS Logging facility so that they follow the same format, and ESM will begin to detect Error and Warning messages from the Workspace Server logs without any further configuration.

If you have previously configured Workspace Server logging without updating the logfile format, and do not wish to update that configuration, Workspace Server logs will still be available for viewing through the ESM interface. However some changes to the ESM log marker configuration will be required to enable the detection of Error and Warning messages that are prepended with a timestamp, or any othe properties specified in logconfig.xml.

Changes to the Log Marker configuration are made using the Log Markers configuration screen in the ESM Admin Settings section.