Release notes for RunMyJobs releases 9.2.8.0 through to 9.2.8.11

Before: When creating a JDBC process definition, with a parameter JDBCEnableParameterSubstitution, then a constraint was automatically created for this parameter. If the constraint definition did not exist, it was created. When this process definition is then exported and imported into another environment, where this constraint definition does not yet exist, the import could fail. A workaround is to create manually a temporary JDBC process definition with a parameter JDBCEnableParameterSubstitution, as this will create the constraint definition. After that the temporary JDBC process definition can be dropped again.

After: Now, the constraint definition is created at startup. Therefore the constraint definition is available when such a process definition is imported. 

Before: When using extension points messages of the following form were logged:

ERROR 2021-04-19 14:54:47,868 GMT [http-nio-10180-exec-50-Executor 1874661] com.redwood.scheduler.extensionpoint.impl.ExtensionPointHTTPSession - Asked for SchedulerSession [ACTIONSUBJECT] but it was not available.

After: In normal usage, this message will no longer be logged.

Before: Readable descriptions for HTTP and X509 credential protocols were not available.

After: Descriptions have been added.

Before: The link to help about SimpleDateFormat on the support page was not clickable.

After: The link is rendered as a link that can be clicked on.

Before: When viewing the output from a generated Object Search process in Chrome, then the "expand/collapse" icon is garbled.

After: This icon is correctly shown in all browsers.

Before: Empty output/error files are shown when you run a System_Sleep job.

After: Empty files are removed and not shown for a System_Sleep job.

Before: Since 9.2.8.0 and in very specific circumstances, it is possible that a process could be scheduled a minute late. E.g. when a chain call of a chain is put on hold, immediately after the chain is submitted to run immediately, then there is a small chance that the chain gets scheduled a minute late.

After: this timing issue is fixed.

Before*:* HTTP 30x redirection messages were incorrectly logged under debug logging as failed responses:
"HTTP server failed response with response code 302"
Also, no distinction was made between HTTP and HTTPS calls in the debug logs.

After*:* 30x messages are not reported as failed responses anymore. Instead are they reported as:
"HTTP server responded with URL redirection code 302"
Also, a distinction is made in the log messages between HTTP and HTTPS.

Before: Errors could be logged in the form of:

ERROR 2015-03-11 12:01:23,158 Europe/Amsterdam [Redwood monitoring for SAP instance SC3 worker 4] com.redwood.scheduler.persistence.tdump - Start Transaction id=392474620 contains 1 objects. Persistence Error: 
com.redwood.scheduler.persistence.api.PersistenceException$NoRowsUpdated: JCS-123100: 0 rows updated in tid=392,474,620

After: These errors may still be logged, and they will be logged to a separate file, norowsupdated.log. The NoRowsUpdated will be logged at INFO. These can be ignored, but may be asked for by support when investigating issues.

Before: It was possible to duplicate System_FTP_mget_Parallel and System_FTP_mput_Parallel, even though nothing could be changed on the resulting duplicate.

After: To prevent confusion, it is no longer possible to duplicate these definitions.

Before: Calling getCurrentValueString() on a Date or Number parameter would cause a ClassCastException.

After: Calling getCurrentValueString() on a Date or Number parameter will return the same value as getInValueString() or getOutValueString depending on whether the parameter is an IN parameter or not, respectively. This makes is consistent with the other getCurrentValue*() methods.

Before: Missing reason translations for HTTP error response codes would result in false positive errors and warnings written to the log file (we generally do not expect to have a reason translation).

After: No errors or warnings are written to the log file when a reason translation is missing for an HTTP error response code.

Before: If a user had sufficiently limited privileges it was possible to cause an internal server error to be shown when clicking on processes.

After: These cases have been fixed, and it is now possible to see these jobs in the user interface even if you cannot see the related process definition.

Before: The information of components that are initialised by lifecycle manager was exposed.

After: Do not expose the information of components that are initialised by lifecycle manager.

Before: When selecting a queue provider for read, the internal routing table would be recalculated, even though the queue provider had not changed. This could occur when setting a field to have the same value that it already has.

After: The internal routing table is only recalculated if a change to the routing information occurs. This avoids unnecessary work from being done.

Before: Under some circumstances it was possible for a resilient process to finish before the system had finished starting up. If this happened, and a second process was waiting for the first process, then a delay of up to one hour could occur before the completion of the process was noticed by the waiting process.

After: This has been fixed so that the waiter will always immediately see the completion of the child process.

Before: Default filter "All closed queues" shows queues which are held, which is confusing.

After: The filter has been renamed to: "All Held Queues".

Before: The JDBC driver for AS400 does not support the Connection.getTypeMap() function and therefore creating a connection failed.

After: When the JDBC driver does not support the Connection.getTypeMap() function, then this will not cause creating a connection to fail.

Before: R2W was denying soap requests from the scheduler due to the missing authentication token.

After: R2W is now able to authenticate the incoming soap request from the scheduler due to the available authentication token.

Before: You could set a number of HTTP Headers when submitting a HTTP/SOAP job.

After: The following headers are set automatically and should not be set anymore by the user.

• connection
• cookie2
• host
• soapaction
• transfer-encoding
• x-rw-hmacmd5

Currently the following headers can be set:

• authorization: if set the HTTP command will use this on, otherwise it will check for a credential and set Basic authentication.
• accept: Only one accept content type is allowed. If set the send command will be using it.
• accept-encoding: only gzip and identity values can be set and will be used.
• content-encoding: if set we will send the body in gzip.
• content-type: if set the body will be sent with the specified content type. By default the content-type will be set to application/octet-stream.
• cookie: if the Cookie header is set we will determine the cookies and pass these through.
• user-agent: This is set automatically to identify the scheduler as the originator of the request, but can be overridden.

For every Header you can also set a job definition parameter prefixed with the JobDefinitionType (e.g. HTTP_Content-Type). For all headers the scheduler will search for the job parameter with the specified name, when not found it will search the [headers] section.

Before: The variables in the body text were not replaced by their parameter values.

After: Variables are replaced with parameter values again.

Before: Due to a cache initialization omission, credentials of type 'proxy' were not used for the system-wide outgoing HTTP proxy until a change was made to this credential or any other credential of type 'proxy'.

After: The HTTP proxy cache is now properly initialized on boot with the available credentials.

Before: A credential was created in the GLOBAL partition when there was no credential found. If no authentication was needed for the SOAP call a "{redwood}:nocredential" credential was created.

After: If authentication is needed for the SOAP call you need to provide a predefined credential. The credentials are shown in a List. If no authentication (the default) is needed we don't create any credential anymore. 

Before: It was only possible to use 1 distribution id when submitting a job to PeopleSoft.

After: You can now use multiple distribution ids separated by either a , (comma) or ; (semicolon).

Before: An NullPointerException could be thrown when we are not able to find a correct credential for the basic authentication.

After: When we do not find a correct credential we will now set the return code of process to be the return code of the SOAP request (e.g. 401)

Before: Replacement expressions are not working with empty strings in SOAP definitions if variable value is null.

After: Replacement expressions are also applied on empty strings when the variable is null.

NOTE: A replacement expression for a variable that doesn't exist will be replaced with the original expression. For example, if your process has a parameter P1 with no value, a parameter P2 with the value test< then the replacement:

<body>
  <P1>${P1}</P1>
  <P2>${P2:xml}</P2>
  <P3>${P3}</P3>
  <empty>${}</empty>
</body>

will result in:

<body>
  <P1></P1>
  <P2>test&lt;</P2>
  <P3>${P3}</P3>
  <empty>${}</empty>
</body>

Before: PeopleSoft jobs can't run when the PeopleSoft system is created in any partition other than GLOBAL.

After: PeopleSoft jobs can now run when the PeopleSoft system is created in any partition.

Before: When starting PeopleSoft PSJob processes, only generic parameters could be set. You could not set any specific parameter value for a PSJob subitem.

After: When importing PSJob definition from PeopleSoft the subitem parameters are imported and shown on separate tabs. Now, for all these processes that are run by the PSJob definition parameters can be set individually.

Before: If a parameter was mandatory and the process is in a chain, the input value always was checked for null values. However the values are not yet passed to the chain call at this time and so the validation incorrectly failed.

After: The check is done at the correct time. That is, they are skipped when the process is a chain call before starting, and are checked at the time that the call is scheduled to start.

Before: After filling in the responsibility it could take a while that you see the next field filled. The reason is that all fields are checked to pre-fill default data.

After: It is possible to influence the behavior pre-filling mapped parameters - keep the recent behavior and prefill all mapped parameters, only pre-fill related mandatory mapped parameters or no pre-fill

Before: When using REL expressions to pass in values in OraApps job parameters this could fail when the job is used in a chain.

After: The REL expression is evaluated before passing the value to the job constraint for checking.

Before: JDBC jobs running against a MariaDB database failed with a java.sql.SQLSyntaxErrorException: invalid callable syntax. must be like

After: JDBC jobs can now execute SQL statements against a MariaDB database.

It is now possible to define custom indexes. In order to do so, create an ObjectIndex with one or more ObjectIndexColumns, specifying the object and fields for this index.

Before: Mail process definitions failed when sending to multiple recipients.

After: Mail process definitions can send to multiple recipients, separated by either , or ;.

Before: Before version 9.2.8, parameter values in HTTP/SOAP requests are not escaped. From version 9.2.8 onwards, parameters are escaped with XML.

After: You now can specify how parameter values should be escaped. Without specifying an escaping rule, there is no escaping of the parameter value.

There are 4 options to escape a parameter value:

1. xml – escape using XML 1.1 character escaping rules, the result will also use character entities for any non-ASCII character
2. json - escape the parameter so that it can be included inside of a JSON string
3. url - escape using URL escaping, assuming a UTF-8 character set
4. html - escape using HTML character escaping rules

To specify an escape rule you need to add an option on the parameter substitution, e.g. ${parameter_name:xml}.

You can specify multiple escape rules on 1 parameter by appending them after each other. They will be handled in sequence of specifying, e.g. ${parameter_name:xml:json} will first escape using XML entities, and then escape that result using json, so an input of ' would become \' whereas ${parameter_name:json:xml} would convert a ' into \', as it would first convert into JSON ( \' ), and then convert that using XML entities.

Before: The PeopleSoft connector reads the WSDL in to determine which SOAP calls can be made. The connector was based on reading WSDL 1.1.

After: The PeopleSoft connector can now read WSDL 1.1 and 2.0 files to determine the SOAP calls that can be used.

Before: Constraints were not supported in custom submit forms in the Runner and Studio, i.e. if you had a String parameter with an LOV in a custom submit form, a Text Field was rendered, without showing a dropdown with the possible values.

After: LOVs are supported in custom submit forms in the Runner and in Studio now.

Before: The native library files are not accessible when running as a Windows Service.

After: The appropriate directory is now on the java.library.path when running as a Windows Service.

Before: The column chooser for databases showed the column Partition twice.

After: This has been fixed.

Promotion system message has been improved to avoid misinterpretation:

Before: Objects {0} and {1} have been promoted to system {2}

After: Objects {0} and {1} are being promoted to system {2}

Some fields have default values in Job Chain Editor Raise/Wait Events. Present default values for those fields by default.

Before: In some cases a Process Server could not be renamed to an earlier used name. This happened if the names of the Process Server and the Queue were the same, but it could happen in some other rare cases as well.

After: This has been fixed. It's possible to rename Process Servers again (reusing old names as well).

Before: Entities (either builtin or custom) could not be deleted if there was a custom entity referencing it.

After: You can now specify the delete type when creating a custom foreign key. This has the following effect when trying to delete a parent record:

• restrictParent/empty: the parent cannot be deleted if it still has children
• withParent: when deleting then parent, all of its children are deleted as well
• noRestriction: the parent can be deleted, even if it still has children

Before: If an additional source for configuration values was configured but could not be initialised, the system would start by not using this source. This could potentially allow a user to have access to parts of the system that would otherwise be blocked.

After: When an additional source can not be initialised, the system will not start until the problem is resolved.

Before: UserMessage jobs showed output files that couldn't be viewed and gave errors when viewing.

After: UserMessage jobs will not show empty files any more. Files that are attached to the job can be viewed.

Before: UserMessageHistory.rtx was incomplete when being downloaded.

After: UserMessageHistory.rtx contains valid and complete xml after being downloaded.

Some SAP definitions could produce superfluous errors in the log, This has been fixed.

Before: Processes marked as ExternalWaitForChild that created no output or log files would erroneously not delete the JobFile objects which could then be accessed via the UI, which would then display an error.

After: ExternalWaitForChild processes will correctly mark their files when completed.

Before: All the PI channels matching wildcard pattern were imported.

After: Import of PI channels based on wildcard match is limited to max 50. 

Before: When requesting the support files some stacktraces were being logged at debug level, but these were not actual errors.

After: Improved the logging for Get support files (part of which is that the stacktraces are not logged anymore).

Before: A user with the admin role didn't have privileges for the object definition 'Built In Web Service' (and couldn't grant it to other users/roles). For the object definitions 'Process definition type' and 'User Login', a user with the admin role could grant the 'view' privilege, but not the 'All' privilege.

After: Fixed the missing/incorrect grants.

Before: When using a process definition with FileSearch and creating a process note when the search string was found, the process failed when the line in which the search string was found had more than roughly 900 characters.

After: If a long matching line is found, JobFileSearch post-running action Success message is being truncated to 800 characters to prevent the process note creation from failing.

Before: It was not possible to have multiple RaiseEvents in a JobChainCall for the same event, but with with a different statuses.

After: It is possible to have the same RaiseEvent with a different status in a JobChainCall.

Before: In rare circumstances (such as a server crash e.g.) some chains would get stuck where steps and calls would be in a Chained status. These chains were not picked up anymore, even after startup of the server.

After: Chains where steps and calls are stuck in Chained are now picked up and processed properly again.

Before: Connecting with OAuth 2.0 authentication to Microsoft Outlook over POP3 and POP3S in a MailConnector was not possible. Also the MailConnector will shut down due to an error when trying to move emails using POP3 or POP3S.

After: Connecting with OAuth 2.0 authentication to Microsoft Outlook over POP3 and POP3S in a MailConnector is now possible. The MailConnector will also not try to move emails using POP3 or POP3S anymore and will not shut down due to the resulting error.

Before: If the expression for a file or table parameter could not be parsed, then the error message didn't help to determine which parameter causes the issue.

After: The error message now includes the name of the parameter, and for which process the parameter belongs.

Before: The Monitoring Dashboard used the field Job.LastModificationTime to determine whether a job was longer than some time in a certain status. This field is not updated for each status change, so this could lead to unexpected behaviour.

After: We introduced a new field Job.LastStatusChangeTime that is updated each time the status of a job changes. This new field is used in the Monitoring Dashboard now.

Before: The monitoring servlet returned an error code if the process server was in a different partition. This was caused by the fact that the services could not be retrieved.

After: This has been fixed. The services are retrieved successfully now and the response indicates OK.

Before: On the Global Configuration screens on the Monitoring Dashboard and the Housekeeping Dashboard the minimum and maximum values for each option were displayed. However, you could still enter invalid values and these were stored in the database.

After: The values for all global configuration options on the dashboards are validated now before they are stored.

Before: System_CollectJobOutput and System_ForceJobStatusUnknown can't be used in a JobChain as the constraints defined on the Job Definition are fired too early.

After: The constraints are now fired correctly on time.

Before: Default expressions for password parameters are normally concealed, but still shown in the tooltip.

After: We now also conceal default expressions for password parameters shown in the tooltip.

Before: When using jcsJobContext.waitForAllChildJobsExternalCompletionStrategy()and if the child job finished at the same time that the parent job script finished, then it was possible that the parent would remain in waiting.

After: This race condition has been eliminated, such that the parent job will always go to Completed, or whichever status is appropriate depending on the child jobs.

Note: There is a very small window where this race condition can occur, in a normally running system these two jobs would both need to finish in the same millisecond to trigger this issue.

Before: The documentation accessed from the product was shipped with the product, meaning that the documentation for a release could not be updated.

After: The documentation accessed from the product accesses a central documentation server provided by Redwood http://docs.redwood.com/. This has improved searching and can be updated when issues are noticed, thus immediately helping all customers.

Upgrades the version of Apache Standard Taglibs to 1.2.5 to address CVE-2015-0254 which allowed XXE attacks through JSTL XML tags which are not used.

Before: It was possible to get a segmentation fault with jscp if connection failures were detected.
After: The potential segmentation faults in jscp have been fixed.

Before: Poor performance on parsing large sectioned files ( noticeable by sending large e-mails as mail-body ).

After: Performance is now significantly improved. In one case, a file that took several hours to parse is  now parsed in less than a second.

Before: Importing a multiple step SAP job as a single process definition created unnecessary constraints which resulted in an error.

After: Unnecessary constraints are not created during the import.

Before: When running the inbuilt system definitions, log entries such as the following would be logged in scheduler.log:

DEBUG 2021-01-01 13:00:00,000 GMT [Redwood Job Thread Pool: GLOBAL.System.System worker 0] com.redwood.scheduler.db.epoch.continuous.jobdef.SystemJobDefinition - isolationGroup r:null w:public void com.redwood.scheduler.system.jobs.RemoveOldJobs.setIsolationGroup(java.lang.String) 
DEBUG 2021-01-01 13:00:00,001 GMT [Redwood Job Thread Pool: GLOBAL.System.System worker 0] com.redwood.scheduler.db.epoch.continuous.jobdef.SystemJobDefinition - jobDefinitionPrefix r:null w:public void com.redwood.scheduler.system.jobs.RemoveOldJobs.setJobDefinitionPrefix(java.lang.String)

After: These are now logged to a different category, and so no longer turn up in scheduler.log.

Before: Under certain circumstances, when running the scheduler on Windows, the cache files for the scripting files may not have been cleaned up.

After: These files will now be successfully cleaned up when no longer in use.

Before: To run the system job System_RemoveOldJobs no locks were required, so this job could run in parallel with other jobs.

After: To run System_RemoveOldJobs the lock System_Maintenance_Lock is needed now, so that the job cannot run in parallel with other maintenance jobs such as System_ProcessKeepClauses.

Before: For String parameters with a simple constraint the values can be language specific. When submitting in the Runner, the values were translated correctly. But when viewing an already submitted job in the Runner, the system switched back to the default locale.

After: This has been fixed; when submitting as well as when viewing a submitted job in the Runner, the values are translated according to the user's locale.

Increased the version of the ISU transports to restore compatibility with 9.2.6 release.

The version of commons-compress has been updated, this fixes CVE-2019-12402 which could potentially affect the PowerBI client.

Upgrades the version of jsoup used by the Redwood_Robotics and Redwood_MailReactor libraries from 1.13.1 to 1.14.2, to overcome CVE-2021-37714 which potentially allowed for denial of service when parsing untrusted HTML

Before: When the jftp tool is invoked with an mput from reads from an absolute directory name, then the local directory is not reset to the current working directory on completion. So another mput in the same command sequence will fail if it specifies the local current directory. For example this would fail :

jftp mput myhostname myusername mypassword "/my/local/nested/file.txt" /my/remote/dir "file_in_current_directory.txt" /my/remote/dir

After: When the jftp tool is invoked with an mput from reads from an absolute directory name, then the local directory is always reset to the current working directory. So another mput in the same command sequence that specifies the local current directory will succeed.

Before: A permanent invalid job-processor message would lead to an endless loop of retry attempts, filling up the log with messages like "Persistent message <message number>.001 does not contain message header".

After: The network-processor will retry the invalid message file for a maximum of 10 times and then moves it to the 'err' directory.

Before: jsecret did not correctly parse a comma separated list of proxy server URLs supplied either as a command line option or within an HTTP_PROXY environment variable.

The Platform Agent failed to correctly parse more than one proxy server URL within a proxy_url Platform Agent configuration file.

After: jsecret correctly parses a comma separated list of proxy server URLs supplied either as a command line option or within an HTTP_PROXY environment variable.

The Platform Agent correctly parses one or more proxy server URLs within a proxy_url Platform Agent configuration file.

Before:

• When deleting a Process Definition from the its details page, an error reporting a ClassCastException would show.
• When deleting a Process Definition related object (Chain, Report, ...) from its overview page, the object would still be visible in the overview with a name prefixed by "System_Internal_" until the page was refreshed.
• When deleting a Process Definition or related object from its details page, the object would still be visible in the overview with a name prefixed by "System_Internal_" until the page was refreshed.
• When deleting a Process Definition or related object, the object would still temporarily be visible using the "show hidden objects" filter.

After: This is all no longer the case, the ClassCastException is no longer shown, objects get removed from the overview and wont show when using the "show hidden objects" filter.

Before: A "Duplicate Key" exception could occur when multiple UserMessages were created based on the same UserMessage definition.

After: This has been fixed.

Before: Configuration options: UI.HelpVersion and UI.ProductInfoDoc were mandatory, not setting them caused logged ERRORS during installation or build if not set.

After: Made UI.HelpVersion and UI.ProductInfoDoc optional in redwood/scheduler/ui/src/xml/configurationoptions.xml config file

Mandatory fields with are now marked with * for Job Chain Editor Raise/Wait Events.

Before: When starting PeopleSoft jobs that create child jobs a NullPointerException could occur with the message that child jobs can't be created. The childjob definition had to be imported before a parent job with these childjobs could be used. When the job definitions are not imported we are searching for a default job definition "System_PeopleSoft_MonitorJob". The search for this job definition caused a NullPointer Exception.

After: The "System_PeopleSoft_MonitorJob" is searched in the correct Application.

Before: SAP jobs with variant names with leading or trailing blanks could not be started.

After: SAP jobs with variant names with leading or trailing blanks can be started.

Before: SAP transactions spool in XLSX format is corrupt when retrieving from a cloud instance.

After: CSV file will be generated when retrieving spool in XLSX format. It is converted to XLSX format in Finance Automation.

Before: The Novum theme was missing some images that caused buttons not be be clickable

After: All images are now provided, making the buttons visible and clickable again.

Before: Killing a batch SAP job did not kill its child jobs

After: Killing a batch SAP job kills its child jobs, when /configuration/sap/xbp/EnableKillXBPChildJobsWithParent = true

Before: A SOAPRequest call called within RedwoodScript could only be used to produce a SOAP call. However previously it could also be used to produce a HTTP call.

After: Only when the appropriate values are set  (e.g. SOAPAction, Accept etc.) these values are set on the SOAPRequest. It means that when not set it can be used as a HTTP request.

Before: Transport requests via CTS+ were always created with RFC user as owner

After: Transport request via CTS+ can be created for the other user with new job definition parameter SAP user.

Before: Managing PI channels was not possible due to the jobs failing with an XML parse exception.

After: It is possible to manage PI channels.

Before: When running System_ReportRun on a Report process definition that did not have a limit defined, it would fail with a NullPointerException, and if a limit had been defined, it would be used, even if it exceeded the system wide report limit of 500,000 rows.

After: Whenever a Report process definition is run, it will be limited to 500,000 rows, whether run directly or via System_ReportRun. If run via System_ReportRun, then it will not fail with a NullPointerException, even if the limit is not set.

Before: When importing a CCMS jobs, the recipient type on the generated process definition has been always set as Internet User.

After: When importing a CCMS job, recipient type on the generated process definition is set to the actual recipient type of the SAP job.

Before: Second SAP spool could not be retrieved when process definition default output format is xls

After: Second SAP spool can be retrieved properly when process definition default output format is xls

Before: When sending mails, and the server supports TLS upgrade, the TLS upgrade would fail, causing the mail to not be sent, and the mail process to error.

After: Mail can now be sent when the mail server supports TLS upgrades.

Before: Under some circumstances when logging in during a system upgrade, the upgrade would fail with a NoRowsUpdatedException, requiring the system to be restarted.

After: Changes have been made to retry if a NoRowsUpdatedException occurs during the upgrade, this means that the startup may take a little longer, but the system won't fail to startup.

Before: Specifying the CC or BCC headers in a mail definition would not send the mail to these addresses.

After: Setting these headers works as expected.

Before: Job count in joblog.log and joblog.dat could be null.

After: Job count in joblog will be set properly.

Before: SAP BOBj jobs may go into error when the connection breaks
After: SAP BOBj jobs don't go into error when the connection breaks

Update moment.js, a JavaScript library, to version 2.29.3 to overcome CVE-2022-24785. This update protects against unauthorised file access (a "path traversal" vulnerability).

Before: The version of Jackson shipped used within the product has a HIGH ranked CVE reported against it: CVE-2020-36518.

After: The Jackson libraries have been updated to a version not affected by this CVE.

NOTE: The product is not affected by this CVE, as the component in the library that caused the CVE to be raised is not used in the product. However, this has been updated out of an abundance of caution.

Before: the library okhttp3/okhttp v4.9.1 contains a security vulnerability CVE-2021-0341. This library is used by our mailservice.

After: the library okhttp3/okhttp is upgraded to v4.9.3. to resolve the CVE.

Before: At several places the system creates temporary files, which are cleaned up when their use is completed successfully. However, if the JVM is shutdown while the file is still in use it won't be cleaned up.

After: Temporary files are now always created in the same folder and this folder is cleaned at start up. So, if for some reason a file was not deleted, it will always be deleted at the next start up of the system.

Added OAuth 2.0 authentication support for IMAP and IMAPS for use with Microsoft Office 365 to the Mail Connector.

Configuration:

• Settings for Microsoft Azure Active Directory
  • App registration
  • Public client flow allowed for App authentication
  • Delegated Microsoft Graph API permissions
    • IMAP.AccessAsUser.All
    • offline_access
    • Granted consent for all API permissions

• Additional Connection Settings for MailConnector
  • redwood.connection.oauth.provider
    • The OAuth 2.0 token provider. Currently only "outlook" is supported.
  • redwood.connection.oauth.client
    • The Application (client) ID of the app registered in Microsoft Azure Active Directory.
  • redwood.connection.oauth.tenant
    • The Directory (tenant) ID of the app registered in Microsoft Azure Active Directory.
  • redwood.connection.oauth.authority (optional)
    • The authority URL when using a custom Exchange server. This setting is optional. When not set the default URL https://login.microsoftonline.com/ is used.

Support for the Microsoft Graph REST API added to the Mail Connector.

Configuration:

• Settings for Microsoft Azure Active Directory
  • App registration
  • Public client flow allowed for App authentication
  • Delegated Microsoft Graph API permissions
    • Mail.ReadWrite
    • Granted consent for the API permission
• Settings for the Mail Connector:
  • Protocol: MS Graph
  • E-mail server name or IP: https://graph.microsoft.com/v1.0 (Graph endpoint)
• Connection Settings for the MailConnector
  • redwood.connection.oauth.client
    • The Application (client) ID of the app registered in Microsoft Azure Active Directory.
  • redwood.connection.oauth.tenant
    • The Directory (tenant) ID of the app registered in Microsoft Azure Active Directory.
  • redwood.connection.oauth.authority (optional)
    • A custom authority URL. This setting is optional. When not set the default URL https://login.microsoftonline.com/ is used.

Before: Unix process definitions were unable to return job status codes greater than 256, the reason being that on Unix systems the exit value is always modulo 256.
So for example, exit 513 will result in an exit value of 1.

After: A 'jtool setreturncode' mode has been added.
In situations where the customer wishes to return error codes larger than 255, we recommend the use of jsetreturncode.

Before: jtool scp could fail on sending or retrieval of (very) big files caused by the process accessing invalid memory. This was caused by a single byte buffer overrun.

After: The buffer overrun is avoided and any size files can be transferred using jtool scp.

Before: jtool mail would generate an error message about a double free operation and an error returncode when the -secretfile argument was used. The mail was sent correctly.

After: The memory allocation mixup is fixed and no error is produced.

Before: When transferring files to a server via jtool ftp/ jftp put -ascii the transfer could abort and the program crash when the file content contained a \r character at the exact end of a buffer with a length that would be re-allocated to a different address by the memory allocator (which is OS-dependent.) This was observed on linux-x86_64 but can theoretically happen on any platform.

After: The reallocation is properly accounted for and the transfer no longer crashes.

Before: Some messages created by the Platform Agent were not translated in the log file.

After: Messages that come from the Platform Agent are translated when writing to the logfile.

Before: When many (> 100) file events are defined it is possible that a race condition between two threads in the platform agent results in a temporary loss of detection of some of these events. Restarting the agent or process server resets this error condition.

After: A change in file events always results in detection of all remaining and new file events.

Before: Navigating to a valid Agent Initiated URL from a browser will shutdown the associated Process Server.

After: Navigating to a valid Agent Initiated URL from a browser will result in an HTTP 403 Forbidden error being returned and will have no effect on the Process Server.

Before: jsecret offered an optional isolationGroup argument.

The Platform Agent and some jtool modes, specifically jscript and jevent, mandated the presence of an isolationGroup line within a supplied connection file.

After: jsecret no longer offers an optional isolationGroup argument. However it will still always include isolationGroup=GLOBAL in a generated connection for backwards compatibility.

The Platform Agent and jtool no longer require the presence of an isolationGroup line within a supplied connection file.

Before: SQLPlus connections could only be made with username and password (via a Credential).
 
After: SQLPLUS Connections can now make use of Oracle Wallets, either by using /@MYENDPOINT in the 'Remote Run as User' field or by only setting the JCS_REMOTE_ENDPOINT parameter to MYENDPOINT, while leaving the JCS_REMOTE_USER and JCS_REMOTE_PASSWORD fields untouched.

Before: jftp crashed with a segmentation violation when hostname was missing in the connection file.
After: jtfp returns an error message when the hostname is missing from the connection file.

Before: Windows RDP session could not be created.

After: This regression has been fixed.

Before: The Platform Agent treated all HTTP 30x redirections as temporary redirections.

After: The Platform Agent treats successful HTTP 301 and 308 redirections as permanent redirections, so agent_initiated_url is updated following a successful permanent redirection request.

Before: Retrieving support files from a platform agent in logLevel=trace could cause the logs to fill up will large parts of base64-encoded data.
After: The base64-encoded parts are not entirely logged anymore in logLevel=trace.

Before: Sporadically the jtool getfile tool would be unable to retrieve a file with compression, making it necessary to add the -nocompression flag. The error message given was:

code lengths don't add up (8000)

This happens when a compression block has very specific content with no repetitions.

After: The transfer always succeeds with compression enabled.

Before: When a File Event Definition is modified on the server, all File Event definitions were sent to the Platform Agent, and they were logged as being "modified" even if they might not have changed.

After: When a File Event Definition is modified on the server, only that definition is sent to the Platform Agent, reducing the amount of reconfiguration that needs to be done. When a platform agent restarts and/or reconnects, the full list of events will still be sent. Additionally, only definitions that are actually modified are logged as being "modified".

Before: A resource problem on the platform agent host could result in a large amount of error messages and Operator Messages like "retrying failed fork: Resource temporarily unavailable".

After: Retry attempts due to resource problems will generate warnings and no Operator Messages anymore. When the maximum number of retry attempts is exceeded, an error will be raised and displayed in the Operator Messages.

Before: On Windows, when initial startup and configuration of the platform agent is very fast, the agent could run into an error during switch-over to a new log file where the name of the newly generated log file is identical to that of the initial log file. The following error message would then be reported: "The process cannot access the file because it is being used by another process."
After: The newly generated log file name is verified to be unique.

Before: Starting multiple platform-agent instances with 'scheduler start' using the 'ksh93' shell, could result in an infinite starting loop, raising errors like:
"A process still exists for instance 'test', please verify that it is stopped and remove its pid file '/opt/redwood/agent/etc/pid/test/test.ppid' before starting it again."

After: The instances start normally.

Before: A platform installation path, ending with a slash character '/' resulted in a duplicate PATHS entry in the scheduler script after a version upgrade. This could result in a repetitive startup sequence for platform agents.

After: A trailing slash character in the existing PATHS setting is ignored for existing installations.
For new platform agent installations, a possible trailing slash character is removed from the destination path argument during the installation.

Before: If Platform Agent trace logging was enabled and the log file became large so it needed switching to a new log file then the Platform Agent was likely to crash.

After: When Platform Agent trace logging is enabled and the log file needs switching over, then the Platform Agent performs normally.

Before: Timeout values on platform-agent for Sun Solaris, set for SSL read/write timeout, could end up as incorrect very large numbers. Setting Loglevel=trace would show these incorrect values as e.g.:

TRACE 2022-03-22 23:34:18,530 CET 14269-getmessages #3 opsys.socket - fd=8/10.31.2.179:36525->10.31.2.137:443: timeout set to 21474836480s

TRACE 2022-03-22 23:34:18,530 CET 14269-getmessages #3 opsys.socket - fd=8/10.31.2.179:36525->10.31.2.137:443:: waiting for write, timeout in 21474836480000ms

After: The timeout values are correctly set. E.g.: timeout set to 5s

Before: Platform agent included OpenSSL 1.1.1l.

After: Platform agent now includes OpenSSL 1.1.1n.

The following CVE fixes are included in 1.1.1n.

CVE-2022-0778: Infinite loop in BN_mod_sqrt() reachable when parsing certificates [High severity]

Note: This issue does not affect the scheduler, however the library is being upgraded out of caution. (We do not accept "user input" certificates).

Before: jrfc did not correctly handle invalid UTF (Unicode transformation format) sequences.
In the case of an invalid UTF sequence received by jrfc, we would see hundreds of 'Invalid byte xx at y bytes into UTF8 sequence' for the same offset (ie looping) that will then exceed our internal buffer sizes, because later in the logs when the output is processed we see hundreds of 'StringBuffer maximum capacity (> 512Mb)' exceeded FATAL log messages.

After: The UTF code used by jrfc, and by the Platform Agent in general, now correctly handles the detection of an invalid UTF sequence and no longer loops.

Before: Wild-carded file events could generate a flood of Operator Messages when they encounter files that could not be moved and files that are still in use and therefore cannot be moved yet.
After: Wild-carded file events will generate a single Operator Message at each scan interval for every file encountered that could not be moved.

Before: On Windows, when initial startup and configuration of the platform agent is very fast, the agent could run into an error during switch-over to a new log file where the name of the newly generated log file is identical to that of the initial log file. The following error message would then be reported: "The process cannot access the file because it is being used by another process."
After: The newly generated log file name is verified to be unique. This is fixing an earlier attempt to make this robust, which did not do the job very well.

Before: If a no_proxy file or environment variable NO_PROXY was configured that did not include localhost, then the Platform Agent was unable to run jobs.

After: The Platform Agent can successfully run jobs even when a no_proxy file or NO_PROXY environment variable failed to include localhost.

Before: If a batch of files for a file event contained several files that could not be moved, then the back-off time was doubled for each file found in error. rather than just once for the whole batch. Or if a batch of files contained errors, but the last file processed was successful then backing off was stopped, causing looping Operator Messages.

After: A batch of files for a file event is now handled in a consistent fashion, preventing looping Operator Messages or alternatively backing off that is too aggressive.

Before: Connecting with User SYS with SQLPLUS command fails on windows. The connection creates incorrect connection string syntax.

After: The connection string is now created correctly (with "as sysdba" at the end of the connection string)

Before: Platform agent included OpenSSL 1.1.1n.

After: Platform agent now includes OpenSSL 1.1.1o.

The following CVEs are fixed in OpenSSL 1.1.1o:

1. CVE-2022-1473 - Low severity - OPENSSL_LH_flush() function has a slow memory leak if it decodes lots of certs.
2. CVE-2022-1434 - Low severity - Only affects the OpenSSL 3.0 implementation of the RC4-MD5 cipher suite.
3. CVE-2022-1343 - Moderate severity - OCSP_basic_verify issue when the OCSP_NOCHECKS flag is used.
4. CVE-2022-1292 - Moderate severity - The c_rehash script does not properly sanitise shell metacharacters to prevent command injection.

Note: These issues do not affect the scheduler, however the library is being upgraded out of caution.

Before: If a Windows Platform Agent encountered a non-transient WMI error, such as WBEM_E_ACCESS_DENIED, whilst monitoring an agentless job,, then the WMI operation would be repeatedly attempted with no limit. This would result in a very large number of job log messages for the Redwood Platform Server.

After: If a Windows Platform Agent encounters a WMI error whilst monitoring an agentless job, then the WMI operation is retried a limited number of times, in case the condition is transitory (such as WBEM_E_SERVER_TOO_BUSY). If the WMI error continues to occur, then the job fails and a small number of Operator Messages will be generated.

The BObj connector has been hardened against intermittent network failures.

Before: jrfc jobs in Windows could fail due to loading of incompatible SAP nwrfc libraries that were found earlier in the PATH setting than the agent's own 'saplibs' directory.

After: The sequence to search the PATH for SAP nwrfc libraries has changed in favor of the platform-agent's own 'saplibs' directory.

Before: Importing SAP calendars did not update TimeWindows referencing these calendars if the SAP system was in any Partition other than GLOBAL.

After: Importing SAP calendars does update TimeWIndows referencing these calendars regardless of the partition of the SAP system. SAP systems in GLOBAL partition must not be prefixed with the partition name when used in period functions of TimeWindows.

Before: SAP BObj process server stayed in partial running few seconds after shutting down. 

After: SAP BObj process server can be shut down directly. 

Before: SAP BObj connector created operator message every minute when connection is broken.

After: Operator message will be created every 1 hour. 

Before: In rare circumstances after a network outage SAP jobs may fail due to a timing issue.

After: The timing issue is now handled properly.

Before: Spools of SAP jobs could not be retrieved as HTML format.

After: Spools of SAP jobs can be retrieved as HTML format with the function module BAPI_XBP_GET_SPOOL_AS_HTML.

Before: SAP_ImportCcmsJobs could not import multi-steps SAP job as single definition without template definition.

After: SAP_ImportCcmsJobs can import multi-steps SAP job as single definition without template definition.

Before: There was a small time window during the update process of an old platform-agent (containing SAP nwrfc libraries) to a new platform-agent, not containing the SAP nwrfc libraries anymore, in which an already pending SAP spoolhost 'get' action using jrfc was told to continue while the agent update was still searching and copying the SAP libraries from the old agent location to the new location.

After: A pending jrfc job waits until the platform-agent update process has finished to make sure that possibly existing SAP libraries in the old agent are copied to their new location.

Before: If JOBNAME parameter is not set, job name in joblog.log and joblog.dat is null.

After: Job name in joblog will be set properly.

Before: Using standard SAP BAPI variant function module, >!< didn't work as expected. The parameter value would always be overwritten.

After: Additional parameters in BAPI function module have been added  and the usage of >!< and ! was recovered.

Before: There was a timing issue in XBP job control rules which could cause the product to create operator messages for synchronized SAP jobs.

After: The timing issue in XBP job control rules has been fixed.

Before: When failing to connect to an SAP process server, the sender object of the corresponding operator message is not set.

After: The sender object is set properly.

A rare issue has been fixed which could cause a SAP job to get stuck in Assigned status if the central server was restarted while the SAP connector was unassigning that job.

Before: SAP DataService job and SAP BOBj job could stay in status Running forever despite errors reported by the API.

After: SAP DataService job and SAP BOBj job will set to status Error when the API reports an error. Communication errors (eg. interrupted connection to the SAP system) will be treated as transient, though, keeping the job in status Running until they are resolved.

Before: Incorrect message has been written into the job log file when the template job definition was not set. 

After: The incorrect message has been removed from the job log file.

Before: joblog.dat could not be retrieved when running in the cloud.

After: joblog.dat can be retrieved when running in the cloud.

Before: SAP connector created joblog.dat every time when it was instructed to retrieve joblog.

After: joblog.dat will be created only if job parameter FL_SHOWLOG is set.

Before: SAP mass activity processes may terminate prematurely before their sub-processes are finished in SAP.

After: SAP mass activity processes wait until their sub-processes are finished in SAP before terminating.

Before: If LANGUAGE parameter is not set in the job definition, multistep SAP jobs always use English as step language, otherwise it will use the the provided value

After: If LANGUAGE parameter is not set in the job definition, multistep SAP jobs will use SAP system default language as step language, otherwise it will use the the provided value

Before: BOBj job could go into status Error due to comminication errors with the SAP BOBj system but completed in SAP BOBj.

After: Communication errors (eg. interrupted connection to the SAP BOBj system) will be treated as transient, though, keeping the job in status Running until they are resolved.

Before: Cannot retrieve spool output in HTML format, and large spool output in PDF format via the Spool Host.

After: Large spool output in PDF format can be retrieved via the Spool Host for RMJ and RMF, spool output in HTML can be retrieved via the Spool Host for RMF.

Before: Cannot retrieve SAP joblog in pipe delimited format via the Spool Host when FL_SPOOL is set.

After: SAP joblog can now be retrieved in pipe delimited format via the Spool Host.

Before: If a BW process is skipped in SAP, the remote status of the corresponding process on the central server would not be updated to reflect this.

After: If a BW process is skipped in SAP, the remote status of the corresponding process on the central server will be updated to reflect this.

Before: When downloading large files no other AS400 jobs were monitored. Also when replying to an operator message on the AS400, the process in the scheduler was not updated.

After: Finalizing AS400 jobs is done asynchronously, so downloading large log files doesn't prevent other AS400 jobs from being monitored. When replying to an operator message on the AS400, the process in the scheduler is updated to reflect the correct status.

Before: When running in a clustered environment the slave node can consume a lot of CPU even when it is not busy. The Action Component is stuck in a busy loop waiting to become the master node (if necessary).

After: The Action Component will now correctly idle when it is not the master and will not consume CPU needlessly.

Before: Any user could publish any process definition as a web service.

After: In order to publish a process definition as a web service, the user must have the Configure_Published_WebServices global privilege. This is granted by default to the administrator user.

Note: This allowed a process definition to be published, however users could still not submit this definition via web services unless they had permissions to submit it locally. This means that the security impact of this was low.

Before: If a process is deleted before a process definition alert with a delay was begun, then ERROR messages would be logged, despite these being normal occurrences.

After: These messages are now logged at DEBUG level.

Before: If a user had the global privilege Support_Files_Get, they could request server log files with the GetSupportFiles functionality.

After: The global privilege Support_Files_Get is still needed to get the support files, but to include the server log files a user needs the global privilege System_Support as well. Additionally, the global privilege System_Dynamic_Trace is granted to the redwood-administrator role now by default (for cloud customers).

Before: You could set the 'template' flag on a process or chain definition, but it was impossible to remove this flag later.

After: Now it is possible to remove a 'template' flag from a process definition that has been marked as template.

Before: A Classcast exception was thrown when using a ProcessServer check when raising adhoc alerts.

After: This has been fixed.

Before: Performing complex Object Search queries could result in high memory usage on the server.

After: The memory usage has been improved to release unused memory more often when performing an Object Search.

Before: Running jobs that make use of the RTXWriter that contained RTX without column descriptions on a platform agent would result in a NullPointerException and the job consequently failing.

After: Running jobs that make use of the RTXWriter that contains RTX without column descriptions on a platform agent proceed to completion successfully.

Note, the DataTransfomer extension point uses RTXWriter, and needs to be upgraded to get this fix.

Before: When accessing the server from an initial request originating from outside of the server, an error could be produced in the browser about too many redirects.

After: This issue has been fixed.

Before: Upon each start of redwood-platform lines like the following are logged in scheduler.log: 

logs/scheduler.log:INFO 2021-07-06 22:20:58,088 GMT [Redwood Background Startup] com.redwood.scheduler.lifecycle.impl.ConfigurationComponent - Fixing JCS_REGISTRY0 entry name=CONFIGURATION፨␟ uniqueid=1
logs/scheduler.log:INFO 2021-07-06 22:20:58,088 GMT [Redwood Background Startup] com.redwood.scheduler.lifecycle.impl.ConfigurationComponent - Fixing JCS_REGISTRY0 entry name=SYSTEM፨␟ uniqueid=11429
logs/scheduler.log:INFO 2021-07-06 22:20:58,089 GMT [Redwood Background Startup] com.redwood.scheduler.lifecycle.impl.ConfigurationComponent - Fixing JCS_REGISTRY0 entry name=USER፨␟ uniqueid=5559

After: These are no longer logged, this also minorly improves startup speed, as these actions are no longer executed unnecessarily.

Before:

1. There was no limit on the number of restarts for a process, chain, step or call.
2. If a job had unlimited restarts, the default delay would be applied.

After:

1. The default restart limit and default delay will only be applied if the user does not specify anything. That is, if the user specifies a set number of restarts and a delay, that is what will be applied.
2. If no restart limit is specified, then we will use the default restart limit.
3. If the restart limit (default or specified) is greater than or equal to the default, and no restart delay is set, the default restart delay will be applied.

Before: The Default Expression REL editor was not displaying the method Logic.if and Logic.case for use when editing the REL Expression.

After: These methods are correctly displayed and available for use in the editor.

Before: Right clicking on the Monitoring Dashboard and Housekeeping Dashboard gave access to the browser's context menu.

After: Right clicking on an item in one of the dashboards never gives access to the browser's context menu anymore; if the item has its own context menu this menu is displayed, otherwise nothing happens.

Before: An internal map of the logging infrastructure was keeping more information than needed. This map was cleaned in System_ProcessKeepClauses, but when this job did not run or not often enough, the map could grow too large even causing an OOM.

After: The internal map is automatically cleaned up (and much sooner). The chance that the map grows too large is now minimal.

Before: It was possible, if two people tried to access the support servlet at exactly the same time, for a user to be able to access the support servlet show page, or conversely for a user who should be able to see the page to not be able to see it.

After: This has been fixed so that the correct user is always checked when accessing the support servlet.

Note: The impact of this is very limited, as all actions that can be performed from the page perform the correct checks, so it isn't possible to do anything that you should not be able to, except to see the security servlet page.

Before: RTXWriter could fail if null values were attempted to be written for RTX elements on platform agents.

After: RTXWriter omits empty optional elements and will not attempt to write content for  empty mandatory elements.

Before: Calling the function Time.now("Invalid timezone") would not fail with an error message; the function would use the system time zone instead.

After: Calling the function Time.now("Invalid timezone") with an invalid time zone will now cause the function to fail with an exception.

Before: If the database connection cannot be retrieved, recover was tried for two minutes, after this the transaction would fail. Depending on where this happened, this could result in processes getting stuck in unexpected statuses.

After: When the database disappears, recovery is attempted indefinitely or until a recovery attempt fails for non-connection based issues.

Before: Connections from the Platform Agent could produce warning statements in the tomcat.log file mentioning that org.xml.sax.ErrorHandler has not been set.

After: These messages will not longer be produced.

Before: If the database could not be contacted, then clustering would log every 100ms an error and full stack trace.

After: If the database cannot be contacted, then a stack trace will be logged on the first instance, and thereafter only a message every second.

Before: On Windows compiling a RedwoodScript definition with a Library could fail as the packages defined in the library were not found. This was due to issues resolving the class path used for compiling, including failing to handle special characters in file names. 

After: The issue has been resolved.

Before: Installing audit-rules-app-production.car gave an error about the following two files in the car: Production_GMQuery_DiffOnly.xml and Production_GMRepository_DiffOnly.xml.

After: The files have been removed from the car, so installing them works again.

Before: When creating an AuditRule you could enter the fields ReasonRequired and ReasonRequiredRegEx, but these were not used correctly.

After: The fields have been marked as deprecated to indicate that they should not be used.

Before: In the user settings it was possible to configure a proxy user (out of office), which allowed to delegate user messages to this proxy user.

After: Now it's also possible to specify a start date and end date for these out of office settings.

Before: When running on Oracle RAC, the Process Server System_Oracle could stop working, when it was configured incorrectly. The Process Server could not be completely stopped nor started anymore, except by restarting the product completely.

After: When the OracleConnectString is misconfigured, then the System_Oracle process server will just not start.

Before: When using System_RemoveOldJobs, steps of chain definitions, marked as "keep force", are not deleted.

After: When using System_RemoveOldJobs, steps of any deleted chain definitions will be deleted. Any calls from within the chain that have been marked as "keep force" will not be deleted.

Before: Under some circumstances RedwoodScript could not be executed, leading to a NoClassDefFoundError.

After: RedwoodScript is now properly executed under all circumstances.

Before: Empty optional RTX elements were written to the RTX output.

After: Empty optional RTX elements are omitted from RTX output.

Before: Under certain circumstances objects were being retained in memory where they were not needed where short lived user sessions were being used.

After: These objects are no longer retained, reducing retained memory usage.

Before: A potential deadlock could occur running a lot of jobs in parallel, and calling one of SchedulerSession.waitForJob(), SchedulerSession.waitForJobs(), or SchedulerSession.waitForAllChildren().

After: This has been fixed.

Before: When configuring the system with the Admin server, if an invalid database connection was entered initially, later initialisation steps may fail.

After: The invalid database connection details will not interfere with the requirements for the initialisation steps, and installation can continue.

Before: On databases that have case sensitive table names and fields, streaming Documents and upgrading old RedwoodScript definitions may fail to retrieve the data due to unknown table/field exceptions being thrown.

After: The case used for the queries is consistent with the table creation and will now correctly find the tables and fields in case sensitive database.

Before: Mail_Attachments specified on UserMessageJobs (Workflows) weren't forwarded as Mail attachments.

After: Mail_Attachments specified on UserMessageJobs (Workflows) are now forwarded as Mail attachments.

Before: An exception is thrown when the JSON output contains JSONArray's.

After: All JSON elements can be parsed.

Before: reading a job file residing on a platform agent would not close the socket immediately, but only after the Java garbage collector ran. Opening many (thousands) of files in rapid succession could result in starvation (out of open files or sockets).

After: reading a job file that resides on a platform agent will now immediately close the socket in such a way that there is no remaining open socket (not even in CLOSE_WAIT or TIME_WAIT) on client (java server) or server (platform agent).

Before: There were some missing database indexes that, in some situations, caused the system to slow down.

After: Extra indexes have been added to help in this situation

Before:
$ character is not allowed in AS400 JobName. Only getting the logfile from an AS400 Job was not possible.
It was not possible to retrieve only the logfile for a job.

After:
$ character is allowed in the JobName.
Retrieving only the logfile is now possible by setting two parameters on the job:

• OutputFileRetrieval=false
• IndependentQPJOBLOG=true

It is also possible to configure this for all jobs, by setting the corresponding ProcessServer parameters:

• As400JobOutputFileRetrieval
• As400JobIndependentQPJOBLOG

Before: A missing Content-Type header could cause the browser to not process the SSO login request properly when using proxy servers.

After: This has been fixed.

Before: Under certain circumstances local REL resolving would fail when being part of a job chain step.

After: This has been fixed.

Before: RedwoodScript Scripts using waitForJobs API functions, with infinite wait, stop waiting after one hour unless the process finishes before then.

After: Using waitForJobs with infinite wait will wait until the process has finished.

NOTE: This regression was introduced in 9.2.4.9, 9.2.6.6, and 9.2.8.1.

Before: When there is no route to the process server when the process server is started, you may still get an error and the PS goes to shutdown.

After: When starting the ProcessServer the AS400 service, if there is no route to the proces server, it will continue to try and connect to the AS400 system until it is available.

Before: Under certain specific timings, it was possible that a Process Chain could run future steps before they should get started. This chance is higher if the chain has preconditions and/or final status handlers.

After: The Process Chain runs correctly in these situations.

Before: A fix was added to 9.2.8.9 that allowed an AS400 to retry connection if the system could not connect. This retried the connection immediately, potentially leading to a high CPU usage while waiting for the AS400 system to become available.

After: The system now waits for a second between retry attempts.

Before: If you have a very small interval for file events (less then 2 seconds) it could be that when deleting a file event definition a NullPointerException was thrown in the AS400 event monitor.

After: The AS400 event monitor now holds all information that is needed for an event monitoring. After the monitoring the event can be removed without problems.

Before: After a restart of the AS400 ProcessServer jobs in status "Console" were not monitored anymore. The jobs must be changed to a final status manually.

After: Jobs in status "Console" will be picked up after a restart and checked for status changes.

Before: When the AS400 starts up in the cloud and the Secure Gateway is not up yet a lot of operator messages are produced with an unknown host exception. 

After: When duplicate messages are created within a certain timeframe (Default 5 minutes) only one message is written.

Added the french language to the AS400 connector.

Before: Slowdown could be observed in systems with a lot of Events.

After: Optimization has been done in queries in the EventComponent, including extra indexes.

Before: In rare circumstances it was possible for two processes to attempt to create a directory at the same time, one of these would fail, potentially causing a process to not be run.

After: The creation of the directory is properly checked, and if two processes attempt to create the same directory at the same time, this will no longer fail.

Before: Using the java.util.logging classes in Redwood Script could result in leaking memory.

After: This has been fixed and logging will no longer leak memory.

Before: api-tools.jar event returned the eventid as the return code which made existing customer scripts fail as this was a change of behaviour.

After: api-tools.jar now returns 0 for success.
The event id is reported as "Successfully raised event with ID 85" for example.
Unless the new -silent argument is specified.
This matches the behaviour of jtool event.

Before: When the scheduler has a large number of queues and/or process servers, greater than about 100 of each, then startup could take a long time.

After: The process has been optimised to ensure that startup occurs quickly, regardless of the number of active process servers or queues.

Before: The Platinum Jubilee Bank Holiday would not happen on the date of 3 June 2022. The Spring Bank Holiday would happen on the date of 30 May 2022

After: The Platinum Jubilee Bank Holiday was added to happen once on the date of 3 June 2022. The Spring Bank Holiday was moved only in 2022 from 30 May 2022 to 2 June 2022

Before: The following maintenance jobs all started at the fixed times based on a whole hour. These times seem to be the times that most of the jobs are scheduled, causing peaks in the job schedules:

• System_UpdateJobStatistics
• System_ProcessKeepClauses
• System_DeleteJobFiles
• System_Ignored_Alert_Reporter
• System_Aggregate_History

After: The maintenance jobs above are now scheduled on random times, but with the same interval as before, reducing the peaks.

The randomisations happens only once, and happens the first time that the system starts up after installing this version.

The randomisations of the above mentioned maintenance jobs happens only if they have the default 15 minutes set as time interval in between executions or if no time interval is set.

Upgrades the version of the PostgreSQL JDBC driver used in RunMyJobs and other products to 42.4.0, this fixes some minor issues in the driver.

Upgrades the version of the fasterxml Jackson products used in all products to 2.13.2, to overcome CVE-2020-36518 which could potentially allow a denial of service by uploading specially crafted JSON documents.

Drag-to-copy feature of JobChain Chain editor now works on Mac.

Before: The ckeditor would not display if the user's default browser language was not English. Also, the file upload field was displayed but not populated.

After: The ckeditor will now always use English as a default language. The file upload field is now hidden and only displays the upload button.

Before: Right clicking a process in the Process monitor offered the option to get the support files for this process. 

After: This option has been replaced by the option Collect process output.

Before: The screen reader functionality could be enabled by passing in the request parameter sap-accessibility or by assigning the role scheduler-screen-reader.

After: This has been replaced by the flag Subject.ScreenReader (either on a User or a Role); if the current user or one of its roles has the flag set, the screen reader functionality is enabled now. This also made the field LDAPProfile.ScreenReaderRole redundant, so this has been marked as deprecated.

Note: For backwards compatibility the role scheduler-screen-reader and any roles that are used in the field LDAPProfile.ScreenReaderRole are marked as screen reader roles (i.e. the flag Subject.ScreenReader is set to true).

Before: Audit records can have additional info specified, but it was not possible to see this information as a column in the Audit Trail overview.

After: Now it is possible to select the additional info column in the column chooser.

Before: If you entered an invalid value in one of the fields on the Global Configuration screen on the Monitoring Dashboard or Housekeeping Dashboard, you could still click the Save button. It was not clear which value had actually been saved.

After: If you enter an invalid value the Save button is disabled now; you can only save the values if all values are valid.

Before: If you tried to promote to a remote system, an exception was thrown if no remote system was selected or if the user didn't have the required privileges.

After: The exception has been fixed. If the user doesn't have the required privileges an error message is displayed now. If the user hasn't selected a remote system, it is still possible to click 'Ok'. This will be fixed in a future version.

Before: Opening the details of a retention record on the Retention tab returned an error.

After: This has been fixed.

Before: Processes in statuses Assigned and Waiting were not considered active causing them to not be identifiable as such in the Runtime Diagram.

After: Both process statuses Assigned and Waiting now have the active (blue) color. Never now has the attention (yellow) color, as it is not a final state.

Before: In the submit page, some recurrence options were always disabled and could not be used anymore.

After: The recurrence options are now enabled together with the radio group, so can be used again.

Before: On some edit pages, after user interaction, specifically crafted input could be sent to the server in a way that posed a security risk.

After: We changed the way we handle server-client communication in those cases to remove the security risk.

Before: Under some circumstances, if there were a lot of updates, Data Transformer jobs were not able to find their configuration.

After: This has been fixed.

Before: After executing a search (Intellisearch) on an overview, the overview is filtered on the search, a temporary "queryfilter" is created for it and that filter is selected as the current overview filter. However on various Firefox versions, the temporary "queryfilter" is not correctly selected, leaving the previously selected queryfilter set as selected (even though the filtering based on the search does happen).

After: This is now fixed and Firefox behaves in the same way as other browsers.

Upgrades the version of CKEditor4 used in Redwood Platform to 4.18.0 to overcome CVE-2022-24728  and CVE-2022-24729 which potentially allowed users to execute arbitrary javascript by uploading documents with specially crafted HTML, and tricking another user into opening them in the editor.

Before: The version of the Eclipse plugin was not increased, causing Eclipse to ignore the new version.

After: The version has been increased, allowing Eclipse to download the version.

Before: External database authentication extension could become very slow and/or run out of memory with a large amount of users and roles.

After: The external database authentication extension has been rewritten and now reacts fast as well uses minimal server memory.

Before: The generated name of a duplicated Role could be modified but would not be persisted.

After: Role names can be changed during creation, also when duplicating Roles.

Before: Prior to 9.2.1.0, it was possible to configure login page to show a message on the login screen by setting the /configuration/security/title registry entry. This functionality was removed in 9.2.1.0.

After: The functionality to add a message to the login screen has been restored. This can be configured by setting the ui.LoginMessage configuration item, eg by setting the /configuration/jcs/UI/LoginMessage registry entry, or by setting the legacy registry entry /configuration/security/title.

Before: Extension points could hold on to class loaders even if the extension point was no longer used.

After: The class loader is no longer referenced when not in use.

Before: Due to the processing required to enable the "Collect Job Output" menu, accessing processes could be slower if the scheduler has many process servers configured.

After: This menu option is removed, and thus the speed of selecting processes in the overview is improved.

Before: Notes could not be deleted in Workbench.

After: Notes can be deleted in Workbench.

In both HTTP and SOAP definitions, response header values can be mapped to output parameters by specifying a parameter with the header name of direction Out or In/Out.

In SOAP definitions, when the response contains a JSON object, Out or In/Out parameters will be filled with the value of the corresponding field in the JSON of that parameters name. When the field contains a list and the parameter is listed as an array, the elements will be put in individual items of the array.

In both HTTP and SOAP definitions, the 'Out' suffix on output parameters is made optional.

Before: In the core product in-parameters of a notification only inherited the in-value from the corresponding parameter of the user message. In FCA the notification parameter then got set again (and possibly overwritten) by an action of the notification with the out-value if it was available.

After: In the core product in-parameters of a notification now inherit the current value from the corresponding parameter of the user message by using getCurrentValue(). In FCA the notification parameter now does not get set by an action anymore.

Streaming methods have been added to Documents to allow streaming the data from the Documents directly, reducing the memory usage in retrieving and updating the documents.

Added the ability to modify wait and raise events on calls in the chain editor.

Before: There is no option to monitor Redwood Script threads (either from Processes, Triggers, Shell etc).

After: It is now possible via the support servlet to see which Redwood Script threads are running, who started them, how long they've been running and to either show the process associated with it, or to interrupt the thread (in case of shell threads)

Before: The HistoryJobParameter was maintained, however the usages of of this have been removed. Particularly when creating SAP processes, this table could see a lot of inserts and deletes, increasing load on the database.

After: HistoryJobParameter and its references have been removed. This object was never on the external API.

The version of Tomcat that Redwood Platform is based on has been updated to 9.0.46. This includes the fixes for the following security issues released in 9.0.43, and announced later:

• CVE-2021-25329: Fix for CVE-2020-9484 was incomplete
• CVE-2021-25122: Request mix-up with h2c

Note: HTTP/2 is not enabled by default in Redwood Platform, nor are sessions persisted, so the default configuration is not vulnerable to these two issues.

Before: Not all JDBC drivers were up to date.

After: Updated the JDBC drivers to the latest versions. The new versions are:

• db2: 11.5.7.0
• mssql: 9.4.1
• oracle: 21.4.0.0.1
• postgresql: 42.3.1
• jt400: 10.7

Before: If a process definition was modified, and then submitted by multiple threads in parallel, then every thread would attempt to do the work to create the new definition. This could result in a lot of unnecessary work, as only one of these could succeed, and the other threads would then have to reload the data. This is a general issue, but was most often seen in FCA loops, which would create a new thread for every iteration of data, which would all get submitted at the same time.

After: This is resolved by using an internal cache, and if multiple threads are waiting for the same prepare, then they will wait until the first one has completed, to use its result.

The version of Tomcat that Redwood Platform is based on has been updated to 9.0.58. This includes the fixes for the following security issues released in 9.0.58, and announced later:

• CVE-2022-23181: Local Privilege Escalation

Note: The product doesn't use FileStore to store sessions, and is unaffected by the above vulnerability, this is being updated out of an abundance of care.

Before: Following a file event error, the file event was not retried for an hour, then further retry intervals were doubled until a maximum retry period of 1 day is reached.

After: Following a file event error, file event is retried at an interval of 4 times the configured poll interval. Further retry intervals are doubled until a maximum retry period of 1 day is reached.

Before: Platform agent included OpenSSL 1.1.1g.

After: Platform agent now includes OpenSSL 1.1.1k.

The following CVE fixes are included in 1.1.1k.

• CVE-2021-3450 (OpenSSL advisory) [High severity]
• CVE-2021-3449 (OpenSSL advisory) [High severity]
• CVE-2021-23841 (OpenSSL advisory) [Moderate severity]
• CVE-2021-23840 (OpenSSL advisory) [Low severity]
• CVE-2020-1971 (OpenSSL advisory) [High severity]

Of these, only CVE-2021-3449 might have relevance if you have configured your platform agents to serve local content via HTTPS.

Before: Platform agent included OpenSSL 1.1.1k.

After: Platform agent now includes OpenSSL 1.1.1l.

The following CVE fixes are included in 1.1.1l.

• CVE-2021-3712 (OpenSSL advisory) [Moderate severity]
• CVE-2021-3711 (OpenSSL advisory) [High severity]

Neither of these issues affect the scheduler, however the library is being upgraded out of caution.

SAP JCo version 3.1.3 has issues with MSHOST connections and a memory leak in JCoServer. SAP has fixed the issue with SAP JCo 3.1.5. Default SAP JCo version has been set to 3.1.5.

Before: If a job definition was still referenced from a branched chain definition, it could not be deleted.

After: We now mark job definitions as discarded when trying to delete them. Normally discarded job definitions will be deleted by System_ProcessKeepClauses, so the end result will be the same. If for some reason the discarded definition cannot be deleted, it will still be visible as hidden object, but marked as 'discarded'.

Before: CAR files exported from the scheduler included code that allowed it to be run from the command line.

After: Based on usage patterns, this code has been removed from the exported CAR files, making the exports smaller.

Before: The Promotion interface has the option to add a comment to the promotion action. However when using a custom Export using the Exporter interface it is not possible to set the comment in the generated car file.

After: The setComment method has been exposed to the Exporter api interface. This allows for setting the comment in the export-info.xml file in the generated car file from an Exporter.

Before: Only Process Definitions of type RedwoodScript provided support for local REL expressions e.g. for the default expressions of parameters.

After: All Process Definition types support local REL expressions now. The additional optional field RELRelatedLibrary was introduced in the Process Definition entities for this purpose. If this library is specified then it is used to lookup the available REL entry points that can be used without specifying Partition and Library prefix directly in the REL expression. For backward compatibility reasons the old behaviour of RedwoodScript definitions is still supported, i.e. if the RELRelatedLibrary is not specified the REL entry points of the Library are retrieved that is assigned to the Script object as it was done before the change.

Before: When finalising or starting a new process in process group, a query was made to the database to check for other processes in the process group.

After: The product has been modified so that this check is no longer required, resulting in a minor performance improvement in the general case. In some cases, this query has been seen to interact badly with System_ProcessKeepClauses, in these cases the performance of System_ProcessKeepClauses has been significantly improved.

Before: If a customer got a "growth alert" (an alert about a table that was growing fast), it was hard to identify the cause of the problem.

After: System_MonitorTables now creates a separate csv file with the table growth since the last run of the job. If an alert is sent now, the output of the previous runs of the job can be collected to analyse what happened and when.

Reduced the maximum presubmit count from 5000 to 50, reducing the number of jobs that can be submitted for an indefinitely scheduled job group to 50 .

Before: If a job was killed or threw an exception we saw two log messages and identical exception traces appear in scheduler.log

After: If a job was killed or threw an exception it now only yields a single log message and exception traces in scheduler.log

Before: Etcd was not supported by the scheduler configuration framework.

After: An etcd backend has been added to the configuration framework for storing configuration values.

Before: When a car file is imported through the REST api or promotion, the import is not visible in the Promotion->Imports overview.

After: Car files imported through Promotion are also visible in the Imports overview.

Before: Upgrading can be blocked by RedwoodScript definitions if the API has an incompatible change, this normally requires commenting out or removing the definitions that cause the problem.

After: When importing a new flag can be set 'Invalidate on Error' that will mark the definitions that fail to compile as Invalid, rather than failing the import. Invalid definitions cannot be used until they are fixed, but the import can continue.

Upgraded the ckeditor to the newest version, to address a vulnerability issue that exists in the previous version.

Before: In some circumstances it was possible to pass a specially crafted XML document to the SOAP API that would allow you to read any file on the system that the scheduler has access to.

After: This is resolved.

Before: The monitor tree is enabled which can lead to a negative impact on performance when many updates are done.

After: The monitor functionality is disabled by default.

NOTE: To enable the monitor function there are two changes required:

1. Set the registry entry /configuration/jcs/monitoring/enabled to true to enable the monitoring tree functionality. The system must be restarted after this change is made for it to take effect.
2. To enable the monitor functionality for the platform agents you need to set on the process server that you want monitoring the process server parameter MonitorInterval to a value larger than 0. The previous default was 60.

Before: The XAL and XMW tabs for SAP systems was shown in the UI. These options have been deprecated, and should not be used anymore.

After: The XAL and XMW tabs will only be shown if they were already used.

Before: User login audit trails would show the (reverse) proxy ip address for clients.

After: An attempt is made to find the original client ip address on the request to store in the audit trail.

Note:** this change has introduced a backwards imcompatible change on the AuditSubjectLogin API where the setters have been removed. Although chances are low this is used in the field, be aware of this change.

Before: AutoSuggest control in Cronacle UI did not appropriately escape HTML characters, meaning that specially crafted names could potentially result in a cross site scripting attack vector.

After: HTML encoding is provided to autoSuggest control in Cronacle UI.

Cloud only

Before: The default glibc memory allocator in Linux interacts very badly with Java, causing significant memory overallocation. In our cloud environment, this then caused environments to use too much memory, which resulted in them getting erroneously killed by the Linux Out Of Memory Killer (OOMK).

After: The memory allocator has been changed such that memory usage is much more consistent over time and load. Internal testing, which was previously able to reliably cause Linux to kill the environment with its OOMK, now passes and the environments do not see these issues, nor do they see memory increases over time.

Note: For on-premise customers, the memory allocator that we are are using is tcmalloc.

The Tomcat version has been increased to 9.0.52. This resolves CVE-2021-33037 which allowed for HTTP request smuggling.

The version of Tomcat that Redwood Platform is based on has been updated to 9.0.54. This includes the fixes for the following security issues released in 9.0.54, and announced later:

• CVE-2021-42340: Denial of Service when using websockets

Note: The product doesn't use web sockets in a way that would be affected by the above denial of service, this is being updated out of an abundance of care.

Support for SAP JCo 3.1.4 has been added. This JCo version is required for SAP SNC connections.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-44228.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

For on premise customers who are not able to apply this update then Redwood strongly recommends that you configure the logging to mitigate this by adding -Dlog4j2.formatMsgNoLookups=true to the startup configurations. You will need to restart your server for this to take effect.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-44228.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

Note that the -Dlog4j2.formatMsgNoLookups=true flag is not effective against the latest exploit, however the default logging configuration does not use the vulnerable pattern layouts.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-45105.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

Note that the -Dlog4j2.formatMsgNoLookups=true flag is not effective against the latest exploit, however the default logging configuration does not use the vulnerable pattern layouts.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-44832.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

This vulnerability requires that the log4j2.yaml file be modified, and that the server be restarted, for the system to be vulnerable. We recommend that customers protect this logging configuration to ensure that it can not be modified.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.