SAP Transportation Management Optimizer Demystified

The most mysterious part of SAP’s Transportation Management solution is the Optimizer, and it’s not even close.  With almost every other functionality in SAP TM, we can always fall back on good old ABAP debugging to clear things up.  However, we are not afforded such luxury while troubleshooting or trying to understand Optimizer behavior.  And so, it is often viewed as a “black box” solution never to be fully accessible and comprehended.  While there is some truth to that, we can certainly help you get a much better understanding of the SAP Transportation Management optimizer functionality and ways of influencing its behavior.

Fundamentals of SAP Transportation Management optimizer architecture

Many of Optimizer’s mysteries are sealed away due to the underlying architecture whereas the Optimizer itself is a separate proprietary program created by SAP and written in C++ programming language, as opposed to ABAP, and that is executed remotely by the SAP Transportation Management application through a defined interface with a set of input parameters.  It is important to note that such architecture is not unique to SAP.  Many Transportation Management Systems (TMS) rely on a similar approach for optimization, leaving the actual optimizer engine inaccessible for debugging and any type of enhancements.

Despite our inability to see the actual C++ syntax of the SAP SCM Optimizer, we do have some information that SAP has shared over the years on the inner workings of the optimization approach:

• Most of the SAP SCM optimization scenarios utilize metaheuristic to iterate through possible plans and find the optimal solution.
• The solution generated by the optimizer is subject to a set of a) hard and b) soft constraints.  
• Hard constraints cannot be violated by the proposed solution under any circumstances—for example, the maximum capacity of a vehicle resource.
• Soft constraints can be violated, but such violations result in a “cost” penalty.  The cost we are referring to here is a statistical cost used to guide the optimization solution, not the actual cost from freight contracts.
• Ultimately, the best solution is the one that does not violate the hard constraints and results in the lowest possible statistical cost.

As part of every SAP Transportation Management implementation, following the instructions in the SAP TM Master Guide, we are directed to download the SAP SCM Optimizer software package from SAP Software Center portal, which as of the time of writing this article is on version 14.0.  We then need to upload the unarchived optimizer package to a Windows or Linux server instance of our choice.  Typically, this is a separate server instance from SAP Transportation Management application – one optimizer server for development and QA and a second server for production.  The reason for such separation is not only due to Operating System requirements but also due to the spiky nature of optimizer utilization, which is sitting idle most of the time, followed by short bursts of high CPU and memory usage.  

Finally, we follow the remaining steps in the SAP TM Master Guide to set up the Remote Functional Call (RFC) integration between the SAP TP application and optimizer server executables.  Technically, there are 3 executables on the optimizer server, dealing with 6 different application scenarios, as represented in the table below:

Planning profile configuration

In addition to the basic connectivity setup between SAP Transportation Management and SAP SCM Optimizer, we need to maintain the necessary planning profile configuration.  This is done using the TM Fiori menu path of Profiles & Settings > Edit Planning Profile and involves a significant number of settings that can influence the optimizer behavior.  

While we will not go into a detailed explanation of every setting within a planning profile, let’s highlight a few key areas of configuration:

• Optimizer Settings
  • FO Building Rule
    • This is also sometimes referred as “cutting rule”
    • Determines when one Freight Order stops and a new Freight Order begins, thereby avoiding the creation of Freight Orders with excessive duration.
    • Normally set to “New FO when Resource is Empty and Depot Location Reached”
  • Optimizer Runtime
    • Attributes that influence the relationship between duration and quality of optimization.
    • We recommend using the Automatic Runtime Regulation settings
      • Basic consolidation with single pick-up and destination should work perfectly fine even with “Fastest” setting here
      • For more complex scenarios, such as multi-stop route optimization with private fleet we recommend using the “Balanced” setting
  • Incremental Planning
    • Enables the use of incremental planning, where existing capacity documents (Freight Orders, etc.) can be re-optimized with new additional transportation demand (Freight Units, etc.).  Meant for scenarios where planners start building partial Freight Orders and then fill them with new demand as it trickles in.
• Load Planning Settings
  • These settings are associated with TVSO Load Optimization scenario.
• Constraints and Costs Settings
  • Area of planning profile configuration where most of the soft and hard constraints are maintained
  • Most settings here are maintained specific to a particular Means of Transport (Common Carrier FTL Dry Van vs LTL vs Private Fleet, etc.), but can also be maintained specific to a particular vehicle resource (in rare scenarios).
  • Statistical costs
    • Fixed cost per capacity document and/or vehicle resource
      • This is an important setting that prevents the optimizer from creating an excessive number of separate Freight Orders or Bookings
    • Penalty costs
      • Cost associated with earliness and lateness of pick-up or delivery.  This cost is applied against the soft “requested” dates on the Freight Unit.
    • Quantity costs
      • Cost per quantity of product shipped.  Typically only applies in specific means of transport, such as LTL and parcel.
    • Duration costs
      • Hard constraint of maximum duration per capacity document
      • Soft constraint of cost per duration unit of measure (for example hour or day).  This is another important setting that provides an incentive for minimizing/compressing the duration of capacity documents as much as possible.
    • Distance costs
      • Hard constraint for the maximum distance per capacity document
      • Soft constraint of cost per distance unit of measure (mile or kilometer).  This setting is often maintained for means of transport that incur higher cost with higher distances, such as FTL, Rail, etc.
    • Additional stops
      • Hard constraints for maximum number of pick-up, delivery and intermediate stops
      • Soft constraint of cost per additional intermediate stop
    • Utilization Settings
      • Minimum target utilization of the vehicle resource.
• Scheduling Settings
  • Settings maintained in this area primarily deal with vehicle resource processing durations and are utilized in both TVSR and TVSS scenarios
  • Coupling and uncoupling durations, when utilizing a combination of active (for example tractor) and passive (for example trailer) vehicle resources
  • Loading and unloading durations can be maintained as fixed and/or variable.  Variable durations often come into play with bulk goods such as liquids, where X number of quantity is loaded/unloaded per hour.
  • Prepare and finalize processing durations per stop
• Incompatibility Settings
  • SAP Transportation Management provides a wide range of incompatibilities that can be maintained and utilized by the optimizer.  Some common examples include:
    • “Freight Unit – Vehicle Resource” when certain orders are incompatible with certain types of vehicle equipment.  For example, a liquid bulk order would be incompatible with dry van truck.“Carrier – Transportation Order” when certain carriers are not allowed on particular Freight Orders/Bookings. 
    • “Vehicle Resource – Location” when certain vehicles are not compatible with certain loading/unloading locations.  For example, a customer might not allow trucks without a power liftgate.
• Carrier Selection Settings
  • Most of the settings in this section deal with TSPS carrier selection scenario of optimization
  • We assign the process controller strategy that will execute the carrier selection process. Standard strategy TSPS_DEF is commonly used but is sometimes swapped with a separate strategy configured to utilize a routing guide condition in addition to transportation lanes as the source of carrier assignment.
  • Overall Carrier Availability setting allows us to enable a more strict check for carrier assignment for multi-stop scenarios, insuring that acceptable carriers are not only assigned to a lane from the first to last stop, but also on lanes of all of the intermediate stages.
  • A number of settings control the logic of Continuous Move functionality whereby carrier assignment and ranking on a particular Freight Order is influenced by carrier assignment on previous Freight Orders, attempting to provide continuous move opportunities for carriers, which often result in cost and/or quality of service savings for the shipper.
  • We can also turn on the use of transportation allocation and business share functionality that are utilized when we want to influence the distribution of freight assignment to carriers on particular lanes. 

TM and Optimizer integration

Ultimately once all of the necessary configuration is in place we are able to launch optimization either interactively from within the Transportation Cockpit or in background via program /SCMTMS/VSR_OPT_BGD.  When an optimization request is initiated SAP Transportation Management assembles an input file that is passed to the SAP SCM Optimizer executable.  

The input file is nothing more than a space-delimited text file with extension *.in containing a number of tables and rows of data that follow.  

Here is an example of the formatting within the input file:

// TABLE ET_MOT
// MOT    MOT_CAT DESCRIPTION
// STRING STRING  STRING
   01     1       Road
   02     2       Rail
   03     3       Sea
   04     3       Inland Waterway
   05     4       Air
   06     1       Postal Service

This section of the input file represents table ET_MOT which is used to pass the relevant Modes of Transport from TM to the optimizer.  This table contains three columns – MOT, MOT_CAT and DESCRIPTION – and 6 rows of actual data for the various modes of transport configured in TM.

As a result, the input file contains all the data necessary for the SCM Optimizer to perform its function, including:

• Transactional document details – freight units, transportation units, freight orders, etc. – including ROOT/header, item and stage-level information
• Vehicle resources
• Location master data, including inbound and outbound calendars and handling resources
• Transportation zones and lanes, including associated Means of Transport assignments
• Incompatibilities
• Hard and soft constraints maintained in the planning profile that we discussed above

Similarly, the output of the SCM Optimizer is a delimited text file with extension *.out that is sent to and interpreted by the ABAP logic maintained in SAP Transportation Management.  It follows the same formatting rules as the input file, containing tables along with data, detailing the proposed solution, including the proposed creation of new capacity documents (freight orders, bookings, etc.), assignment of freight units to capacity documents, as well as the breakdown of statistical costs associated with the proposed solution.

Both the input and output files can be viewed in the raw format using SAP transaction RCC_LOG, if the appropriate logging settings have been maintained:

• Optimizer dump level has to be set to value of 1 in transaction /SCMTMS/OPT10 like so:

• The specific user performing the optimization in the Transportation Cockpit or running report /SCMTMS/VSR_OPT_BGD has to have Parameter ID /SCMTMS/EXP maintained like so:

As long as the aforementioned settings are maintained, the next time that particular user executes an optimization, we can proceed to transaction RCC_LOG and review the results, including access to the raw input and output files.  If the optimization run was executed from the Transportation Cockpit application, then using the Explanation button at the top of the screen will take you to the same place.

Transaction RCC_LOG initial screen shows a table of optimizations that have been launched and sorted by date and time of execution:

• The “Appl.” column displays the optimizer application described in Table 1 above
• User Name refers to the user that executed the optimization
• Profile refers to the planning profile that was utilized

Clicking on the icon in the “Ext.Column” will take you to the Optimizer File Analysis screen, which provides a UI to review the inputs and outputs of a particular optimization run.  It’s worth noting that this icon only appears in a select few application scenarios – TVSR and TVSO.  For all other application scenarios we have to rely on review of raw input and output files.  

Once inside the Optimizer File Analysis application, we can utilize the tree structure on the left to navigate around.  All the information associated with the *.in input file is displayed under the “Input” folder, while data associated with the *.out output file is displayed under folder titled “Result”.

From here we can navigate to detailed information associated with each optimization.  For example, by double-clicking on the Freight Units folder, we can review root and stage-level data of Freight Units that were part of the optimization.

Note that anytime you see a piece of data that is underscored, such as FU UI or Source Loc in the example above, it is actually a hyperlink that will take you to display transaction associated with that particular object.

One area that is particularly useful while troubleshooting optimization results is under Result > Freight Units > Messages for Stages where we can often locate clues as to why a particular Freight Unit stage failed to plan through optimization.

In certain cases, using the Optimizer File Analysis is not sufficient or even possible.  For example, in cases when troubleshooting requires the creation of an incident with SAP you will need to attach raw input and output files.  To access those files, select an optimization in transaction RCC_LOG and then use the top menu “Extras”.  For SAP incidents you will want to choose the “Compressed Archive File” option.  However, if you just want to review the raw input and output files yourself, you can choose “Download Data File”. Keep in mind that in both cases, you will download archived files with extension *.gz that you will need to extract to get to actual *.in and *.out files, which you can then open in any text editor such as Notepad++ or TextMate.

Enhancement options

Looking at the structure of the *.in input file is helpful for understanding the options we have with enhancing optimizer behavior.  While we cannot change or enhance the syntax of the optimizer executable itself, we can change the data inside the input file, thereby impacting the results produced by the optimizer.

The primary mechanism for enhancing the data inside the input file is “Optimizer Pre-Processing” BAdI. There are several of such BAdIs present, but the one is typically utilized the most for VSR optimization is BAdI /SCMTMS/PLN_PRE_PROC with method CHANGE_OPT_INPUT.  This BAdI provides full access to all of the tables that are passed to the optimizer via the input file and allows us to insert/update/delete entries in those tables.  Here are some examples of enhancements that can be achieved via this BAdI that are otherwise not currently possible via standard configuration in SAP Transportation Management:

• Modify the use of acceptable dates (hard constraints) on freight units, depending on the settings in the planning profile.  
  • With standard SAP acceptable dates are enforced by the optimizer simply if they are maintained in freight units passed to optimization.  But what if you want this behavior to be dynamic and have some planning profiles use acceptable dates, and others ignore them?  
  • There is a flag in planning profile Scheduling settings called “Consider Requirement Document Dates” that achieves such results for scheduling (VSS) but it does not impact route optimization (VSR).
  • To achieve similar behavior in VSR optimization, we utilize the pre processing BAdI to check the Scheduling setting above in the planning profile, and if it only calls for requested dates (soft constraints) clear out the acceptable dates in table ET_TOR_STOP of the input file.
• Enforce that only 1 FU is assigned to 1 FO.  This is often a common requirement for LTL scenarios.
  • First, we define a custom dimension in table ET_DIM with a name of something like “NumOfReqLTL”.
  • Next, we assign this dimension to pertinent vehicle resources in table ET_VEH_CAPA with a capacity value of 1.  In our example, it would be all LTL vehicle resources.
  • Finally, we assign this custom dimension with a quantity value of 1 to every single Freight Unit in table ET_TOR_QUAN.
  • The same approach can be taken in other similar scenarios.  For example, enforcing that only 1 Container Unit is assigned to a Freight Order with a chassis truck.

Another less-known mechanism for influencing optimizer behavior is through the use of RCCF Expert Parameters.  Such parameters can be maintained globally or specific to a particular planning profile.  They are maintained via transaction RCC_PARAM and are generally considered “last resort” as they often lead to unwanted side effects.  On top of this, SAP does not document the use of these parameters very well (on purpose) and so there is no formal list of all possible parameters along with their impact on optimizer behavior documented in a single concise list.  Instead, we have to rely on SAP support or existing SAP notes (for example 578044) to help understand what kind of behavior is achievable via such Expert Parameters.  Here are some examples:

• Compactification in Scheduling
  • Section – VSR_ELS
  • Name – COMPACTIFICATIONINSCHEDULING
  • Switch – Integer
  • Integer value – 1
  • Purpose – this parameter promotes scheduling to return as compact dates as possible, avoiding any pauses or gaps.
• Ignoring requested dates
  • Refer to SAP note 3259719
  • Section – VSR_RELAX
  • Name – RelaxRequestedPickupDates, RelaxRquestedDeliveryDates
  • Section – VSS
  • Name – IgnoreDesiredPickupDates, IgnoreDesiredDeliveryDates
  • Switch – Boolean Value
  • Char. String – X
  • Purpose – the use of these parameters allows you to ignore requested dates in FUs and thereby rely only on acceptable dates.

Additional Information

Thomas Engelmann from SAP optimizer team has published a wonderful series of articles on TM optimization that you locate here.

A good starting point for the topic of Planning in SAP Transportation Management can be located at the following link on SAP Help portal.

This SAP help page provides an overview of constraints used by VSR optimization in SAP Transportation Management. 

For general SAP Transportation Management topics, refer to our FAQs for SAP Transportation Management and S/4HANA Embedded TM.