Programme SAP RBDMIDOC_RETAIL - Article Master Data Distribution (Packet Processing)

Title

Purpose
The distribution of changes to material master data is based on ALEchange pointers that are generated when master data is created.
ALE standard triggers the distribution of master data using transactionBD21. Subsequent processing is handled in an individual process for allmaster data that is to be distributed. If a particularly large amountof data is to be distributed, main memory consumption may increae andyour system may experience particularly long runtimes.
Report RBDMIDOC_RETAIL can be used to avoid these problems. After thechange pointers have been read by the system, the master data changesare sent in data packages. It is also possible to use parallelprocessing internally to handle the data. In doing so, main memoryconsumption is reduced. The runtime for the distribution of data isalso considerably reduced.

Prerequisites
If you want to use data packages and parallel processing in yoursystem, you are required to create a suitable aRFC Server Group inyour system. Use transaction RZ12 to do so. When doing so, you mustensure that all the servers to which data is distributed havesufficient dialog work processes available.
The algorithm for distributing master data functions as follows:

  • System reads all unprocessed change pointers for the relevant message
  • type.
    • System analyzes data to ascertain the materials for which changes
    • exist. A worklist for sending master data is generated.
      • System sends generic material including the assigned variants (if any
      • exist). The generic material with the greatest number of changepointers is sent first.
        • System sends single materials (if any exist). The single material with
        • the greatest number of change pointers is sent first.
          Data packages can contain both generic materials and single materialsif data package creation could not be completed for the genericmaterial that was sent last.
          The distribution logic ensures that every data package with a genricmaterial also contains all the changes to the variants of the genericmaterial. This, therefore, ensures that the changes to the genericmaterial and its variants can be sent in an IDoc at a later time.
          In addition to the distribution of master data changes, the simulationmode enables the user to check the paramaters that have been selectedfor data package creation, without actually having to send any data. Inaddition, the user can also select to have a detailed distribution loggenerated by the system.
          To ensure that data packages and parallel processing are usedcorrectly, the user must ensure that the selection parameters for datapackage creation are correctly configured. The most important selectionparameters are detailed in the next section of this document.

          Selection

          • Frame "Selection of Changes":

          • User selects the message type that is to be used.
            Message Type:
            Only message type ARTMAS or associated message types can be used withthis report to distribute master data changes.
            • Frame "Configuration" : "Parallel Processing":

            • User can choose to process data in data packages or using parallelprocessing.
              aRFC Server Group:
              User enters the name of a valid aRFC Server Group. If data packageprocessing without internal parallel processing is to be used formaster data distribution, this field can be left empty.
              Number of Parallel Processes:
              If data is to be distributed internally using parallel processing, usethis selection parameter to define a maximum number of parallelprocesses. This limit will not be exceeded when data is distributedeven if the server group that is bein used does not allow more than thedefined number of processes to be used. This means that other processescan be run on the same server group at the same time, without all av
              ailable resources being used to distribute changes to master data.
              • Frames "Configuration" : "Package Processing":

              • Targets (limitation of main memory consumption/parallel processing) canonly be achieved most efficiently if the selection parameters for datapackage creation have been configured appropriately.
                For each logistical material unit (single material or generic materialwith all relevant variants), the system checks if one of the parametersfor data package creation has been exceeded. If this is the case, datapackage creation is completed and the package is sent. Note that thedata package creation process groups different logistical materialunits into a single package when one of the parameters that is used isactually undercut. The dimensions of these parameters should not,therefore, be too limited. Depending on the parameters that areselected for data package creation, a package can contain both genericlmaterials and single materials.
                Data package creation aims to create exactly the right amount of datapackages - not too many but not too few. If too many data packages arecreated (for example, one data package for each individual material),too many individual RFC (parallel processes) have to be managed andcommunication requirements become excessive. It could also be the casethat the distributing processes cannot distribute the new packagesquickly enough into processing. If too few data packages are created,main memory consumption cannot be restricted enough. It may also be thecase that larger, individual data packages may dominate the overalldistributio runtime despite the magjority of the other packages alreadyhaving been processing in a worklist.
                Using data package creation correctly is reliant on the fixed datastructure on which master data distribution is based, as well as on theavailable system resources. It is, therefore, very difficult to makegeneral recommendations. Default values are displayed on the selectionscreen. The simulation mode also empowers the user to monitor theselected parameters without first having to trigger distribution. Thismeans that no parallel processes have to be triggered, no IDocs aregenerated and no change pointers are flagged as having been previouslyprocessed.
                Number of Materials Per Process:
                Defines the maximum number of materials per process. The variants of ageneric material are counted as a single material. As the variants haveto be processed in the same data package as the base generic article,the number of materials in the package may actually exceed the valuethat has been entered by the user.
                Number of Change Pointers Per Process:
                Defines the maximum number of change pointers per process. If thisvalue is exceeded, the data package will contain few materials than the'Number of Materials Per Process'. As the variants always have to beprocessed in the same data package as the base generic material, thedata package may contain more process change pointers than the maximumnumber entered by the user.
                • Frames "Configuration" : "Log / Simulation":

                • The system can generate a detailed log for master data distribution.Alternatively, the user can choose to simulate distribution.
                  Display Log:
                  Log can be output on the screen or in the spool list for the job.
                  Simulte Distribution:
                  When distribution is simulated, the change pointers are read and datapackages are created in accordance with the parameters described above.When distribution is simulated, a log detailing the process is alsocreated. If medium-sized packages are used in distribution simulation,the amount of main memory consumed represents an approximation of theamount of main memory that would be used, were distribution actually tobe run.

                  Output
                  Report RBDMIDOC_RETAIL provides a detailed log that documents the useof system resource during the distribution runtime. The log is brokendown into five sections:

                  • Distribution Log 1:

                  • Displays the task number (sequence number), the number of materials andchange pointers per process, thus indicating the quality of datapackage creation.
                    • Distribution Log 2:

                    • Displys the task number (sequence number), the start, end and runtimeper task, the server and the number of parallel processes duringruntime, thus indicating the quality of data load distribution.
                      • Generic Material Statistics:

                      • Displays the generic materials sent, the number of variants and thetotal number of change pointers per generic material and variant.
                        • Runtime Overview:

                        • Logs start time, end time and the runtime for the individual stepsduring processing (for example, read change pointers, structure keytables).
                          • Data Volume / Parameter Selection:

                          • Displays the number of materials sent subdivided into genericmaterials, variants and other materials (for example, single materials)as well as selection parameters (number of materials per process,number of change pointers per process, number of parallel processes).
                            Example 1: Distribution of Generic Materials With Variants

                            Distribution of Material Master Data (Data Package Processing)
                            Distribution Log 1:
                            Task Materials Change Pointers
                            1 + 11 + 1.041
                            2 + 11 + 1.041
                            ....
                            98 + 11 + 1.041
                            99 + 11 + 1.041
                            100 + 11 + 1.041
                            Distribution Log 2:
                            Task Start End Runtime ServerNumber of Parallel Processes
                            00000001 + 14:18:39 + 14:19:01 + 00:00:22 + PWDF0635_AL0_55 +1
                            00000002 + 14:18:39 + 14:18:49 + 00:00:10 + PWDF0636_AL0_55 +2
                            00000003 + 14:18:39 + 14:19:11 + 00:00:32 + PWDF0635_AL0_55 +3
                            00000004 + 14:18:39 + 14:18:49 + 00:00:10 + PWDF0636_AL0_55 +4
                            00000005 + 14:18:39 + 14:19:14 + 00:00:35 + PWDF0635_AL0_55 +5
                            00000006 + 14:18:40 + 14:18:49 + 00:00:09 + PWDF0636_AL0_55 +6
                            00000007 + 14:18:43 + 14:19:09 + 00:00:26 + PWDF0635_AL0_55 +7
                            00000008 + 14:18:43 + 14:18:53 + 00:00:10 + PWDF0636_AL0_55 +8
                            00000009 + 14:18:47 + 14:19:10 + 00:00:23 + PWDF0635_AL0_55 +9
                            00000010 + 14:18:47 + 14:18:55 + 00:00:08 + PWDF0636_AL0_55 +10
                            ...
                            00000099 + 14:20:33 + 14:20:49 + 00:00:16 + PWDF0635_AL0_55 +10
                            00000100 + 14:20:33 + 14:20:49 + 00:00:16 + PWDF0635_AL0_55 +10

                            Generic Material Statistics:
                            Generic Material,,Variants,,Change Pointers
                            SA10VAR___00001 + 10 + 1.041
                            SA10VAR___00002 + 10 + 1.041
                            SA10VAR___00003 + 10 + 1.041
                            ....
                            SA10VAR___00099 + 10 + 1.041
                            SA10VAR___00100 + 10 + 1.041
                            Runtime Overview:
                            Processing Phase,, Start______End _____Runtime
                            Read Change Pointer 14:18:19___14:18:37__00:00:18
                            Build Key Table 14:18:38___14:18:39__00:00:01
                            Prepare Package Creation 14:18:39___14:18:39__00:00:00
                            Distribute Generic Materials 14:18:39___14:20:33__00:01:54
                            Create Log 14:20:51___14:20:51__00:00:00
                            Total Runtime 14:18:18___14:20:51__00:02:33
                            Data Volume / Parameter Selection:
                            Total Materials: 1.100
                            Generic Materials: 100
                            Variants: 1.000
                            Other Materials: 0
                            Number of Materials Per Process: 8
                            Number of Change Pointers Per Process: 800
                            Example 2: Distribution of Single Materials
                            Material Master Data Distribution (Data Pacakage Processing)
                            Distribution Log 1:
                            Task Materials Change Pointers
                            1 + ,,,,31 +,,,,3.069
                            2 + ,,,,31 + ,,,,3.069
                            3 +,,,,31 +,,,,3.069
                            4 +,,,,31 +,,,,3.069
                            5 +,,,,31 +,,,,3.069
                            6 +,,,,31 +,,,,3.069
                            7 +,,,,14 +,,,,1.386
                            Distribution Log 2:
                            Task Start End Runtime ServerNumber of Parallel Processes
                            00000001 + 15:14:42 + 15:15:20 + 00:00:38 + PWDF0635_AL0_55 +1
                            00000002 + 15:14:43 + 15:15:22 + 00:00:39 + PWDF0636_AL0_55 +2
                            00000003 + 15:14:43 + 15:15:25 + 00:00:42 + PWDF0635_AL0_55 +3
                            00000004 + 15:14:43 + 15:15:19 + 00:00:36 + PWDF0636_AL0_55 +4
                            00000005 + 15:14:44 + 15:15:28 + 00:00:44 + PWDF0635_AL0_55 +5
                            00000006 + 15:14:45 + 15:15:22 + 00:00:37 + PWDF0636_AL0_55 +6
                            00000007 + 15:14:45 + 15:15:10 + 00:00:25 + PWDF0635_AL0_55 +7
                            Runtime Overview:
                            Processing Phase Start_____End ______Runtime

                            Read Change Pointers 15:14:15__15:14:39__00:00:24
                            Build Key Table 15:14:40__15:14:40__00:00:00
                            Prepare Package Creation 15:14:40__15:14:40__00:00:00
                            Distribute Other Materials 15:14:41__15:14:45__00:00:04
                            Create Log 15:15:28__15:15:28__00:00:00
                            Total Runtime 15:14:15__15:15:28__00:01:13
                            Data Volume / Parameter Selection:
                            Total Materials: 200
                            Generic Materials: 0
                            Variants: 0
                            Other Materials: 200
                            Number of Materials Per Process: 50
                            Number of Change Pointers Per Process: 3.000
                            Number of Parallel Process: 10
                            Example 3: Distribution of Single Materials, Generic Materialsand Variants
                            Material Master Data Distribution (Data Package Processing)
                            Distribution Log 1:
                            Task Materials Change Pointers
                            1 + 12 + 1.044
                            2 + 12 + 1.044
                            3 + 12 + 1.044
                            4 + 10 + 942
                            5 + 10 + 990
                            6 + 10 + 990
                            7 + 10 + 990
                            8 + 10 + 990
                            9 + 10 + 990
                            10 + 10 + 990
                            11 + 10 + 990
                            12 + 10 + 990
                            13 + 10 + 990
                            14 + 4 + 396
                            Distribution Log 2:
                            Task Start End Runtime ServerNumber of Parallel Processes
                            00000001 + 18:18:06 + 18:18:50 + 00:00:44 + PWDF0635_AL0_55 +1
                            00000002 + 18:18:06 + 18:18:44 + 00:00:38 + PWDF0636_AL0_55 +2
                            00000003 + 18:18:06 + 18:18:18 + 00:00:12 + PWDF0637_AL0_55 +3
                            ....
                            00000012 + 18:18:18 + 18:18:27 + 00:00:09 + PWDF0637_AL0_55 +10
                            00000013 + 18:18:22 + 18:18:43 + 00:00:21 + PWDF0636_AL0_55 +10
                            00000014 + 18:18:27 + 18:18:31 + 00:00:04 + PWDF0637_AL0_55 +10
                            Generic Material Statistics:
                            Generic Material Variants Change Pointers
                            SA3VAR____00001 + 3 + 348
                            SA3VAR____00002 + 3 + 348
                            ....
                            SA3VAR____00008 + 3 + 348
                            SA3VAR____00009 + 3 + 348
                            SA3VAR____00010 + 3 + 348
                            Runtime Overview:
                            Processing Phase Start_____End_______Runtime

                            Read Change Pointers 18:17:53__18:18:05__00:00:12
                            Build Key Table 18:18:05__18:18:05__00:00:00
                            Prepare Package Creation 18:18:05__18:18:05__00:00:00
                            Distribute Generic Materials 18:18:05__18:18:06__00:00:01
                            Distribute Other Materials 18:18:06__18:18:27__00:00:21
                            Create Log 18:18:54__18:18:55__00:00:01
                            Total Runtime 18:17:53__18:18:54__00:01:01
                            Data Volume / Parameter Selection:
                            Total Materials: ________________________140
                            Generic Materials: ______________________10
                            Variants: _______________________________30
                            Other Materials: _______________________100
                            Number of Materials Per Process:__________10
                            Number of Change Pointers Per Process: 2.000
                            Number of Parallel Processes:_____________10
                            The following measurements were recorded in an SAP developmentsystem. Note that the results from these measurements may not becompletely applicable for other systems.
                            Runtime Measurements Data Package Creation + Parallel Processing:
                            Runtime,,Number of Materials,,Maximum Materials,,MaximumPointers,,Number of Packages
                            9 min. 11 sec.,,5,,20,,3.000,,170
                            4 min. 26 sec.,,10,,20,,3.000,,170
                            3 min. 41 sec. ,,15,,20,,3.000,,170
                            3 min. 30 sec.,,15,,40,,3.000,,107
                            3 min. 18 sec.,,20,,40,,3.000,,107
                            Runtime Measurement for RBDMIDOC In Series (Reference Measurement):
                            Runtime,,Number of Materials,,Maximum Materials,,MaximumPointers,,Number of Packages
                            31 min. 04 sec.,,1,,0,,0,,1
                            An equal number of master data IDocs were distributed in bothexamples detailed above Due to the fixed share of the runtime (forexample, for reading change pointers), the increase in performance isnot equal to the increased number of parallel processes.
                            The runtime for 15 parallel processes does, however, represent 10%of the runtime recorded if no data packages were created nor parallelprocessing run.