Overnight data load optimisation

The graphs below illustrate the changes that were made to the overnight data load process. The top graph show the average load on the server and it's distribution over the entire day, using a 24 hour clock. The graph directly beneath it is the average cost per row, an indicative measure of the efficiency of the code in loading data into the warehouse.

The second set of graphs show the same measures after the changes to the overnight data load process, the graphs share a common Y axis scaling. The second set of graphs shows a significant reduction in the load on the server and a more compact period of activity on the server between 3:00 and 5:00 am. The change in the efficiency of code loading the data warehouse can be seen as a reduction in the effort to load individual data rows.

.