In a former life I was a user in between stints as a programmer. Although I would say the experience taught me humility and respect for those programmers who I requested things for, I would be lying. Every time I would ask the programmer why this report was taking a lot of time, the response would invariably be: "Do you know how much data you are requiring me to sort through to get you what you wanted?" Of course, I played ignorant and said: "It just a few million records, it should not take hours." The next reply from the programmer was: "If you think this is so easy, why don't you write it yourself!"
A year ago I was helping out on a simple report that should run in a few minutes. It took 1 hour and 40 minutes to run an accounts receivable aging against 2 million plus records. First I looked at the select and found it was taking about three to five minutes to get all 2 million records. I changed the select to use alternative keys and reduced the criteria to select against. Those coding changes reduced the select time to under 3 seconds.
The code that processed the data was building JSON arrays that are passed on to the open source report writer from Eclipse.org called Business Intelligence Reporting Tool (BIRT). Building those arrays took about 1 hour and 30 minutes using a standard UniBasic code structure of appending to a dynamic array. I could not change the code to a dimensioned array because the size of the report varies from 1 pages to over 4000 pages. I tested it against my better judgement with a dimensioned array that would cover a 5000-page report and it did run in about an hour.
There is a relatively new structure in UniBasic called UDO which stands for U2 Data Objects. I created the JSON arrays directly using the UDO function calls. The time to process the data was down to 50 seconds. Wow, that was more improvement that I had anticipated. Our BIRT report in production finishes in a little over one minute.
Maybe your programmer is right that it takes hours in today's world to process large amounts of data. Or maybe your programmer is using outdated coding technique that can be improved by being a lot more knowledgeable.
A year ago I was helping out on a simple report that should run in a few minutes. It took 1 hour and 40 minutes to run an accounts receivable aging against 2 million plus records. First I looked at the select and found it was taking about three to five minutes to get all 2 million records. I changed the select to use alternative keys and reduced the criteria to select against. Those coding changes reduced the select time to under 3 seconds.
The code that processed the data was building JSON arrays that are passed on to the open source report writer from Eclipse.org called Business Intelligence Reporting Tool (BIRT). Building those arrays took about 1 hour and 30 minutes using a standard UniBasic code structure of appending to a dynamic array. I could not change the code to a dimensioned array because the size of the report varies from 1 pages to over 4000 pages. I tested it against my better judgement with a dimensioned array that would cover a 5000-page report and it did run in about an hour.
There is a relatively new structure in UniBasic called UDO which stands for U2 Data Objects. I created the JSON arrays directly using the UDO function calls. The time to process the data was down to 50 seconds. Wow, that was more improvement that I had anticipated. Our BIRT report in production finishes in a little over one minute.
Maybe your programmer is right that it takes hours in today's world to process large amounts of data. Or maybe your programmer is using outdated coding technique that can be improved by being a lot more knowledgeable.
