Anyone know of any rules of thumb or recommendation for max number of arsloads to run with -d ? AIX 7, DB2 9.5, local cache for everything, each loading from different directory. Goal is to run concurrent loads of up to 100,000 objects.
Thanks,
-ww
This depends on the capacity of the system like ram, speed and number of processor. But having 100,000 instances of arsload would not be good idea since in one of the performance testing I did, when loading around 10 loads concurrently, the load time of 1 instance had increased 4-5 times compared to loading sequentially. Typically when you use indexers (ACIF or PDF ) they are generally processor intensive process.
I'd written a reply, but I'm not sure what I did... Maybe I forgot to click 'Post'. Ah well...
10,000 individual arsload processes are going to eat up most, if not all your RAM. It's also going to ensure you have a LOT of trouble with locking in tables, and it will perform terribly.
You didn't give us much information on the (very strange) requirement. With no additional information, the most sane suggestion I can come up with is... Split the 100,000 objects into 1000 directories. Configure 100 instances of arsload to check 10 directories each, and go from there. Even that solution isn't optimal.
Questions:
Are you really going to have 100,000 different reports? Even at 15 minutes to create and test each Application, running on a 24x7x365 basis, that would take 3 years to create them all.
What's the turnaround time expected to be on a report (how long from arrival on the server to availability to end users in CMOD)?
How many AGs are you expecting to do this in?
What type of hardware (CPU / RAM / Disk) will you be using?
-JD.
Ive seen this done with like 15 daemons another 15 or so custom coded java apps that call ARSLOAD and it performs nicely.
Jeff,
How many db2 connections you have to support 15 daemons and 15 others arsload?
Just curious.
Thanks.
-jw
Hi,
I'm also interested to know the reason of this requirement, maybe you want to load 100 000 documents in a very short time and have to do that periodically? We had such requirement once, and as a solution we created one common indexfile for all documents for certain application - over 300k objects (small PDF). They were all loaded almost at once...much faster then loading it from deamon one by one. ;)
BR
Maciek
Quote from: jw on July 25, 2014, 06:33:17 PM
Jeff,
How many db2 connections you have to support 15 daemons and 15 others arsload?
Just curious.
Thanks.
-jw
I do not work at that company anymore and I do not know off the top of my head. Sorry :(