Business folks are asking for a count of all documents loaded in 2020 for a specific folder. Usually not a big deal, only 6 AG's in that folder..
How I've done it in the past is use the system load / system log folder, and put it into excel..and some simple parsing. But these are large application groups, some have 100k+ loads per month. I've learned obviously that the CMOD client doesn't like copying out more than 30-40k at a time.
My Idea? -- I'm thinking possibly also of just hitting up the SA* tables, of which we have two of. Anyone have any more ingenious ways of doing this?
The CMOD System Load (SA*) tables should be your first stop. There's no AppGroup field in the OnDemand System Log without doing a full-text search, which is terrible for performance. A few tweaks of SQL should help you get what you need.
Also, loading 100k files per month is terrible. Load larger files less frequently to reduce the overhead in CMOD. :)
-JD.
Quote from: Justin Derrick on February 24, 2021, 01:24:06 AM
The CMOD System Load (SA*) tables should be your first stop. There's no AppGroup field in the OnDemand System Log without doing a full-text search, which is terrible for performance. A few tweaks of SQL should help you get what you need.
Also, loading 100k files per month is terrible. Load larger files less frequently to reduce the overhead in CMOD. :)
-JD.
Yep, i tinkered with the SA* tables tonight for a few minutes after I posted this and got my results.
I agree with you about the loads. There�s literally a month worth of files. Tens of thousand. With one single document per file.
Yup, the overhead for this is tragic. Probably more overhead in OnDemand's internal database tables than actual index data.
-JD.