I'm processing thousands of XML documents where each document has between 150 and 210 looping nodes. Something like this:
After about 4,000 documents (corresponding to 800K <Data> nodes), the JVM runs out of memory and the whole thing crashes.
Is this a known issue?
Is there a workaround?
I have attached an example transform that demonstrates (though the actual transform is much more complex and runs with map-reduce).
Code:
<DataReport>
<ReportNodeA />
<ReportNodeB />
<Details>
<Data />
<Data />
... 200 nodes ...
<Data />
</Details>
</DataReport>Is this a known issue?
Is there a workaround?
I have attached an example transform that demonstrates (though the actual transform is much more complex and runs with map-reduce).