Java Heap Space with Table Input, Multiple Joins and Groupings

Dear all

I am trying to load more than 500'000 records from a table input and around 23 database joins whereas 4 joins are grouped with the group by function. The whole result set should be exported to Excel using Microsoft Excel Writer.

I set the memory to 18000 M but I still get a java heap space error. Once the machine was not used at all, the export was successful, however I wonder if there is no other way to load and write the data with less memory and this constant problem about the heap space.

Nr. of rows in rowset is set to 5000
Feedback sice is set to 25000

I would appreciate any kind of help!

Thanks in advance.

Best regards
Michael

Stack Trace:
2016/07/13 08:14:39 - lastPerformanceFH.0 - Finished processing (I=290014, O=0, R=246958, W=246958, U=0, E=0)
2016/07/13 08:14:40 - lastPerformanceFC.0 - Finished processing (I=286105, O=0, R=246958, W=246958, U=0, E=0)
2016/07/13 08:14:40 - lastPerformanceOtherCounter.0 - Finished processing (I=270445, O=0, R=246958, W=248685, U=0, E=0)
2016/07/13 08:14:50 - woEventLinkPendingStatus.0 - Finished processing (I=490806, O=0, R=248685, W=242121, U=0, E=0)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : UnexpectedError:
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : java.lang.OutOfMemoryError: Java he
ap space
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at java.util.Arrays.copyOf(Arrays.java:2271)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.poi.openxml4j.opc.internal.MemoryPackagePartOutputStream.write(MemoryPackagePartOutputStre
am.java:88)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.xmlbeans.impl.store.Cursor._save(Cursor.java:590)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.xmlbeans.impl.store.Cursor.save(Cursor.java:2544)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.xmlbeans.impl.values.XmlObjectBase.save(XmlObjectBase.java:223)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.poi.xssf.usermodel.XSSFSheet.write(XSSFSheet.java:3072)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.poi.xssf.usermodel.XSSFSheet.commit(XSSFSheet.java:3027)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:332)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.poi.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:336)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.apache.poi.POIXMLDocument.write(POIXMLDocument.java:195)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.pentaho.di.trans.steps.excelwriter.ExcelWriterStep.closeOutputFile(ExcelWriterStep.java:253)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.pentaho.di.trans.steps.excelwriter.ExcelWriterStep.processRow(ExcelWriterStep.java:207)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2016/07/13 08:40:20 - Microsoft Excel Writer.0 - at java.lang.Thread.run(Thread.java:744)
child index = 2, logging object : org.pentaho.di.core.logging.LoggingObject@56552c8c parent=17d5fa54-c6c0-4ca0-bf49-74ff60e329ec
2016/07/13 08:40:20 - partRequirements - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Something went wrong while trying to stop t
he transformation: org.pentaho.di.core.exception.KettleDatabaseException:
2016/07/13 08:40:20 - partRequirements - Error cancelling statement
2016/07/13 08:40:20 - partRequirements - Invalid state, the PreparedStatement object is closed.
2016/07/13 08:40:20 - partRequirements - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleDatabas
eException:
2016/07/13 08:40:20 - partRequirements - Error cancelling statement
2016/07/13 08:40:20 - partRequirements - Invalid state, the PreparedStatement object is closed.
2016/07/13 08:40:20 - partRequirements -