Quantcast
Channel: Pentaho Community Forums
Viewing all articles
Browse latest Browse all 16689

New Fields created in Plugin (custom) step does not get through to the next step

$
0
0
Hello , I am fairly new to creating custom components in kettle. I created my own plugin step say ParseLog in which I have a Field1 input coming into this step from say a file and I am doing some custom logic and splitting it into n number of fields. I have created an another plugin step say LoadToOurDB which takes the output of previous step and loads into our proprietary database . So my problem is that I am able to see all the fields created in ParseLog step come to my LoadToOurDB step and also everything comes through properly when I preview but if I throw in a say Output To File step instead of LoadToOurDB and do Get Fields while setting up , I just get the Field1 and none of the other fields that I created in my ParseLog plugin step. This is what I do and I was wondering what is going on ? Thanks for the help in Advance.


public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException {
meta = (MyLogParserStepMeta) smi;
data = (MyLogParserStepData) sdi;


Object[] r = getRow();

if (r == null) // no more input to be expected...
{
setOutputDone();
return false;
}

if (first) {

data.outputRowMeta = new RowMeta();
Enumeration enuKeys = columnMapping.keys();
while (enuKeys.hasMoreElements()) {
String key = (String) enuKeys.nextElement();
RowMetaInterface newField = new RowMeta();
String type = columnTypes.getProperty(key);
ValueMetaInterface newFieldMeta = null;

if (type.equalsIgnoreCase("VARCHAR"))
{
newFieldMeta = new ValueMeta(key, ValueMeta.TYPE_STRING);

}else if (type.contains("INT32"))
{
newFieldMeta = new ValueMeta(key, ValueMeta.TYPE_INTEGER);
}else if (type.equalsIgnoreCase("TIMESTAMP"))
{
newFieldMeta = new ValueMeta(key, ValueMeta.TYPE_DATE);
}else if (type.contains("BOOL"))
{
newFieldMeta = new ValueMeta(key, ValueMeta.TYPE_BOOLEAN);
}else
{
newFieldMeta = new ValueMeta(key, ValueMeta.TYPE_STRING);
}

newField.addValueMeta(newFieldMeta);
data.outputRowMeta.addRowMeta(newField);
}

row = new Object[ data.outputRowMeta.size()];

first = false;
}

if (r[0] != null)
{
String rowStr = r[0].toString();
String columns[] = rowStr.split("\\|"); // its Has lot more processing but I have simplified it for better understanding


for (int i =0;i < columns.length ;i++)
{
row[i] = columns[i];
}

putRow(data.outputRowMeta, row);

}

return true;
}

Viewing all articles
Browse latest Browse all 16689

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>