After searching the forum, and Jira, I can't seem to find an answer, explanation or bug relating to what I'm experiencing, so hoping someone can either explain this situation, or confirm it is a bug, which I'll raise in Jira.
I have a transformation which loads a csv into an Oracle table, though I've found the target is irrelevant as it is the handling of the data that seems to be the problem. I have a numeric field in the source file, which depending on the length of the number, seems to lose accuracy.
Here is the output of the run of the test I have made, the same input is going into both fields, however as can be seen, the output which had been defined as a number, varies as the length of the number reduces). I have tried any number of combinations of formats, lengths, precisions and setting the decimal delimiter, but none achieve what I would expect, and that is to pass through the number as it is stored. It is not "just a format issue" writing to the log, as I have also been inserting into a table, which always matches the displayed values.
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 1------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000098760614115342832.000 Rounded?
2013/12/05 15:06:08 - Write to log.0 - str_input = 98760614115342826.123
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 2------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000000760614115342826.100 Rounded/Truncated?
2013/12/05 15:06:08 - Write to log.0 - str_input = 760614115342826.123
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 3------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000000060614115342826.125 Rounded?
2013/12/05 15:06:08 - Write to log.0 - str_input = 60614115342826.123
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 4------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000000003614115342826.123 Accurate
2013/12/05 15:06:08 - Write to log.0 - str_input = 3614115342826.123
The results don't seem particularly predictable...:confused: I've attached the transformation and input file to illustrate the definition.
If this is a restriction, I think it should be documented somewhere, ironically, if it is left as a string, it inserts OK. I'd rather tie the types down though as it maintains an audit of the different data types being mapped through. I don't like the idea of it changing the content with no warning, it isn't like it is an invalid number as found in other posts, where the parser just gives up.
Any suggestions welcome.
Cheers, Will
I have a transformation which loads a csv into an Oracle table, though I've found the target is irrelevant as it is the handling of the data that seems to be the problem. I have a numeric field in the source file, which depending on the length of the number, seems to lose accuracy.
Here is the output of the run of the test I have made, the same input is going into both fields, however as can be seen, the output which had been defined as a number, varies as the length of the number reduces). I have tried any number of combinations of formats, lengths, precisions and setting the decimal delimiter, but none achieve what I would expect, and that is to pass through the number as it is stored. It is not "just a format issue" writing to the log, as I have also been inserting into a table, which always matches the displayed values.
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 1------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000098760614115342832.000 Rounded?
2013/12/05 15:06:08 - Write to log.0 - str_input = 98760614115342826.123
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 2------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000000760614115342826.100 Rounded/Truncated?
2013/12/05 15:06:08 - Write to log.0 - str_input = 760614115342826.123
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 3------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000000060614115342826.125 Rounded?
2013/12/05 15:06:08 - Write to log.0 - str_input = 60614115342826.123
2013/12/05 15:06:08 - Write to log.0 - ------------> Linenr 4------------------------------
2013/12/05 15:06:08 - Write to log.0 - num_input = 00000000000000000000003614115342826.123 Accurate
2013/12/05 15:06:08 - Write to log.0 - str_input = 3614115342826.123
The results don't seem particularly predictable...:confused: I've attached the transformation and input file to illustrate the definition.
If this is a restriction, I think it should be documented somewhere, ironically, if it is left as a string, it inserts OK. I'd rather tie the types down though as it maintains an audit of the different data types being mapped through. I don't like the idea of it changing the content with no warning, it isn't like it is an invalid number as found in other posts, where the parser just gives up.
Any suggestions welcome.
Cheers, Will