Quantcast
Channel: Pentaho Community Forums
Viewing all articles
Browse latest Browse all 16689

Large numbers for dimensional values creates problems due to rounding up

$
0
0
Using PostgreSQL, I get wrong result due to large numeric values on dimensions being rounded up. It seems that Mondrian (also internally) formats the number as scientific E notation, but with too low precision.

I create and populate the following table:
Code:

create table public.test (foo bigint, bar int);
insert into public.test values (9123372036854775807,1);

Next I make the following schema:
Code:

<Schema name="Test">
  <Dimension name="Foo">
    <Hierarchy hasAll="true">
      <Table name="test" schema="public"/>
      <Level name="Foo" column="foo" type="Numeric">
      </Level>
    </Hierarchy>
  </Dimension>
  <Cube name="Test">
    <Table name="test" schema="public"/>
    <DimensionUsage name="Foo" source="Foo"/>
    <Measure name="Bar" column="bar" aggregator="sum"/>
  </Cube>
</Schema>

Then I run the following MDX:
Code:

SELECT
[Foo].[Foo].MEMBERS ON COLUMNS
,[Measures].[Bar] ON ROWS
FROM [Test]

The measure is empy, not 1 as expected:
[Foo].[9.1233720368547758E18]
[Measures].[Bar]

Looking at the log, it seems that Mondrian fetches the dimension value (9123372036854775807), rounds it up to 9.1233720368547758E18, and then retrieves all rows where foo equals the rounded up value, not the exact value. Therefore no rows are found.

Viewing all articles
Browse latest Browse all 16689

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>