Kettle Load txt or csv file skip bad rows

July 7, 2015, 7:38 am

≫ Next: BI Server Development ... Build Process, IDE, Debugging ?

≪ Previous: Insert/Update and Synchronize after merge

$

0

0

Hi,
Recently i have a project to use kettle as ETL tool, the problem can't put bad rows into bad files as sqlldr came out. I have read a book but there is no solution. Should i develop java or js code to fullfill my requirement?


Thanks
Jax

---

BI Server Development ... Build Process, IDE, Debugging ?

July 7, 2015, 10:10 am

≫ Next: Pentaho CE 5.3 User Console - Assign created User Role to new User

≪ Previous: Kettle Load txt or csv file skip bad rows

$

0

0

Hi,

I've been trying to set up the BI server source code for development because I wanted to do some experimentation with it.
But up to now I only was able to get a working build by copying what the CI build server does.

I found several pages of documentation that explain build steps but most of them might be outdated maybe (some of them are pre-2013).

My main questions are:

1. What IDE should I use if I want to compile and debug the BI Server platform ? Which IDE do Pentaho developers usually use ?
2. What are the necessary steps to prepare the development environment and perform a successful build ?
3. do I need to install Tomcat myself or is it handled by the build process, what do I need to do to debug the BI server in say IntelliJ IDEA ?

Any help in answering those questions would be much appreciated.

Thanks :)

Pentaho CE 5.3 User Console - Assign created User Role to new User

July 7, 2015, 11:08 am

≫ Next: Modifying CSV data based on conditions

≪ Previous: BI Server Development ... Build Process, IDE, Debugging ?

$

0

0

hello community,

I have pentaho 5.3 CE running for a while and now I am adding new userrole to existing user. I cannot seem to assign any roles from Available to Selected for a selected user. (Browser debugger Dashboards.session.userrole array contains 'Authenticated' only)

(this is under "Administration -> Users/Roles -> Manage Users")

Has anyone else run into this problem and know a work-around ? I did not find any specific wording on CE limitation so if this is the case plz confirm. Thanks!!


Ran

Modifying CSV data based on conditions

July 7, 2015, 11:36 am

≫ Next: Possible bug with tooltips and stacked bar charts.

≪ Previous: Pentaho CE 5.3 User Console - Assign created User Role to new User

$

0

0

Hello! I'm still very new to the PDI so looking for a little help. I've looked around the forum but haven't found exactly what i'm looking for.

What i want to do is read in data from a delimited file and based on certain conditions change the data. For example, one record has a ValueName of "XYZ" and QtyNumber of 2, i want to change that QtyNumber to 0 and leave ValueName = "XYZ"

I've looked at the SetfieldValue, SetFieldValue to Constant, Select Values, but can't see where I can conditionally set one field based on another.


I also looked at the Java Script Value, but can't figure out how to get it work the way I want. I haven't done any programming in a long time so I'm not sure where to go. I know the logic i want to execute, but can't figure out how to implement it.

something like this

Read in file DataFile.csv
for each line;
If ValueName = "XYZ"
then
update QtyNumber = 0
end if

Possible bug with tooltips and stacked bar charts.

July 7, 2015, 6:39 pm

≫ Next: CDA and biserver-ce

≪ Previous: Modifying CSV data based on conditions

$

0

0

Hello,

I think I have found a bug. The scene is the following:

- Biserver 5.3
- A stacked bar chart with several bars (category).
- The element (series) on the top (the first element) only has value in some of the categories.
- Above (and very near to) the bars in the which the element has no value, it's displayed the tooltip of this element.

Is it really a bug, or there is something wrong in my code?

Thank you.

CDA and biserver-ce

July 8, 2015, 5:42 am

≫ Next: Parse xml data to date field

≪ Previous: Possible bug with tooltips and stacked bar charts.

$

0

0

Hello,
I have used kettle for a while and have a test setup that use Carte to execute the Jobs and/or transformations. While this works fairly well in delivering data and report results; I thought I would check out the capability of the biserver-ce and have now run into many roadblocks. I stumbled on the http://www.webdetails.pt/ctools/cda/...-api-reference and thought I had found some detail on what I was looking for. However; now I am again at the same roadblock as with most other avenues of exploration I have tried for the past week; when using the sample API localhost:8080/pentaho/plugin/cda/api/doQuery?path=public/plugin-samples/cda/cdafiles/compoundJoin.cda&dataAccessId=2&paramstatus=Shipped I get a 404 error.

So I am left with a couple of simple questions; does CDA work with the community edition of the biserver as stated in the reference content and where could I find a definitive feature/function document comparing the community editions and the enterprise edition.

I appreciate any feedback you may be able to provide.

Thanks

John

Parse xml data to date field

July 7, 2015, 3:30 pm

≫ Next: Pentaho Aggregation Designer 5.4 Error

≪ Previous: CDA and biserver-ce

$

0

0

When I process a date from a xml file I use the "get data from xml" step.
I create a date field with the format yyyy-MM-dd'T'HH:mm:ss to get the following value 2014-10-19T00:00:00.


But when I want to get the following value 2014-10-19T01:00:00 and use the same format I get an error:

Code:

---

2015/07/07 19:26:04 - Get data from XML.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : org.pentaho.di.core.exception.KettleValueException:
2015/07/07 19:26:04 - Get data from XML.0 - OnDutySince String : couldn't convert string [2014-10-19T00:00:00] to a date using format [yyyy-MM-dd'T'HH:mm:ss] on offset location 19
2015/07/07 19:26:04 - Get data from XML.0 - Unparseable date: "2014-10-19T00:00:00"
org.pentaho.di.core.exception.KettleException:
Unable to read row from XML file

---

That only happens with this specific date. If I change the day or add at least 1 hour it works.


Does anyone know how to deal with this kind of date?

Pentaho Aggregation Designer 5.4 Error

July 7, 2015, 9:19 pm

≫ Next: Misbehavior CCC Line chart component and a Date range input component?

≪ Previous: Parse xml data to date field

$

0

0

Hi, i'm trying to use Pentaho Aggregation Designer to generate some aggregate tables for one cube but there are some errors when starting .bat file and when i try to load schema xml it doesn find any table.

Logs are:

2015-07-07 09:20:03,261 INFO [org.apache.commons.vfs.impl.StandardFileSystemManager] Using "C:\Users\franco\AppData\Local\Temp\vfs_cache" as temporary files store.
2015-07-07 09:20:03,488 INFO [org.springframework.context.support.ClassPathXmlApplicationContext] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@6767ca0f: display name [org.springframework.context.support.ClassPathXmlApplicationContext@6767ca0f]; startup date [Tue Jul 07 09:20:03 COT 2015]; root of context hierarchy
2015-07-07 09:20:03,587 INFO [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] Loading XML bean definitions from class path resource [applicationContext.xml]
2015-07-07 09:20:03,981 INFO [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] Loading XML bean definitions from class path resource [plugins.xml]
2015-07-07 09:20:04,011 INFO [org.springframework.beans.factory.support.DefaultListableBeanFactory] Overriding bean definition for bean 'uiConfig': replacing [Generic bean: class [org.pentaho.aggdes.ui.UIConfiguration]; scope=singleton; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [applicationContext.xml]] with [Generic bean: class [org.pentaho.aggdes.ui.UIConfiguration]; scope=singleton; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [plugins.xml]]
2015-07-07 09:20:04,012 INFO [org.springframework.context.support.ClassPathXmlApplicationContext] Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@6767ca0f]: org.springframework.beans.factory.support.DefaultListableBeanFactory@4875e6ce
2015-07-07 09:20:04,117 INFO [org.springframework.beans.factory.config.PropertyOverrideConfigurer] Loading properties file from class path resource [aggdes.properties]
2015-07-07 09:20:04,118 WARN [org.springframework.beans.factory.config.PropertyOverrideConfigurer] Could not load properties from class path resource [aggdes.properties]: class path resource [aggdes.properties] cannot be opened because it does not exist
2015-07-07 09:20:04,164 INFO [org.springframework.beans.factory.support.DefaultListableBeanFactory] Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4875e6ce: defining beans [org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,xulLoader,xulRunner,uiMain,uiConfig,validatorList,algorithmRunner,outputService,outputUIService,serializationService,org.springframework.beans.factory.config.PropertyOverrideConfigurer#0,aggregateNamingService,org.pentaho.ui.xul.swing.SwingBindingFactory#0,connectionModel,aggModel,aggregateSummaryModel,aggList,org.pentaho.aggdes.ui.form.controller.MainController#0,aggListController,aggController,dataHandler,org.pentaho.ui.database.event.FragmentHandler#0,connectionController,algorithmController,ddlExecCallbackService,exportHandler,org.pentaho.aggdes.ui.form.controller.StatusController#0,publishController,workspace,mondrianSchemaLoader,defaultSchemaGenerator,fileSchemaProvider,algorithm,algorithmUiExtension,defaultOutputFactory,defaultCreateGenerator,defaultPopulateGenerator]; root of factory hierarchy
2015-07-07 09:20:06,645 INFO [org.pentaho.aggdes.ui.form.controller.AlgorithmController] Adding algorithm overlay: org.pentaho.aggdes.ui.ext.impl.AdaptiveMonteCarloAlgorithmUIPlugin@676e582a
2015-07-07 09:20:18,364 ERROR [org.pentaho.ui.xul.binding.DefaultBinding] Binding Error! Update to XulComponenet (org.pentaho.ui.xul.swing.tags.SwingLabel@736cc380,value) outside of event thread!
2015-07-07 09:20:28,176 WARN [mondrian.olap.MondrianProperties] mondrian.properties can't be found under 'C:\Pentaho\pad-ce-5.4.0.1-130\pentaho-aggdesigner-ui\.' or classloader
2015-07-07 09:20:28,176 INFO [mondrian.olap.MondrianProperties] Mondrian: loaded 0 system properties
2015-07-07 09:20:30,667 ERROR [org.pentaho.aggdes.model.mondrian.validate.DimensionFkValidator] an exception occurred
com.microsoft.sqlserver.jdbc.SQLServerException: El nombre de objeto 'dbo.cb_cxc_hechos' no es válido.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:792)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:689)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:616)
at org.pentaho.aggdes.model.mondrian.validate.DimensionFkValidator.validateCube(DimensionFkValidator.java:150)
at org.pentaho.aggdes.model.mondrian.validate.MondrianSchemaValidatorManager.validateCube(MondrianSchemaValidatorManager.java:44)
at org.pentaho.aggdes.model.mondrian.validate.ValidationHelper.validateCube(ValidationHelper.java:62)
at org.pentaho.aggdes.model.mondrian.MondrianSchemaLoader.validateSchema(MondrianSchemaLoader.java:110)
at org.pentaho.aggdes.ui.ext.impl.AbstractMondrianSchemaProvider$1.run(AbstractMondrianSchemaProvider.java:150)
2015-07-07 09:20:34,403 INFO [mondrian.rolap.RolapUtil] Mondrian: JDBC driver com.microsoft.sqlserver.jdbc.SQLServerDriver loaded successfully
2015-07-07 09:20:34,404 INFO [mondrian.rolap.RolapUtil] Mondrian: JDBC driver sun.jdbc.odbc.JdbcOdbcDriver loaded successfully
2015-07-07 09:20:34,404 WARN [mondrian.rolap.RolapUtil] Mondrian: Warning: JDBC driver org.hsqldb.jdbcDriver not found
2015-07-07 09:20:34,405 WARN [mondrian.rolap.RolapUtil] Mondrian: Warning: JDBC driver oracle.jdbc.OracleDriver not found
2015-07-07 09:20:34,405 WARN [mondrian.rolap.RolapUtil] Mondrian: Warning: JDBC driver com.mysql.jdbc.Driver not found
2015-07-07 09:20:34,434 INFO [org.apache.commons.vfs.impl.StandardFileSystemManager] Using "C:\Users\franco\AppData\Local\Temp\vfs_cache" as temporary files store.
2015-07-07 09:20:35,636 ERROR [org.pentaho.aggdes.ui.form.controller.ConnectionController] Error loading schema:
mondrian.olap.MondrianException: Mondrian Error:Internal error: Reading cardinality for column [null, dbo, cb_cxc_tiempos, y]; sql=[select count(distinct "y") from "dbo"."cb_cxc_tiempos"]
at mondrian.resource.MondrianResource$_Def0.ex(MondrianResource.java:977)
at mondrian.olap.Util.newInternal(Util.java:2404)
at mondrian.olap.Util.newError(Util.java:2420)
at mondrian.rolap.SqlStatement.handle(SqlStatement.java:361)
at mondrian.rolap.SqlStatement.execute(SqlStatement.java:252)
at mondrian.rolap.RolapUtil.executeQuery(RolapUtil.java:350)
at mondrian.rolap.RolapUtil.executeQuery(RolapUtil.java:303)
at mondrian.spi.impl.SqlStatisticsProvider.getColumnCardinality(SqlStatisticsProvider.java:117)
at mondrian.rolap.RolapStatisticsCache.getColumnCardinality(RolapStatisticsCache.java:171)
at mondrian.rolap.RolapStatisticsCache.getColumnCardinality(RolapStatisticsCache.java:137)
at mondrian.rolap.RolapStar$Column.getCardinality(RolapStar.java:1023)
at org.pentaho.aggdes.model.mondrian.MondrianSchema.doTable(MondrianSchema.java:151)
at org.pentaho.aggdes.model.mondrian.MondrianSchema.doTable(MondrianSchema.java:190)
at org.pentaho.aggdes.model.mondrian.MondrianSchema.<init>(MondrianSchema.java:78)
at org.pentaho.aggdes.model.mondrian.MondrianSchemaLoader.createSchema(MondrianSchemaLoader.java:68)
at org.pentaho.aggdes.ui.ext.impl.AbstractMondrianSchemaProvider.loadSchema(AbstractMondrianSchemaProvider.java:198)
at org.pentaho.aggdes.ui.form.controller.ConnectionController$4.run(ConnectionController.java:264)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: El nombre de objeto 'dbo.cb_cxc_tiempos' no es válido.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:792)
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:689)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:616)
at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:205)
at mondrian.rolap.SqlStatement.execute(SqlStatement.java:199)
... 12 more
2015-07-07 09:20:35,660 ERROR [org.pentaho.aggdes.ui.form.controller.ConnectionController] Unable to connect
org.pentaho.aggdes.AggDesignerException: Error loading Schema
at org.pentaho.aggdes.ui.form.controller.ConnectionController.connect(ConnectionController.java:276)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.swing.tags.SwingButton$OnClickRunnable.run(SwingButton.java:71)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$200(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.WaitDispatchSupport$2.run(Unknown Source)
at java.awt.WaitDispatchSupport$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.WaitDispatchSupport.enter(Unknown Source)
at java.awt.Dialog.show(Unknown Source)
at java.awt.Component.show(Unknown Source)
at java.awt.Component.setVisible(Unknown Source)
at java.awt.Window.setVisible(Unknown Source)
at java.awt.Dialog.setVisible(Unknown Source)
at org.pentaho.ui.xul.swing.tags.SwingDialog.show(SwingDialog.java:250)
at org.pentaho.aggdes.ui.form.controller.ConnectionController.showConnectionDialog(ConnectionController.java:314)
at org.pentaho.aggdes.ui.form.controller.ConnectionController$3.run(ConnectionController.java:225)
at java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$200(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)

---

Misbehavior CCC Line chart component and a Date range input component?

July 8, 2015, 1:31 am

≫ Next: Pass data to excel according to a criteria

≪ Previous: Pentaho Aggregation Designer 5.4 Error

$

0

0

Hello everyone,

I'm new with Pentaho CDE, and I'm facing a problem with a CCC Line chart component and a Date range input component.
I'll try to explain it:

The following MDX query:

select Non Empty({Descendants([Indicad.Indicador].[All Indicad.Indicadors] ,[Indicad.Indicador].[Indicador])}) on Columns,
Non Empty ({[Datad.Extra Info dates].[Data].[${fromDate}]:[Datad.Extra Info dates].[Data].[${toDate}]}) on ROWS
from [indica20150610]
WHERE ({[Aread.Area].[COR_CORP]})

works perfectly, only if I have data on both ends of date range, in any other case the component shows "No data found".

Please, could anyone explain why?
Can I avoid this "No data found" messages, and plot the data available in the rest of range dates selected by the user?

Thank You!
Best regards.

Cesc

Pass data to excel according to a criteria

July 8, 2015, 3:02 am

≫ Next: Enrich a row with multiple fields with data from a SQL query in PID

≪ Previous: Misbehavior CCC Line chart component and a Date range input component?

$

0

0

Best regs,
I searched on google but not found anything similar, I need the help.
I have excel file that contain data to copy for another excel file following certain criteria
From source workbook copy range A to target workbook in range A beginning at A11 and for other following cell increasing 5:
source workbook target workbook
A1 A11
A2 A11+5
A3 A16+5
A4 A21+5
A5 A26+5
A6 A31+5
... ...
And copy
From source workbook copy range F to target workbook in range G beginning at G12 and for other following cell increasing 5:
source workbook target workbook
F1 G12
F2 G12+5
F3 G17+5
F4 G22+5
F5 G27+5
F6 G32+5
... ...
And copy
From source workbook copy range F to target workbook in range G beginning at G14 and for other following cell increasing 5:
source workbook target workbook
G1 G14
G2 G14+5
G3 G19+5
G4 G24+5
G5 G29+5
G6 G34+5
... ...

Thanks for your precious time

Enrich a row with multiple fields with data from a SQL query in PID

July 8, 2015, 4:55 am

≫ Next: A new user needs your help - My query don't work in table input (PDI).

≪ Previous: Pass data to excel according to a criteria

$

0

0

I am trying to enrich a row with a value that i fetch from a database based on a specific field in that row.

in doing so, I pass my rows to a table input transformation step. The query inside the input is rather trivial


Hwever, when I try to launch the transformation I get this error:


2015/07/08 13:31:03 - Table input 3.0 - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Unexpected error
2015/07/08 13:31:03 - Table input 3.0 - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:
2015/07/08 13:31:03 - Table input 3.0 - An error occurred executing SQL:
2015/07/08 13:31:03 - Table input 3.0 - select m.MEMBER_ID from member m where m.MEMBER_EXT_ID = ?
2015/07/08 13:31:03 - Table input 3.0 -
2015/07/08 13:31:03 - Table input 3.0 - offending row : [EMAIL String], [MY_ID Integer(15)<binary-string>]
2015/07/08 13:31:03 - Table input 3.0 -
2015/07/08 13:31:03 - Table input 3.0 - Error setting value #2 [Integer(15)<binary-string>] on prepared statement
2015/07/08 13:31:03 - Table input 3.0 - Parameter index out of range (2 > number of parameters, which is 1).
2015/07/08 13:31:03 - Table input 3.0 -
2015/07/08 13:31:03 - Table input 3.0 -
2015/07/08 13:31:03 - Table input 3.0 -
2015/07/08 13:31:03 - Table input 3.0 - at org.pentaho.di.core.database.Database.openQuery(Database.java:1731)

here is the setup of my table input.
The incoming row data contains two fields and my query only cares about one of these to work. Is there a way to chose what field should be used when replacing variables in the SQL script? My problem is that I need both those fields and append the result of the query to the row...

off course, if I insert a Select Values step before my Table Input where I remove the extra field, then my query run smooth... how can I keep all my fields and enrich my row with a SQL result?
Thanks in advance!!



New Bitmap Image.jpg

Attached Images

• New Bitmap Image.jpg (20.6 KB)

A new user needs your help - My query don't work in table input (PDI).

July 8, 2015, 6:51 am

≫ Next: how to properly define output row metadata for mapping a multi-output sub-transform?

≪ Previous: Enrich a row with multiple fields with data from a SQL query in PID

$

0

0

Hi can anyone help me to solve this issue.
I have a query that runs on SQL SERVER MANAGEMENT, it was created there, but when I try to run in a transformation it isn't work.
I took a look at log and its massage was " the statement did not returned a resultset ".It is something related to JDBC.
I'm studying PDI and I can't solve it.
I think there is another way to create the transformation to run this query.

The Query is:

Code:

---

DECLARE @v_tbl_1 TABLE (    DIRETORIO_ID                bigint
    ,TESTSET_ID                    bigint
    ,CT_ID                        bigint
    ,STATUS_EXECUCAO_CT            VARCHAR(MAX)
    ,DATA_EXECUCAO                DATE
    ,HORA_EXECUCAO                TIME   
    ,DDMMYY                        INT
    )


DECLARE @v_tbl_2 TABLE (
    DIRETORIO_ID                bigint
    ,TESTSET_ID                    bigint
    ,CT_ID                        bigint
    ,STATUS_EXECUCAO_CT            VARCHAR(MAX)
    ,DATA_EXECUCAO                DATE
    ,HORA_EXECUCAO                TIME   
    ,DDMMYY                        INT
    )


DECLARE @v_tbl_final TABLE (
    DIRETORIO_ID                bigint
    ,TESTSET_ID                    bigint
    ,CT_ID                        bigint
    ,STATUS_EXECUCAO_CT            VARCHAR(MAX)
    ,DATA_EXECUCAO                DATE
    ,HORA_EXECUCAO                TIME   
    )


INSERT INTO @v_tbl_1
(
    DIRETORIO_ID,
    TESTSET_ID,
    CT_ID,
    STATUS_EXECUCAO_CT,
    DATA_EXECUCAO,
    HORA_EXECUCAO,
    DDMMYY
)
SELECT
    a.DIRETORIO_ID
    ,a.TESTSET_ID
    ,a.CT_ID
    ,a.STATUS_EXECUCAO_CT
    ,a.DATA_EXECUCAO
    ,a.HORA_EXECUCAO
    ,convert(varchar(10),a.DATA_EXECUCAO,112)
FROM (
    select
        cf.CF_ITEM_ID as DIRETORIO_ID   
        ,c.CY_CYCLE_ID as TESTSET_ID
        ,t.TS_TEST_ID as CT_ID
        ,ap.AP_NEW_VALUE as STATUS_EXECUCAO_CT
        ,max(al.AU_TIME) AS DATA_EXECUCAO
        ,max(al.AU_TIME) AS HORA_EXECUCAO
    FROM
        AUDIT_LOG al (NOLOCK)
    INNER JOIN
        AUDIT_PROPERTIES ap (NOLOCK)    on  al.AU_ACTION_ID = ap.AP_ACTION_ID
    INNER JOIN
        TESTCYCL tc (NOLOCK)            on  al.AU_ENTITY_ID = tc.TC_TESTCYCL_ID
    INNER JOIN
        CYCLE c (NOLOCK)                on  tc.TC_CYCLE_ID = c.CY_CYCLE_ID
    INNER JOIN
        TEST t (NOLOCK)                    on  tc.TC_TEST_ID = t.TS_TEST_ID
    INNER JOIN
        CYCL_FOLD cf (NOLOCK)            on  c.CY_FOLDER_ID = cf.CF_ITEM_ID
    where
        al.AU_ENTITY_TYPE = 'TESTCYCL' and ap.AP_TABLE_NAME = 'TESTCYCL' AND ap.AP_FIELD_NAME = 'TC_STATUS'
    GROUP BY
    cf.CF_ITEM_ID
    ,c.CY_CYCLE_ID
    ,t.TS_TEST_ID
    ,t.TS_NAME
    ,ap.AP_NEW_VALUE) a


INSERT INTO @v_tbl_2
(
        DIRETORIO_ID
        ,TESTSET_ID
        ,CT_ID
        ,DDMMYY
        ,DATA_EXECUCAO
        ,HORA_EXECUCAO
)
SELECT
    b.DIRETORIO_ID
    ,b.TESTSET_ID
    ,b.CT_ID
    ,b.DDMMYY
    ,max(b.DATA_EXECUCAO)
    ,max(b.HORA_EXECUCAO)
FROM
    @v_tbl_1 AS b
GROUP BY
    b.DIRETORIO_ID
    ,b.TESTSET_ID
    ,b.CT_ID
    ,b.DDMMYY;


insert into @v_tbl_final
(
    DIRETORIO_ID
    ,TESTSET_ID
    ,CT_ID
    ,STATUS_EXECUCAO_CT
    ,DATA_EXECUCAO
    ,HORA_EXECUCAO
)
SELECT
    a.DIRETORIO_ID
    ,a.TESTSET_ID
    ,a.CT_ID
    ,a.STATUS_EXECUCAO_CT
    ,a.DATA_EXECUCAO
    ,a.HORA_EXECUCAO
FROM
    @v_tbl_1 as a
INNER JOIN   
    @v_tbl_2 as b ON (
                            a.DIRETORIO_ID = b.DIRETORIO_ID
                    AND    a.TESTSET_ID = b.TESTSET_ID
                    AND        a.CT_ID    =    b.CT_ID
                    AND        a.DATA_EXECUCAO = b.DATA_EXECUCAO
                    AND        a.HORA_EXECUCAO = b.HORA_EXECUCAO
                    )


SELECT
        *
    FROM
        @v_tbl_final

---

Yours truly.

how to properly define output row metadata for mapping a multi-output sub-transform?

July 8, 2015, 9:36 am

≫ Next: Extracting exchange Rates from ws

≪ Previous: A new user needs your help - My query don't work in table input (PDI).

$

0

0

I'm using a Mapping (sub-transformation) with 2 outputs. But whenever I try to do anything, it complains:
Unable to find appropriate mapping definition to determine output row metadata.

So... how can I properly map it?

Extracting exchange Rates from ws

July 8, 2015, 11:52 am

≫ Next: Page break after each details iteration

≪ Previous: how to properly define output row metadata for mapping a multi-output sub-transform?

$

0

0

I am trying to load exchange rates from a web service into a Database Table (I am using Amazon Redshift)
I found a service that provides the required data: https://openexchangerates.org/api/hi...46efbfd85b93f5
Tried Using JSON input and HTTP Client, but so far I cant make it work.

Ideas anyone?
Thanks
Noam

Page break after each details iteration

July 8, 2015, 2:00 pm

≫ Next: Test a varable value in IF statement

≪ Previous: Extracting exchange Rates from ws

$

0

0

Hello,

I have soome subreports in the details band and I want to have each iteration in a new page. I have tried to set a page break after the last subreport and in the last element of that subreport, but nothings happens.

How can I do this?

Thank you.

---

Test a varable value in IF statement

July 8, 2015, 2:59 pm

≫ Next: get previous row value in current row

≪ Previous: Page break after each details iteration

$

0

0

Hi,

I have a varible, var. How can i test this variable value in IF statement?

Ex: ${var} = 3
If(${var}=3;"True";"False") -- Is this the correct way to test variables?

Please let me know.

Regards,
Madhu

get previous row value in current row

July 8, 2015, 3:12 pm

≫ Next: Increase Sequece number when the field value changes

≪ Previous: Test a varable value in IF statement

$

0

0

Hi,

How can we get previous row value in current row? I need this for comparision of one of the field values. Please let me know.


Regards,
Madhu

Increase Sequece number when the field value changes

July 8, 2015, 3:28 pm

≫ Next: PDI is Very Slow when trying to acces the repository

≪ Previous: get previous row value in current row

$

0

0

Hi,

I have following data. I have one more column "Sequence". For this column the values would be Increase by 1 when the "Name" field value changes.

S.No	Name
1	Mike
2	Mike
3	Carlos
4	Dave
5	Dave
6	Dave
7	Tim
8	Tim
9	Mark

I need the sequence number for this data as follows like when the "Name" field value change, the Sequence should increase by 1. How can i get this?

S.No	Name	Sequence
1	Mike	1
2	Mike	1
3	Carlos	2
4	Dave	3
5	Dave	3
6	Dave	3
7	Tim	4
8	Tim	4
9	Mark	5

Thanks,
Madhu

PDI is Very Slow when trying to acces the repository

July 8, 2015, 3:41 pm

≫ Next: Usage of Variables

≪ Previous: Increase Sequece number when the field value changes

$

0

0

When trying to access the repository located in the cloud. PDI is very slow. Engine database is Oracle 12C. spoon and version is:Engine database is Oracle 12C. spoon and version is:
Kettle - Spoon General Availability Release 5.2 .0.0Engine database is Oracle 12C. spoon and version is:
Kettle - Spoon General Availability Release 5.2 .0.0

Usage of Variables

July 8, 2015, 4:38 pm

≫ Next: Is it necessary to Initialize Repository for Penatho BI 5.2.0 CE server

≪ Previous: PDI is Very Slow when trying to acces the repository

$

0

0

Hi,

Can we use variables in "Formula Step", which is available in Scripting folder?

Regards,
Madhu