Sunday, May 26, 2013

Pentaho Data Integration notifier job state

This a little sample how create a job notifier of another job is terminated with error or success.
The first step is create a job sample that represents job of ETL process (for example, a job that is responsible to populate DW).

Create a job example

The first step is create a transformation that receive from ${VALUE} variable and if that value match with 'Y' the transformation execute successfully else ${VALUE} have a different value of 'Y' the transformation execute with error.
See the following transformation workflow:

The second step is create job that execute the above transformation. See the following job workflow:

Create the main job

Create a transformation notifier

In this transformation you'll need define how you can be notified. In this sample you would be notified by email, android push notification (using PDI Manager) and apple push notification (is a new plugin, will be available in the few days).
Transformation receive a parameter that indicate if ETL executes with success or not. And basis on that, the message notification alters.
See the following ETL workflow:


Create a job notifier

This job is responsible to execute any job on same folder, what you need is pass by parameter the job name that you want execute. If the job run with success he'll execute above transformation passing 'Success' on parameter STATE.
See the following job workflow:

See the following how execute the job:



You can download all files on this in this link (note: you need configure the kettle.properties file).



5 comments:

  1. First thanks for the app and the PDI plugin. Great work.

    I don't seem to get any content inside PDI Manager. I get the notification on my android phone and the notification has the status and project. However I don't see the Date or Data fields anywhere on the PDI manager and when I click on the notification I just get a blank PDI Manager screen with no issues on it. Am I doing something wrong?

    ReplyDelete
    Replies
    1. Hi,

      Sorry for delay.
      I discover a bug on mapping fields when send push notification. I already fix, I'm just testing android and apple step and I'll launch a new release soon. However, if you want, you can use the new version from github http://goo.gl/PBqy2f.
      The PDI Manager, in the current release, is a proof of concept and my goal is provide a nice Pentaho data integration (aka kettle) android client. So, if you have any idea to improve, please let me know.

      Note: At the moment, are mandatory just using the lower case fields status, data, project and date for send a push. Please, use this sample to test http://goo.gl/f9MFA6

      Thanks.

      Delete
  2. Hello Joel,

    Great app! Unfortunately, I also have the problem of receiving the notification without any data in it. I'm not familiar with github neither with Java compile stuff; would it be possible to have the latest .jar ?
    Thanks!

    ReplyDelete
    Replies
    1. Hi Christian,

      The latest version is published on Pentaho marketplace. But if you're not available to update using spoon, you can download from here: http://sourceforge.net/projects/pentaho-plugins/files/Pentaho%20Data%20Integration/PDI%20Android%20Push%20Notifications/1.0.0/

      The PDI Manager at the moment is a POC version. So, maybe you are receiving empty data because you are not put the exactly data the app need (check here: https://github.com/latinojoel/pdi-android-pushnotifications#pdi-manager-android-app).

      Cheers.

      Delete
    2. Ok then, I do have the latest version. The weird thing is that I'm using your sample. I do receive a notfication on my Android device (I see it in the notifications window with the "Project" defined) but when I open PDI Manager, I don't see anything, neither data nor date; in fact the screen in PDI Manager is empty like I didn't received anything...

      Delete