Connector Rules
Connector rules run before Aggregation rules in the aggregation process.
These rules are used to:
- implement pre-processing of data
- implement post-processing of data
- manipulate, merge or otherwise transform the incoming data as it’s being read
Connector Rules include the rule types listed below.
PreIterate
BuildMap
JDBCBuildMap
SAPBuildMap
FileParsingRule
MergeMaps
Transformation
PostIterate
PreIterate Rule : A PreIterate Rule applies only to DelimitedFile and RuleBasedFileParser connectors. If You want to validate the size of the file or if you want to validate that the file (lastModified) is the latest one etc.
- It can be used to execute any processing that should occur prior to iterating through the records in the file. Commonly it is used to configure global data that will be used by the other rules, unzip/move/copy files, or validate the size of the file.
- If the validation fails then it will throw exception and the further aggregation will be stopped.
- PreIterate rules often work in conjunction with PostIterate rules.
- Data is stored into a custom configuration object by the PostIterate rule and read from it by the PreIterate rule on the next aggregation run.
- A PreIterate rule only runs once during an aggregation of a delimited file connector. As a result, this rule generally has a minimal impact on performance.
PostIterate Rule : This rule is called after the connector processes the data. It could also mark statistics in a configuration object that will be used by the PreIterate rule during a subsequent aggregation.
1. This rule can be used to rename the processed file name , renaming in such a way that current date is suffixed to the file name ( To be used in future, for future purpose) .
2. It can also be used to send an email notifications after the aggregation process.
3. PostIterate rule is used to record information that will be accessed and acted upon by the PreIterate rule during the next aggregation for the application.
3. PostIterate rule is used to record information that will be accessed and acted upon by the PreIterate rule during the next aggregation for the application.
BuildMap
JDBCBuildMap
Transformation
Description A BuildMap rule applies only to applications of type DelimitedFile. It is run for each row of data as it is read in from a connector. A BuildMap rule is used to manipulate the raw input data (provided via the rows and columns in the file) and build a map out of the incoming data.
For example:
Columns: Name, ID, Phone
Record: John Doe, 1a3d3f, 555-555-1212
Map: Name, John Doe; ID, 1a3d3f; Phone, 555-555-1212
JDBCBuildMap
Description A JDBCBuildMap rule applies only to applications of type JDBC. It functions for JDBC applications just like the BuildMap rule does for Delimited File applications: it is used by the JDBC connector to create a map representation of the incoming ResultSet. The rule is called for each row of data as it is read in from the JDBC connector. It is used to manipulate the raw input data (provided via the rows and columns) to build a map out of the incoming data.
Description This rule is run for every account or group read from a delimited file or JDBC application. It runs after the BuildMap rule (and MergeMaps, if applicable) and is used to control the transformation of each map into a ResourceObject. Connectors must get data into this ResourceObject format before it can be processed by the aggregator and the aggregation rules.