Salesforce Data Replication: In today’s business world, customers/users face difficulties to take back-up or make changes to data in cloud-based applications like Salesforce, NetSuite Solution, etc.  Data needs to be replicated from all objects or a specific object with a full load or partial load based on the business requirements. And if users want to use salesforce objects as look-up objects, then there will be a performance issue.

Challenges to User/Developer

• In IDMC (Formerly known as IICS), (Informatica cloud services) the above challenges can be solved by “Data Replication”.
• The Salesforce add-on connector needs to be enabled in the IDMC (Formerly known as IICS) administrator to create a salesforce connection.
• To create a salesforce connection salesforce User Name, Password, Service URL, and Security token are needed.
• For target objects, either a database connection or flat-file connection is required.

 Solution

Approach-

• Using data replication, the solution for the above-mentioned challenges can be achieved.
• Take an example, some data in salesforce needs to be masked, the data has to be replicated to the database, masked, and loaded to salesforce using Id again.

Technical Solution-

• Create a source connection(salesforce) and target connection (Database or flat file) in IDMC (Formerly known as IICS).

Salesforce Connection:

Database Connection:

• Data Replication:

• Objects to replicate: During each data replication, either replicate all objects or a specific object should be run.

Include objects:

• If an error occurs while processing an object, the remaining objects to be processed can be canceled or it can be continued by enabling the options.
• For target, a connection has to be created as mentioned above, and “load type” has to be selected.
• In ‘incremental load’ the data replication task determines new, changed, and deleted data in the source objects since the last run and propagates the changes to the target objects.
• In partial load, the rows which are created or modified after a specified time are loaded to target objects.
• For the full load data replication task, it drops all the data in the target objects and then performs a full data refresh from the source objects.
• In “field exclusion” some fields from each object are excluded as per requirements.
• Using row limits, no rows to be processed can be selected.
• Using data filters the rows can be further filtered based on a filter expression.
• This job can be run manually or scheduled as needed.

Advantages

•  If the source is salesforce and the target is a database then there is no need to create a target table manually, the target will generate automatically.
• DRS is supper efficient, take an example there are 200 tables in the source and all the tables are required in the target, and a DSS task is chosen then 200  DSS tasks have to be created (if selecting multiple object option is not available)  because generally in DSS one to one mapping is used i.e. one source table/object to one target table/object, also selecting multiple objects in DSS additional complexities have to be added. So, DRS is the best way to go in these scenarios.

Issue

None

Conclusion

With data replication task performance issue of loading a large number of objects from source to target with operations like insert, update (upsert)” can be avoided.

The post Salesforce Data Replication using IDMC appeared first on My Blog.

Informatica Cloud File Processor Connector Several times a user would want to upload and download files from the remote machine to a local machine using FTP/SFTP. As to transferring files from the Protocol system the user needs privileges along with a proper connector to transfer a file from the Informatica platform to another system or vice versa.

Challenges to User/Developer-

• The above problem can be resolved with the help of the Informatica file processor Connector.
• The User needs to create a connection with appropriate credentials to transfer files between two platforms.
• Need to test the connections.

Solution

Informatica Cloud File Processor Connector

Approach-

• You can use File Processor Connector to perform the file transfer. You perform FTP and SFTP transfer to upload or download files from your remote machine to the local machine with File Processor Connector.
• You can do encryption and decryption of files with a File processor. You can also copy, rename, move, compress, decompress, encrypt, archive, and unarchive files with File Processor Connector.
• Informatica Connector for File Processor helps customers in managing the file system.

Technical Solution –

1) Create the Connection (This will end up being the ‘Source Connection’)

• Pick your secure agent. The ‘Source File Directory’ should have a ‘/’ in front of it if you are using a sub-directory.
• The ‘Target File Directory’ is a directory on the same server as the security agent. The ‘Select -File’ drop-down is great because you can specify all kinds of filters on what type of files to download.
• In this screenshot, I chose ‘Select File’. I have left nothing in the File pattern so all kinds of files should be selected.
• Relied more on the ‘Filename’ data filter in ‘Step 4’ below, but it is nice to have options.  Also, put your password in the PassKey1 field so it can be used later.

2) Create another Connection (This will end up being the ‘Target Connection’)

• In the new connection that will be used for the ‘Target’, we need to specify the directory where the files should be saved.
• The ‘Type’ drop-down should be ‘Flat File’ and the ‘Directory’ should be the directory to which the files should be copied.

3) Create the ‘Data Synchronization Task

Step 1 – Definition

Give the task a name and description and then specify it as in the ‘Insert’ job.

Step 2 – Source

In the Connection, drop-down select the ‘File Process Connector’ connection created in#1.
The ‘Source Object’ pick-list should be set to ‘SFTP_GET’.

Step 3 – Target

In the Connection, the drop-down specifies the ‘Flat File’ connector that was created in #2. The Target Object selected should be Infa_SFTP_GET.csv.

Step 4 – Data Filters

• We need to Select to process all rows to get complete data from the source.
• As filters are the certain conditions depend on that the connection enters to SFTP server with respective Hostname, user name, and password.

Step 5 – Field Mapping

• The field mapping should be done for you. The Source and the Target fields are the same.
• After the field mapping the schedule options will come to set and appropriate timing for the job to run, we can schedule or can run Manually.

Advantage –

The main Advantage of this process is that the random data can be shared from A FTP location to some other location. As SFTP is a very protective way to transfer data from one location to another.

Issues –

None

Conclusion:

Informatica Cloud File Processor Connector The file will be moved from the SFTP location to the target directory with a secure protocol.

The post Informatica Cloud File Processor Connector appeared first on My Blog.

Randomization In Informatica Cloud: Customers/Users face issues in picking multiple CSV files with having the same base name from a directory and generates multiple Excel files for each CSV file. The Excel files need to be generated based upon either percentage of available filtered records or an exact number of records and maximum records allowed in a single excel file. The filtration of records is based upon the specific column and its specific value.

Challenges to User/Developer

Randomization In Informatica Cloud :

• In IDMC (Formerly known as IICS), the above-mentioned problem can be solved through Java Transformation.
• The Excel files need to be generated based upon either percentage of available filtered records or an exact number of records and maximum records allowed in a single excel file. The filtration of records is based upon the specific column and its specific value.

• Move the original CSV file to another location.

Solution

Randomization In Informatica Cloud (IDMC)

Approach –

Randomization In Informatica Cloud: Using Java Transformation, we can achieve the solution for the above-mentioned challenges.

Java transformation is used to add custom integration logic into mappings using a built-in developer-friendly editor or by plugging in custom JAR files.

 Technical Solution –

• Develop Java code that performs CSV to Excel split upon either percentage of available filtered records or an exact number of records and maximum records allowed in a single excel file. The filtration of records is based upon the specific column and its specific value using IDE (ex. Eclipse)
• Create an Executable jar file for the above-developed java code.
• Place the above-created jar file at any directory at the local machine (where the security agent is installed).
• Copy the above directory path including the jar name with extension.
• Create the CLASSPATH Environment variable if not available.
• Paste the above-copied directory to the CLASSPATH Environmental variable.

• Restart the secure agent.
• Create the mapping using source and target with java transformation in between them.

• Edit java transformation to make a call to the specific method of the class available in a jar which is responsible for splitting the CSV to Excel split upon either percentage of available filtered records or an exact number of records and maximum records allowed in a single excel file. The filtration of records is based upon the specific column and its specific value.

• Compile the above java code, validate the mapping and run.

Advantage –

The CSV file will get split into Excel based upon either percentage of available filtered records or an exact number of records and maximum records allowed in a single excel file. The filtration of records is based upon the specific column and its specific value

Issues –

None

Conclusion

The CSV file will split into excel based on certain conditions.

The post Randomization In Informatica Cloud (IDMC) appeared first on My Blog.

Easily split your CSV In Informatica cloud: Customers/Users face issues in splitting a CSV file into multiple CSV based upon some specific column and its specific value of that column.

Challenges to User/Developer

Easily split your CSV files in IICS

1. In IICS, the above-mentioned problem can be solved through Java Transformation.
2. The CSV file needs to be split into multiple CSV files.
3. Move the original CSV file to another location.

Solution

CSV Splitting In Informatica Intelligent Cloud Services (IICS)

Approach

Easily split your CSV Using Java Transformation, we can achieve the solution for the above-mentioned challenges.

Java transformation is used to add custom integration logic into mappings using a built-in developer-friendly editor or by plugging in custom JAR files.

Technical Solution –

• Develop Java code that performs CSV split based upon some specific column and specific value of that column using IDE(ex. Eclipse)
• Create an Executable jar file for the above-developed java code.
• Place the above-created jar file at any directory at the local machine (where the security agent is installed).
• Copy the above directory path including the jar name with extension.
• Create the CLASSPATH Environment variable if not available.
• Paste the above-copied directory to the CLASSPATH Environmental variable.

• Restart the secure agent.
• Create the mapping using source and target with java transformation in between them.

• Edit java transformation to make a call to the specific method of the class available in a jar which is responsible for splitting the CSV file into multiple CSV files based upon some specific column and column value.

splitting the CSV file into multiple CSV file

• Compile the above java code, validate the mapping and run.

Advantage –

The CSV file will get split into multiple CSV files based upon some specific column and specific value of that column.

Issues –

None

Conclusion –

So, the CSV file will be split based on the column provided by the user.

The post CSV Splitting In Informatica cloud platform (IICS) appeared first on My Blog.

Convert Excel to CSV  Customers/Users face issues with loading Excel files from SFTP remote server to local machine and convert the excel file to CSV file in mapping on IICS.

Challenges to User/Developer

• In IICS, the above-mentioned problem can be solved through Java Transformation.
• The Excel file needs to be transferred to the local machine (where the security agent is installed).
• The locally available Excel file needs to be converted to a CSV file.
• Move the original Excel file which we got from SFTP remote server to another location.

Solution: 

Approach –

Excel to CSV using SFTP In Informatica Intelligent Cloud Services (IICS)

Convert Using Java Transformation, we can achieve the solution for the above-mentioned challenges.

Java transformation is used to add custom integration logic into mappings using a built-in developer-friendly editor or by plugging in custom JAR files.

 Technical Solution –

• Develop Java code that performs Excel files transfer from SFTP to Local machine and converts Excel file to CSV file using IDE (ex. Eclipse)
• Create an Executable jar file for the above-developed java code.
• Place the above-created jar file at any directory at the local machine (where the security agent is installed).
• Copy the above directory path including the jar name with extension.
• Create the CLASSPATH Environment variable if not available.
• Paste the above-copied directory to the CLASSPATH Environmental variable.

• Restart the secure agent.
• Create the mapping using source and target with java transformation in between them.

• Edit java transformation to make a call to the specific method of the class available in a jar which is responsible for transferring the file through SFTP to Local machine and converts Excel file to CSV file.

• Compile the above java code, validate the mapping and run.

Advantage –

The file will get a transfer from the SFTP directory to the Local machine directory and convert from an Excel file to a CSV file.

Issues –

None

Conclusion

The excel file will now be transferred to the local machine and will be converted accordingly.

The post Excel to CSV using SFTP In IICS appeared first on My Blog.