Search and Validate Emails Using Talend and Minelead

 

 

 

 

Minelead and Talend

 

 

 

 

 

Talend is a powerful ETL (Extract transform and load) tool, It provides various software and services for data integration, data management, enterprise application integration, data quality, cloud storage and Big Data.
And Minelead is an open-source Email finder and email verifier tool. It generate leads and find professional email addresses and verify their quality

 

 

Plan:

 

 

1-Iterate over companies' domain names from a csv file
2-make an api call to the Minelead Api to search for companies' emails

3-filter verified emails and put the in a csv file
4-validate every unverified email and put verified emails in the same csv file

 

requirements:

 

1- You need to have Talend 7.3.1.

2- Get an api key from your Minelead api.

 

 

Full architecture:

 

 

 

step-6part2

 

 

1-Read a list of companies from CSV file:

 

 

We start with a csv file containing a list of domain names, to read it in Talend,we should use the tFileInputDelimited components,

 

 

part1-tinputdelimited


We have to specify the the path of the file and the delimiter as shown below:

 

 

part1-json-file-params

 

 

 

2-Iterate over companies and make an API call:

 

 

To iterate over the rows of the input file, we use the tFlowToIterate component 

 

step2-itearate-trest

 

To access the domain name and pass it as a parameter, we should refer to it in the url as (String)globalMap.get("row1.domain"), row1 depends on the name of the link between tFileInputDelimited and the tFlowToIterate components:

 

 

step-2-api-call

 

 

 

3-Extract the Fields:

 

 

We are interested in the 'emails' field from the response that contain the email address and whether it is verified or not, to get the fields we are looking for, these are the parameters we need to declare:


 

step3-extract-json-params

 

 

 

4-Filter the emails:

 

 

The filter operation is based on the verified field, so the settings of the tFilterRow should be as follows:

 

step4-filter-params

 

 

The tFilterRow will give access to two links, Filter and Reject, the first contains the rows that match the condition and last is for the rest,
We need to store the verified emails in a csv file and pass the unverified ones to the next part to validate them,
 

step5-output-delimited

 

To store the results we have to use tFileOutputDelimited component and specify path of the output file, the field seperator and check the append box to not overwrite the content of the file if it already exists

 

 

step5-output-basic-params

 

 

 

5-Validate the unverified emails:

 

 

Now that we filtered the already verified emails, we have to pass the Reject link of the first tFilterRow component to validate the rest of them:

 

step-6part2

 

The same way we iterated over the companies, we'll iterate over the content of the Reject link with tFlowToIterate and pass the rows to the tRest.

 

Notice that the link between the tFilterRow and tFlowToIterate is called row5, that's the variable name that we need to pass in the url:


 

step6-trest2

 

After getting the responses we need, we should now extract the fields we are interested in and filter them and put the verified emails in the same file we used in the first part:
 

We'll be looking for the 'email' and 'exist' fields in every response, so the settings of the tExtractJSONFields should be as follows:
 

step6-extract-part2

 

 

All that's left for us to do, is to filter based on the exist field and store the results, and that's how we will do it:

 

 

The Filter settings:

 

 

step6-tfilter-part2

 

 

 

The tFileOutputDelimited settings:

 

 

Make sure to check the append option to not overwrite the existing content we got from the first step

 

step6-output-part2

 

 

 

Conclusion:

 

 

And that's it, we learned how to use multiple Talend components and make API calls to the Minelead API and store it in a CSV file,
There are multiple useful components on Talend , and Minelead has a lot of good services, you can now do many other opeartions using these tools, like generaing companies from keywords and gettings their emails or simplify verify your list of emails.

 

Searching ...
×
Set a Password
You have created your account using Google SSO. You need to set a password.