How To Build a Web Scrapper
How to build a web scrapper using python and Beautiful Soup
Search emails for any company using a domain name.
Verify email quality and deliverability.
All what minelead has to offer quickly accessible in your browser.
Find a professional email using a full name.
Google sheets, Zoho, Hubspot and much more ...
Generate companies from keywords
Implement all the minelead features into your service.
Unlock Higher Sales Potential with Minelead
Docker is an essential tool for developers as it allows them to quickly and easily create, deploy, and manage applications in a containerized environment. It provides a simple way of packaging applications with all their dependencies into one unit that can be deployed anywhere.
Docker images are the basic building blocks of docker. They consist of a set of layers that contain the application code, libraries, configuration files and other resources needed to run the application. DockerHub is the official repository for docker images which makes it easy for developers to find and use pre-built images from trusted sources.
The importance of docker lies in its ability to simplify development processes by providing developers with an easy way to package their applications into containers that can be deployed anywhere. This makes it easier for developers to collaborate on projects faster as they don’t have to worry about setting up complex environments or dealing with compatibility issues between different systems.
In this blog we will create a python script that verifies email using Minelead API, containerize it in a docker image and run it 24/7 in the background
- Docker and python installed
- Requests package installed
To create a virtual python environement, we need to open our favourite terminal in the folder that we want it to contain the project and execute the command:
python -m venv <name_of_virtual_environnement>
we will name our virtual environement "venv"
it will take couple of seconds to finish, after that we can activate it with the command:
source venv/bin/activate, or for windows users source venv/Scripts/activate
Now that the environnement is ready we can install the packages required for our scripts to run, for thix example we will only need the requests package, so we install it using the command : pip install requests
We need to keep track of all the installed packages so that we can install them on the docker image later, to do that we should write all the package with the version used in a file that we will name requirements.txt, the command for that is:
pip freeze > requirements.txt
To be able to verify email addresses we'll use Minelead Verifier product, for that we need to sign up to and get the api key after that's done, we can start preparing the script :
import requests
api_key = '<api_key>'
emails_verify = ['john@minelead.io','jane@minelead.io']
def main(email):
api_url = f'https://api.minelead.io/v1/validate/?email={email}&key={api_key}'
response = requests.get(api_url).json()
exists = response.get('exist')
print(exists)
for email in emails_verify:
main(email)
This simple script will check if the emails given exists or not and print for us the results.
After confirming that it is working, we can start building the docker image that contain our project
After installing docker, we should be able to use the docker commands on our command line interface.
The command to build a docker image is docker builld [options]
The paramters that we need to give are the path of the Dockerfile and the tag of the image. For that, we need to create the dockerfile that specify the steps to follow in order for docker to build the image
The dockerfile should be named Dockerfile and be in the same directory as the requirements.txt and app.py file, and its content is as follows:
FROM python:3.8-slim-buster
ADD requirements.txt /
RUN pip install -r requirements.txt
ADD app.py /
CMD [ "python", "./app.py" ]
And now we are readt to build the image, we will execute the command:
docker build . -t test/verify:latest
The dot after the build means that the Dockerfile is one the same folder where we are in the CLI.
The -t means that the next paramter is the tag of our image, the name before the slash is our dockerhub username (optional), after the slash is the name of image and name after the colon is the tag of that image
We have our image ready to be ran now, the command to do that is : docker run [options]
The options that we need to specify now are the name of the image that we want to run and the name of the container, so the command will become:
docker run --name verifier test/verify:latest
For testing purposes, the script we wrote only last couple of seconds, but if we have a script that would take longer or need to be running all the time, we need to run the container in the background so that it does not get interrupted, we need to add -d in the command to tell the docker engine to run it in the background. the command becomes now:
docker run --name verifier -d test/verify:latest
To check if the container is running or not. we can list all the containers by executing the command: docker container ls.
If we have a dockerhub account we can login using the command; docker login
Insert the username and password and execute the command: docker push test/verify:latest
After that's done, we can take a look at our dockerhub account and see if we find it there.
If it is there, we can finally pull it from any other machine that has docker installed and expect it to run the same way it did on our machine
Instances of containerized apps use far less memory than virtual machines, they start up and stop more quickly, and they can be packed far more densely on their host hardware. All of this amounts to less spending on IT.
In this blog, we explained what is docker and why it is very important, we wrote a small python script to prepare ourselves for more serious projects,
We are now able to verify a large number of emails and keep the container running 24/7 by running it in the background.