In this article, we will tell you how to scrape company information from Monster using ScrapeStorm’s “Smart mode“.
Introduction to the scraping tool
ScrapeStorm is a new generation of Web Scraping Tool based on artificial intelligence technology. It is the first scraper to support both Windows, Mac and Linux operating systems.
Introduction to the scraping object
Monster is the world’s largest professional recruitment website and the world’s largest recruitment service provider. Its long-term visits are within 30, providing a full range of recruitment agents, online recruitment, headhunting services and recruitment of yellow pages. Global recruitment solutions.
title, title_link, image, website, headquarter, company size, industry, description
Function point directory
Preview of the scraped result
Export to Excel2007:
1. Download and install ScrapeStorm, then register and log in
(1) Open the ScrapeStorm official website, download and install the latest version.
(2) Click Register/Login to register a new account and then log in to ScrapeStorm.
2. Create a task
(1) Copy the URL
(2) Create a new smart mode task
You can create a new scraping task directly on the software, or you can create a task by importing rules.
Click here to learn how to import and export scraping rule.
3. Configure the scraping rules
(1) Set the fields
Intelligent mode automatically recognizes the fields on the page. You can right-click the field to rename the name, add or delete fields, modify data, and so on.
Click here to learn how to how to configure the extracted field.
(2) Manually set the page
Some web pages have special buttons on the next page, and the system may not recognize them. In this case, you need to manually set the page to “Select Page Button”, then select the button for “NEXT” in the page.
Click here to learn how to manually select the page.
(3) Use the “Scrape into” feature to scrape the detail page data
There is only partial data on the list page, you can use the “scrape into” function to enter the detail page to scrape the data.
Click here to learn how to extract the list page plus the detail page.
4. Set up and start the scraping task
(1) Running and Anti-block settings
Click “Setting”, set waiting time based on web page open speed. You can check “Block Images” and “Block Ads”. The anti-block settings follow the system default settings. Then click “Save”.
Click here to learn more about how to configure the scraping task.
P.S. “Block Images” will reduce the load time and speed up the scraping process. And this operation does not affect the scraping and downloading of images.
(2) Start scraping data
Premium Plan and above users can use “Scheduled job” and “Sync to Database”. If you want to download images, you can check “Download images while running”. Then click “Start”.
Click here to learn about scheduled job.
Click here to learn about sync to database.
Click here to learn about download images.
(2) Wait a moment, you will see the data being scraped.
5. Export and view the data
ScrapeStorm provides a variety of export methods to export locally, such as excel, csv, html, txt or database. Professional Plan and above users can also post directly to wordpress.
Click here to learn more about how to view the extraction results and clear the extracted data.
Click here to learn more about how to export the result of extraction.