In this article, we will tell you how to scrape culture news from Wired using ScrapeStorm’s “Smart mode“. No Programing Needed. Visual operation.
Introduction to the scraping object
Wired is a monthly American magazine, published in print and online editions, that focuses on how emerging technologies affect culture, the economy, and politics. Owned by Condé Nast, it is headquartered in San Francisco, California, and has been in publication since March/April 1993.Several spin-offs have been launched, including Wired UK, Wired Italia, Wired Japan, and Wired Germany.
Official Website: https://www.wired.com/
Introduction to the scraping tool
ScrapeStorm (www.scrapestorm.com) is an AI-Powered visual Web Scraping Tool，which can be used to extract data from almost any websites without writing any code.
It is powerful and very easy to use. For experienced and inexperienced users, it provides two different scraping modes (Smart Mode and Flowchart Mode).
ScrapeStorm supports Windows，Mac OS and Linux operating systems.
You can save the output data in various formats including Excel, HTML, Txt and CSV. Moreover, you can export data to databases and websites.
title, title_link, Thumbnail, date, summary, author, content
Preview of the scraped result
Export to Excel2007:
1. Download and install ScrapeStorm, then register and log in
(1) Open the ScrapeStorm official website, download and install the latest version.
(2) Click Register/Login to register a new account and then log in to ScrapeStorm.
Tips: You can use this web scraping software directly, you don’t need to register, but the tasks under the anonymous account will be lost when you switch to the registered user, so it is recommended that you use it after registration.
2. Create a task
(1) Copy the URL of Wired
(2) Create a new smart mode task
You can create a new scraping task directly on the software, or you can create a task by importing rules.
3. Configure the scraping rules
(1) Set the fields
Intelligent mode automatically recognizes the fields on the page. You can right-click the field to rename the name, add or delete fields, modify data, and so on.
Add or remove fields as needed, and rename the fields. The results of the field settings are as follows:
(2) Manually set the page
Some web pages have special buttons on the next page, and the system may not recognize them. In this case, you need to manually set the page to “Select Button”.
(3) Scrape into
There is only partial data on the list page, you can use the “scrape into” function to enter the detail page to scrape the data.
Then we add the required fields: content
4. Set up and start the scraping task
(1) Run settings
Once the rules are configured, we can start scraping tasks. Click Start and then jump out of the taskbar. We can set Schedule, Anti-Block, Automatic Export, Download Images and Speed Boost.
Click “Anti-Block” to set the waiting time according to the web page opening speed. The anti-blocking settings follow the system default settings. Then click “Start”.
(2) Start scraping data
Premium Plan and above users can use “Scheduled job” and “Sync to Database”. If you want to download images, you can check “Download images while running”. Then click “Start”.
(3) Wait a moment, you will see the data being scraped.
5. Export and view data
(1) Click “Export” to download your data.
(2) Choose the format to export according to your needs.
ScrapeStorm provides a variety of export methods to export locally, such as excel, csv, html, txt or database. Professional Plan and above users can also post directly to wordpress.