In this article, we will tell you how to scrape technical articles from Medium using ScrapeStorm’s “Smart mode“. Introduction to the scraping tool ScrapeStorm is a new generation of Web Scraping Tool based on artificial intelligence technology. It is the first scraper to support both Windows, Mac and Linux operating systems. Introduction of scraping objects Medium is an online publishing platform developed by Evan Williams, and launched in August 2012. It is owned by A Medium Corporation. The platform is an example of social journalism, having a hybrid collection of amateur and professional people and publications, or exclusive blogs or publishers on Medium, and is regularly regarded as a blog host. Official Website: https://medium.com/ Scraping fields title, title_link, abstract, publisher, claps, labels Function point directory How to manually set the page How to extract the list page plus the detail page Preview of the scraped result Export to Excel2007: Let’s take a closer look at how to scrape technical articles from Medium. The specific steps are as follows: 1. Download and install ScrapeStorm, then register and log in (1) Open the ScrapeStorm official website, download and install the latest version. (2) Click Register/Login to register a new account and then log in to ScrapeStorm. Tips: You can use […]
Scrape website: Website: https://www.imdb.com/ Steps: 1. open web browser, enter www.imdb.com, find a movie you want to scrape its user reviews. 2. click reviews to open the reviews page, copy the url. Open ScrapeStorm, create task with smart mode, paste the url and click “create” button. 3. Just wait a few of seconds, ScrapeStorm can auto distinguish list info. We just need to delete some useless fields, and rename some fields. 4. Finally, we need to setup next page. Click ‘No Page’ -> ‘Manually Select’ -> ‘Select Page Button’, and then find “Load More” in the web page, click it. 5. To here, we have done. Just save and start to scrape.