Webscraping for Grocery App
Published:
Wanted to create a grocery app that contained all the groceries from all the major online grocery vendors, allowing users to choose the groceries from each store individually.
For this, a configuration of AWS servers was created that scraped data from each vendor, matched products with slightly different descriptions, and created a unified database.
The nuance was that products differed based on the zipcode.
Several EC2 instances were created, each with a different zipcode, and set them to scrape the data of each vendor from each zipcode.
Whenever an IP address got blocked, the server would automatically delete itself and a new one would be created to continue scraping data.