Python Scraping to CRM Database

Project Details

I have a simple Beautiful Soup scraping script that opens a page, opens links that conform to a wildcard format, and returns the contents of a div on the detail page.

I also have a Google spreadsheet that lists the URLs, and the details above.

I need to modify the script to loop over the values form the sheet, and extract the data., and probably write it to a database for safekeeping.

Then, loop over the text blocks and send to ChatGPT in a particular way.

Get the data back and update my CRM database.

Customize the script to not scrape old data, probably using the URL as a key.