Simple Web Scraping Colly App with Fiber
This is a basic Go application using the Fiber framework to create scraping tasks in colly.
How to Run
- Clone the repository.
- Navigate to the project directory.
- Run
docker compose up --build. - Visit
http://127.0.0.1:3000/api/healthcheckerin a web browser or use a tool likecurlto test it. - Send
GETrequest tohttp://127.0.0.1:3000/scrape/courserato start scraping Coursera courses. Andhttp://127.0.0.1:3000/scrape/quotesto scrapequotes.toscrape.com.
What It Does
- Scrapes data from websites and stores in PostgreSQL database.