sw-jobs-go/readme.md

IT jobs scraper

This is a simple scraper that extracts job details from the The Hub website and itjobbank.

Filtering

The scraper filters out a list of keywords like senior, architect etc. as I wrote it for entry and mid level roles. It also filters out unpaid form the hub, and keyword based from itjobbank.

Usage

To run the scraper, simply execute the following command:

go run scraper.go

The scraper will create a thehub.json and itjobbank.json file in the current directory, which contains a list of job details in JSON format. It caches the pages, so very light on resources and requests.