Getting ready

The first thing that we will do is add a scraping function to the sojobs module.  This function will be named get_job_listing_skills.  The following is the code for this function:

def get_job_listing_skills(job_listing_id):    print("Got a request for a job listing skills with id: " + job_listing_id)    req = requests.get("https://stackoverflow.com/jobs/" + job_listing_id)    content = req.text    bs = BeautifulSoup(content, "lxml")    script_tag = bs.find("script", {"type": "application/ld+json"})    job_listing_contents = json.loads(script_tag.contents[0])    skills = job_listing_contents['skills']    return json.dumps(skills)

This function retrieves the job listing, extracts the JSON provided by StackOverflow, and then only returns the skills property ...

Get Python Web Scraping Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.