Let’s say you have a situation where you need to read JSON data from an API and you don’t have plug-in credentials to access the API directly, but you can see the JSON data that is output from the API on a page. In this case, you can still get the data if you “scrape” the output from the page, rather than connecting to the API directly. JavaScript is not very good at scraping web data, but Python is!

Example JSON data on a page:

Python can get this data with two lines of code, using the “requests” module. For example:

jsonData = requests.get(r'https://api.stagingjobshq.com/madgexWidgetJobs.php?callback=displayJobs&city=Moorhead&state=Minnesota', verify=False).text.strip()
return jsonData

JavaScript can use this data returned from Python by simply calling the Python script in a jQuery AJAX get(). For example:

// The name of the Python script that returns the scraped data is scrapeJSON.py
let scrapeJSON = 'http://bluegalaxy.info/cgi/scrapeJSON.py'

$.get(scrapeJSON, function(data) {
   // Get JSON data from Python script
   if (data){
      console.log("Data returned:", data)
   }
   jobDataJSON = JSON.parse(data)
})

Notice the syntax for the jQuery AJAX call is “$.get(nameOfOutsideScript, callback function)”

 

For a real working example of this, see my json-jobs-widget code on github:
https://github.com/chris-relaxing/json-jobs-widget

For more information about AJAX get(), see:
https://api.jquery.com/jQuery.get/