Pycurl Javascript

I created a python 3 script that allows me to search on a search engine (DuckDuckGo), get the HTML source code and write it in a textfile. import pycurl from io import BytesIO buf

Solution 1:

Duckduckgo's html pages uses javascript to load their search result into their html markups, so curl or PyCurl will not be able to get the same html content you'd see in a browser since curl/pycurl merely fetches internet resources but does not provide any javascript processing.

Use https://duckduckgo.com/api instead of scraping to find search results in their servers/databases.